I have collected a significant number of 1TB 3.5 hard drives and various sized 2.5 hard drives that are still working. I have a large tower with a number of bays empty with an older Intel CPU and 8GB of RAM. My thought is to take this older hardware, create a file server that I would use to backup my home server and all my personal files (I have very little media, so 300GB will probably be enough to hold all of my data at this point.)
What operating system should I go with?
I don’t believe TrueNAS is an option because of the mismatched drives that I have and therefore ZFS is out as well.
I’m thinking Btrfs might work as a file system, but then do I run Btrfs on Fedora server (what runs the current home server in the homelab) or Debian 11 (what runs the server at the office) or OpenSuse because of their mature Btrfs tool set. Or something else? Keep in mind this will probably be headless but doesn’t have to be.
If we can get wakeonlan working on the tower the plan will be for the home server to turn it on once or twice a week, backup the data, then a cron job would shut the file server down again so it is only running to act as a file server for backups.
What to use as the backup software?
Right now we are thinking of just running an rsync script.
Our home server runs Syncthing and is the central point of truth for all of our computers at home and at the office. All of my computers sync with the home server, which has a 1TB drive that is far from full. My son has mostly text programming files, and I have mostly office files that I back up.
Should we just run Syncthing on this file server too? Is that really a backup solution? I’m afraid that if I use Syncthing that it would just spread a ransomware attack onto my backup file server, but perhaps I don’t understand some of the settings of Syncthing.
I think for safety vs ransomware you’d need to make incremental backups of your data to another server, where the backup software can’t delete or overwrite previous backup data. And the other server wouldn’t have any file-sharing going on or anything, and would have its own very restrictive firewalling etc.
For OS, you can try Debian 11 now that it’s their LTS.
Maybe for your disks you can put 4 of the 1TB ones in as RAID 10? I wouldn’t try to put tons of disks cuz it will just waste power and be more things that can go wrong.
For the operating system, go with what you know… or want to learn. Over the years, I have settled on one distro for everything. Fundamentally they will all work about the same.
I use wake-on-lan to turn on my backup server every week at 2 am on Saturday morning. Starting with a simple rsync script is a good way to start. Then you can modify it to meet your needs.
I use syncthing to keep some of my working directories synced to a central point.
To get a bit more specific. My NAS/server is the center of my network. I started backing up all my systems to the NAS once per day using Apps/DejaDup - GNOME Wiki! which is a front end for rsync. The NAS has a folder called backups. In that folder, there are subfolders that have the same name as the hostnames for all the machines I would like to back up. Even if I get rid of a system, I often have the backup folder hang around until I am sure I do not need anything from it.
As my needs got more complicated I switched from dejadup to a custom backup script.
Then once, per week, my NAS/server backs itself up to my remote NAS/server over rsync. The remote NAS lives at my sister’s house. The remote NAS wakes up, sits and waits to be the rsync target for the backup, and then does some housekeeping before shutting down. Using rsync I can choose which folders to remotely back up.
I back up the backups, syncthing, and git-server folders to the remote nas.
How many 1 TB drives do you have? I would actually consider ZFS a good option, no matter what, since you can add any number of vdevs to a zpool.
I would use all your 1 TB drives in a RaidZ (or RaidZ2 if you have 8 drives or more), and then if you have any spare SSD’s (all sizes) use these as cache disks.
If you have any number of disks of other sizes (say like 3 500GB disks), these can be combined into another RaidZ, and added to the same Zpool.
ZFS is VERY flexible when it comes to hardware. The only thing you need to be aware of is that when you combine in Raid, the smallest disk size counts. So if you have 2x 500 GB and 2x 750 GB disks, and combine them in RaidZ, they will all effectively be 500 GB disks.
I think it’s better to keep a couple drives on the shelf to swap in case of a disk failure; since it sounds like these are already older drives. Just use what’s needed.
You’re right, and actually that’s also what I thought, but I didn’t write it. But let’s say you have 10 1TB drives. Then I would use 8 of them in a RaidZ2, and keep 2 as spares, for instance.
For a working file server maybe, or if they’re old drives (1TB are prolly pretty old now) I’d keep more spares even.
But remember that he said they have like <400GB of data, so even with space for incrementals 2TB is plenty, and he can always add more. Also, it’s an older PC so prolly not tons of SATA ports and there’s just 8GB of RAM. I think ZFS likes lots of RAM? So it’s prolly not as good of an option. OTOH, mirroring and striping with mdadm is real easy and has plenty of performance for a backup server.
Since I don’t have it booted or a live USB handy, I’m not sure which A Series AMD CPU is in here.
With the spare sticks I have or DDR3 memory sticks and only two slots I might be limited to 4GB or 6 GB of memory.
I have 5 1TB 3.5 drives. 1 500GB 3.5 drive, 1 250GB 3.5 drive, 2 160GB 3.5 Drives, and I believe I have 1 additional 1TB 2.5 drive.
The tower has 5 3.5 drive bays, and 2 5.25 drive bays.
I’m really thinking this is going to be too limited for TrueNAS which is what I would like to run if I were to go the ZFS route. This is a homelab experiment, that I mostly wanting to do for the following reasons:
Turn some old hardware that is just gathering dust into something useful in the homelab (this would provide an additional backup besides the backups on the external drive hooked up to our homeserver that is the main node in our Syncthing setup). Yes our needs are not great < 400GB currently and not growing that quickly because most files are text or office documents.
Learn to use some new technologies: wakeonlan, file systems, backup technologies for incremental backups, and such.
I’m thinking that I like the idea of our pool being 4 of the 1TB drives, and then using one of the 160 GB drives as the system disk. Since our motherboard only has 4 sata ports, I’m willing to invest in a SATA PCI card that has 4 additional ports (recommendations welcome). I’m thinking we will go with Debian 11 as our OS. I found this article which I thought it might be a good mix of technologies for this project, JBOD (Just a Bunch of Disks) and SnapRAID. Although, I noticed that he used ext4 as the file system, and I’m wondering if Btrfs might be a better choice in 2021. I know Debian allows you to format and set up RAID 10 in Btrfs because I have done it on a Debian 10 box.
I think to get the above up and running I would just need a PCI SATA card and perhaps a few power cables to get enough SATA power connections from the power leads off of the power supply. I have plenty of SATA data cables.
@ameinild thanks for the idea about OpenMediaVault. I think of that as a RasberryPi solution, and forget that it can be installed on x86 hardware and used in a similar way. I’m a little hesitant to go that route because I don’t know if OpenMediaVault gives you access to the terminal and root file system. In order to experiment with wakeonlan we might need access to some terminal applications and be able to edit the /etc/network/interfaces file. Also my son (a programmer) would like to experiment with automation scripts that would run through cron. I will have to look into OpenMediaVault a bit more. It might simplify things for us.
Thanks @dfarning and @Buffy for your thoughts as well. Having read that article I referenced I don’t believe SnapRAID is needed for this if we opt for Btrfs because the filesystem and its tool set can set up the RAID 10 pool and I believe that Btrfs will provide us with bitrot protection as well automatically similar to ZFS from articles I have been reading in the last couple of days. I don’t think I have everything figured out, but I think it is time to buy that 4 port SATA card and power leads and start experimenting. Thanks again.
One thing to consider as you move forward is the idea that a backup server and a home lab server for experimentation have very different goals and requirements.
The backup server should be rock solid. As simple and reliable as possible.
The experimental home lab server changes frequently and tends to be overly complex as you learn new technologies.
At some point, as you understand your needs better, you might want to consider separating these two roles.
in addition to openmedivault, https://xpenology.org/ is also worth researching. Synology provides a lot of good workflows that help you learn to get the most out of a storage sever.
@dfarning, I definitely agree. I see this as a homelab experiment that if it goes well and proves to be dependable, we might turn it into our rock solid backup server. For right now it is going to be another source of backup of our important files until we are able to be absolutely sure that it is dependable and stable. For now we are going to use this to learn about Btrfs and its tool set as well as learn some scripting and automation technology for use in a stable build. If this doesn’t work out, I have some slightly better speced hardware that is just sitting around too, and we might just bite the bullet and turn one of those towers into a TrueNAS box.
As far as I remember (it’s been a while since I’ve used OMV), you can absolutely go to terminal, use sudo and install aps in terminal as root etc. (See this post for reference)
I believe OMV is mostly a configuration layer running on top of Debian. You should be able to do whatever you want with the underlying Debian system, regardless of what OMV offers. Good luck!