I will attempt to make a very long story as short as possible.
Buy refurbished HP ProDesk 600 G1 > Intention: Run pfSense, NAS on top of Proxmox
Install four port Intel NIC
External hard drives a bust > read/write errors, buffer issues etc. mentioned in logs
Buy bits and bobbles to move two additional drives (shucked from external) to SATA > success
Attempt to install pfSense > discover chip/board combo does not support IOMMU > pfSense via Proxmox aborted, budget gone, despair ensues
Attempt to install open source firmware on consumer router to have at least some more control > router not supported
Install TrueNAS Core (beta) > runs well, but insufficient HD space with mirrored drive (two 1 TB drives + OS on SSD), also no ECC
Decide to ask for advice
As I was trying to figure out how to get TrueNAS setup so that I could actually write to the drives (reading was working) I began to wonder if perhaps I need to take a step back and âsimplifyâ things. Perhaps now that pfSense is not going to happen any time this year, and my âseverâ is not particularly well speced for TrueNAS, it would be wise to simply install Ubuntu Server, learn how to set up NFS, and leave it at that. I could then either a) figure out how to install Nextcloud and Plex on the same server or b) install Nextcloud and Plex on my Mac Mini and use the server as the backend.
While I am very down about how things played out (I bought a UPS, and other bits and pieces as well), I would like to find a way to salvage something, and maybe come back to pfSense in the future. I would really appreciate your thoughts about how best to âuse what I haveâ.
Well, you could install a Linux distro on it and still run lots of different services in Docker rather than VMs. Thereâs still lots to learn and use that way.
You can look at Jayâs Ansible series to see how to use the pull configuration; then you can at least have that for learning and for setting your configurations in case you end up wanting/needing to reinstall while youâre experimenting, and as a handy starting point if/when you add to your lab.
TBH, for router/firewall, we decided it was cheaper and more straightforward to get a Protectli one, where you can pick regular BIOS or Corboot, and pfsense or opensense. Theyâre real nice and just work, and you can still learn about either one and theres lots of packages for both pfsense and opensense.
Whatever you do, though, make sure that you have good backups for the data you care about.
That is a great idea. You suggested it earlier but I thought I would buy one machine to do pfSense and NASâŚthat went really well. I may do that when the coffers refill. Right now I need to get some sort of network storage up and running (which is one of the reasons I started this adventure). I could do Docker or even LXC/LXD I suppose. Ansible is on the listâŚsetting everything up after I mess it up can get old.
Now youâve got me thinking about spending more money !! ⌠I have two already (three if ya count the UDM Pro), and I really like my Netgate pfSense combo, but was a bit worried about the CPU horsepower, not that I need a massive amount.
In any case, those (Protectliâs) look pretty nice for the $$.
Iâd agree with @Buffy on the Firewall.Router choice. Spending a few extra $$ on that piece will payoff (in lots of ways) in the long run. Having said that, I went a bit overboard buying three different products until I settled on what I have now, so thereâs definitely a learning curve to it.
As for virtualization hardware, Iâve had a lot of failures there also. I canât say what the right choice is for your situation, but, I can tell you that buying cool (expensive) gaming hardware is a waste of $$ and adds a good bit of frustration. If I were to make any recommendations it would be along the lines of:
Invest in Good Hard Drives for your NAS, and plan ahead for storage capacity. Whatever you think youâll need, have a plan to double it, especially if youâre using ZFS.
@Jay had a Server build using an embedded EPYC CPU. Thatâs a nice chip, and certainly worth considering. I wish I had seen that long ago.
I found that I didnât need high CPU clock speeds, as the core count(s) matter more for my work loads. I use a 2700X in my TrueNAS box, and the clocks are way more than I need. Could of used the 1700X and saved some cash or even went with a cheap older XEON.
RAM, canât have enough really. Most consumer grade MBâs tap out at 64GB, some 128GB where the many server board are upwards of 256GB and beyond. I saved my nickels and went the ECC route, especially on virtualization hosts and my TrueNAS box, and glad I did.
Iâve got a pile of hardware that I âthoughtâ would be a good choice over the years (I was wrong, a lot). If I had saved for the âright hardwareâ rather than the cool / fast stuff, Iâd be much better off today for the workloads I run (mostly web services of some kind). If youâre need is video rendering, editing, storage, the price tag goes up if performance is a consideration.
The same could be said of networking gear. That one I actually did ok on, apart from overdosing on Routers/Firewalls.
Anyways, I think the moral of my story is, I didnât have a well thought out plan, and I put emphasis on the wrong hardware for my needs regarding Homelab. I would of been fine if all I was doing was development work.
Totally agree; especially for core pieces of your network. We keep our data on a set of Synology NAS (DS1618+) so have local backup, then also backup to Synology C2. We also run our GitLab CE on the main one in a Docker container. So our data is well-protected and gives us a lot more freedom in both homelab and daily workstations.
Urgggg. I feel your pain. Getting started can be a frustrating process. Hardware/software compatibility can be a bear until you learn the vocabulary⌠and it is nearly impossible to learn the vocabulary until you and have working gear on which to experiment. At least for me, much of the learning comes from problem-solving.
A couple of thoughts that I have regarding your situation:
You donât need to set up everything at once. A standard home router with off-the-shelf software will work just fine for 90% of what you need to do. To be honest, it doesnât hurt to have worked with a couple of different brands of consumer-grade routers in order to become familiar with some of the ways different companies try to direct you to learn and become âinvestedâ in their gear.
For the NAS, one option to consider is Open Media Vault. It is a NAS system that runs on most standard hardware. The GUI is pretty complete and logical. It can also do 90% of what you need a NAS to do in a home lab; Samba server, NFS server, backup targets, and docker server.
As for virtualization, you can do a lot with Virtual box running on your laptop and docker running on your Open Media Vault server.
A lot of home labs work with retired enterprise gear for many of the reasons you mentioned. It is guaranteed to run enterprise-level (or home lab) grade software.
This might seem odd, but Synology makes some really good SMB NAS equipment. It might be hard on your budget, but in most cases, it just works. Another thing in Synologyâs favor is SHR. (I think it means Synology hybrid raid) You can add drives of different sizes as your needs change. The only restriction is that each new drive must be at least as big as the smallest drive in the system. This can be a really handy feature until you figure out what your storage needs actually are.
The purists are probably having heart attacks just thinking of SHR⌠for me it was a reasonable way to help me climb the learning curve without breaking the bank.
Interestingly, when you upgrade your NAS to something new the old NAS can have many years of life as a backup server. For me, my retired nas is working as a remote backup at my sisterâs house. It turns itself on at 1 AM friday morning and rsyncs all the important stuff from my primay NAS for offsite storge.
Which protectli did you go with? I am looking at the FW4B.
Internet in our area is pretty slow. I am currently on a 200Mpbs plan. There are boxes for fiber in our neighborhood⌠but it is not turned on yet.
I currently run Wireguard VPN as containers on my i5 NUC. As a family, it is not unusual for us to run three separate VPNs. So, I donât think processing power on the Protectli is a big issue.
I currently, have an Unifi USG Security Gateway â Ubiquiti Inc. which seems to meet my processing needs. But, the software is driving me a bit nuts. For a couple of years, I was really happy, but when I got a new WIFI6 AP I had to update the controller software which has some frustrating bugs and quirks.
We have the FW4B; it doesnât even notice when we run 2 VPNs, even with heavy loads on them. We have Coreboot and OpnSense on ours, but you can also get it with pfSense. I think both are really good and easy to use.
Speaking of failures⌠My goal for the weekend was to install xcp-ng on a laptop for testing. The iso starts booting and then fails with a kernel panic. Rather than testing xcp-ng I guess I will end up trying to figure out why it kernel panics.
@buffy I ordered a FW4B with coreboot and opnSense. Iâll test opnSense for a few weeks before switching to pfsSense to test that out. Kind of blew my budget for a couple of months.
I didnât know much about xcp-ng until I started watching videoâs from @Jay and Tom Lawrence. Iâve installed it a number of times, but still end up back on Proxmox for one reason or another. That is one of the great benefits (one of many) of these Homelab deals, you can just blow it away and re-do it as many times as you like. I do like the Dokcer aspect of xcp-ng though. I think that is what keeps pulling me back to it.
Yes, I havenât dug into kernel panics for years. Iâll probably just spend a couple of hours getting familiar with the kernel output that files past the screen during boot. I donât see a simple way of logging them to the boot USB.
The laptop has a third-generation i5 mobile processor. I think it is from 2013. That is not exactly one of the XCP-NG target processors so my hopes are not too high.
