This is my first post on this forum so I thought I would share a bit about my home lab. My network consists of a pfSense firewall running on a Qotom device. If feeds an 8 port 2.5 gbps managed switch. I have a WAP hanging off of the switch as well as multiple wired connections, most of which are running at 2.5gbps. My network is broken up into multiple VLANs (trusted devices, TVs, IOT devices, my server, and apps exposed to the internet). My server is an old HP Z640 with an E5-2690V3 processor and 64GB of ram. I am running Proxmox on a pair of 2 TB SSDs in a ZFS mirror, and I have a pair of NVME M.2 drives for my VM storage via an Asus Hyper M.2 card in one of my PCIE x16 slots with bifurcation. I run 3 different SAN devices: A Synology DS 220+, a home brew raspberry pi4 running OpenMediaVault, and an OpenMediaVault VM using a couple of pass through spinning disks, running in software raid with BTRFS. All of my VMs are debian 12 instances. I run three instances to host docker, in separate VLANs, and a couple of VMs for apps I expose to the internet (wordpress and nextcloud via Clouflare tunnels). I also am running a couple of apps in LXC containers.
Overall, I am running the following apps: Wordpress, Nextcloud, Grocy, Mealie, Tracks (GTD software), Lean Time, Monica, Portainer, Photoprism, Cloudflare connectors, Homeassistant, Guacamole, and Heimdall.
I also run a couple of Wordpress instances in AWS and back up all my data to AWS glacier. That may change now that AWS has laid me off. I got caught up in the last round of AWS layoffs.
I also run an application called Pi-Star on a separate raspberry pi. It is an amateur radio program to create a local radio repeater connected to the internet. I use it for DMR radio access.
Have you considered running OMV on NFS instead of passthrough? Especially for the Pi.
Do you keep your homelab up 24/7? How much power does that use, it must be insane. My homelab consists completely of ARM SBCs and with my PC combined (excluding the threadripper which hasn’t been powered in a long time now), mine uses about 45W. Should probably go up about 5 to 30w (depending on the load) once I get 2 HDDs for my planned backup server (it’s already up, I just need the drives).
Aww, that sucks. I hope you find a job soon. Were you a data center engineer, programmer, or?..
OMV on NFS? OMV is serving up a couple of NFS shares for persistent docker storage, but I am not sure about your question.
I do not keep my “server” on 24x7, but you would be surprised I think at how little power it uses. Most of the workloads I run on it are not that CPU intensive and I am pretty much the only one accessing the server at the moment, so even if I crank up the CPU cycles on one application or another, it is a pretty light workload for the CPU. This machine only draws 70 watts at idle and it is idling most of the time (as measured at the plug on a “kill-a-watt” device).
I leave the Qotom/pfSense firewall, the switch, the WAP, the synology, and the raspberry pi NAS on 24X7. I haven’t measure what they collectively draw for power. I might try that later today.
re the AWS thing, I was in the sales org. I had an interesting job helping customers use cloud to achieve business outcomes, such as increasing their customer retention, closing their books faster at the end of the month, etc.
For things like the RPi, which only has USB (unless you have a CM4 with a SATA board or PCI-E for SATA card or NVME drives), instead of attaching a USB HDD to them and serving data from there, or worse, serving data from a SD card. Instead of using local storage on the Pi, mount a NFS on it and then use OMV to share data from somewhere else. Similarly, instead of passing the PCI-E card on your server and doing a BTRFS array in OMV, instead just ZFS share from proxmox to your VM and having the data in the general larger pool.
It doesn’t have to be NFS, it could be iSCSI too. The general idea is: have the data somewhere else, mount it inside the VM or RPi and use OMV just for the file sharing.
Nothing wrong with the SATA passthrough, but I really don’t like the idea of using SD or USB HDDs / SSDs for data sharing (despite myself having done it for quite a long time without any issues - I moved away from that a few months ago).
Are you hoping to move to an IT dep. now, or is homelabbing just a hobby?
Ah, I understand. I run three NAS devices: the synology, my pi NAS which has two SSD drives attached via USB and they are mashed together into a single pool with LVM. I run EXT4 on those because the only purpose of that NAS is to be the second, or in some cases third copy of my data for local back up purposes. The pi NAS is running an Rsync server, and my synology backs up to it using hyperbackup. Inside of my Proxmox machine I have a couple of TB of spinning disks that I don’t use for any purpose related to Proxmox, so I pass the through to OMV using a command such as “qm set 103 -scsi2 /dev/disk/by-id/ata-ST1000DM003-1CH162_S1DFPKZY,serial=myserial002”. It works fine for the most part. I have three such disks available to the OMV VM and I have them in a software raid with BTRFS as the file system. I was trying to recreate how Synology implements BTRFS in OMV, mostly as an experiment.
Re your second question, this is just a hobby. I am an accountant/CPA, but I have been in the IT industry for over 30 years, mostly in the IT outsourcing space before going to AWS about 6 years ago. You can’t help but learn a bit about the technology when you control the IT budget. I got into the homelab thing last year. My wife asked me to help her set up a food blog on Wordpress. I knew nothing about wordpress, and started googling youtube videos to teach myself how, and stumbled onto the concept of building a NAS from a raspberry pi. That was my gateway drug. After that, it just kind of went a little crazy.
Also I slapped the kill-a-watt onto the power strip feeding my 24x7 devices (router/firewall, switch, WAP, Synology, Pi NAS, Pi-star radio repeater hot spot) and it is averaging 45 watts for all of that. So roughly $7/month at my current electric rates.I did the math, and if I ran my server 24X7 at the average of a 70W draw, it might be another $10/month on the electric bill. I think my daughter leaving every light on in the house, PLUS my wife leaving multiple TVs on all the time, might be a larger hit to my electric bill. LOL
