I’ve been building (and re-building) by Homelab setup the last month or so. Mainly figuring out what will work best for me long term. One thing is for certain, TrueNAS will be at the center.
The Problem
I work with Big Data a good bit, mainly Amateur Radio related. I tested copying over NFS then with iSCSI. While I didn’t “benchmark” the two, iSCSI was miles ahead on speed (visually at least). The data set I’m working with is broken down into Year/Month csv.gz files (WSPRnet Archive). I convert those to Parquet for Spark Cluster processing and produce smaller datasets / metrics. Each of these files, the later years at least, have over 70 Million rows per month, and it’s increasing at a rapid pace. Fast transfers across the wire is important, particularly with things like Jupyter Notebooks. The initial configuration and reconnect / mounting is not for the faint at heart, and leaves a lot left to be desired in terms of “ease-of-use”.
What Is Needed
I found the lack of accurate documentation frustrating in regards to repeatable mounting solutions for a given OS. I finally got it figured out on Ubuntu and Oracle Linux servers, but I think this could help “a lot” of home user across many different scenarios. However, my process is still fragile and does not take into account a great deal of ACL (if any)
So, anyone that could produce a video / document trove for this would be greatly appreciated by many, IMHO
Just a Thought