TrueNAS and iSCSI Reconnect Guide Needed

I’ve been building (and re-building) by Homelab setup the last month or so. Mainly figuring out what will work best for me long term. One thing is for certain, TrueNAS will be at the center.

The Problem
I work with Big Data a good bit, mainly Amateur Radio related. I tested copying over NFS then with iSCSI. While I didn’t “benchmark” the two, iSCSI was miles ahead on speed (visually at least). The data set I’m working with is broken down into Year/Month csv.gz files (WSPRnet Archive). I convert those to Parquet for Spark Cluster processing and produce smaller datasets / metrics. Each of these files, the later years at least, have over 70 Million rows per month, and it’s increasing at a rapid pace. Fast transfers across the wire is important, particularly with things like Jupyter Notebooks. The initial configuration and reconnect / mounting is not for the faint at heart, and leaves a lot left to be desired in terms of “ease-of-use”.

What Is Needed
I found the lack of accurate documentation frustrating in regards to repeatable mounting solutions for a given OS. I finally got it figured out on Ubuntu and Oracle Linux servers, but I think this could help “a lot” of home user across many different scenarios. However, my process is still fragile and does not take into account a great deal of ACL (if any)

So, anyone that could produce a video / document trove for this would be greatly appreciated by many, IMHO :slight_smile:

Just a Thought

I don’t know if this will help you directly…

In my small home lab, I added a two TB SATA drive to my application server. I also work with a lot of largish data sets of between one and ten GB of sensor data from autonomous vehicles.

Caching the data sets I use most often on the machine which performs the analysis helps buy me time until I spend the money on upgrading to a 10GB network or screw around optimizing storage protocols.

Honestly, I’m not sure either. I have 2x NVME’s in my NAS box just for Log and Cache (not enabled at this time). However, reading many threads on the subject, it doesn’t “appear” that I need either based on what I am seeing in the metrics.

The box has an 8core/16 thread CPU and 64GB ECC ram. From what I read, it would probably hurt performance more than help, but, I am certainly a “Novice” when it comes to NAS performance tuning.

My main issue at the moment is getting various boxes / VM’s to connect / re-connect consistently with fstab entries. It seems to be a matter of position within the fstab file and where Networking is with respect to mounting physical devices during boot. That’s my best guess anyway.