I'm building a 3 server setup (FreeNAS, FreeNAS replication, ESXi) for a small network with a handful of users and a couple of "power users". The aim, briefly, is to migrate from multiple machines and windows file shares to more centralised and better resource usage, and higher quality and reliability.
The hardware is adequately specced (ESXi on octocore Xeon + Supermicro, and FreeNAS on quad fast Xeon for CIFS with a ton of RAM and Intel SSDs). But I'm expecting a considerable learning curve because of the differences of what I'm moving them from and to.
I was intending to run my file shares as NFS (for ESXi) and CIFS (mostly Windows + a couple of Mac clients). Then I came across VAAI and ODX and thought "wow", as the workloads look like they would really benefit from them. But as with any iSCSI, they'd replace NFS by one initiator/one target per share only unless one really loves corrupt data. So I'm wondering whether I have to give up on the idea of these nice shiney protocols, or whether I can get at least some benefits they offer if not all of them:
If I ran ESXi (but not CIFS) as iSCSI, then the question becomes how to copy VM store files off the iSCSI pool if I wanted to, which will be useful, or indeed, how to upload extra files onto the pool. I don't want constant access but I would like to feel I have a way to access the iSCSI based store either read-only, or read/write when the usual initiator isn't connected.
Is there a way to maintain a second iSCSI initiator running as read-only within FreeNAS itself, so that I can use FreeNAS itself to pull data that I know is idle (old files etc) from an iSCSI target on FreeNAS, to a separate machine?
Ideally it would mean that when ESXi is disconnected, I can locally copy files to/from/between the VM store and an NFS share, or make the files on the VMFS store accessible and shared read-only to other machines
I understand that iSCSI doesn't 'know' about files, and that a file and its metadata which was in use wouldn't be guaranteed to be served correctly to a second 'interloper' read-only initiator if it was changed halfway through the request or something, but I'm thinking here about reading from old ZFS snapshots or when the main initiator isn't connected, or files known to be idle - situations where there may not be the usual risk of conflicting instructions and corruption, and for a small setup it would be helpful.