I am trying to figure out the optimal way to share storage with multiple VMs on the same box. I am aware of a range of options: NFS, iSCSI, fibrechannel, bridged, etc, but I am unclear how they fit together or are mutually exclusive.

Here are a few statments of what I understand to be true based on what I have read so far, but an not certain of. If someone could either affirm or correct my understanding, that would be great. At this time I am thinking about using Proxmox (KVM+ZFS), but if other hypervisors have important differences with respect to these statements, please do explain.

True or False?:

  1. For sharing storage devices between VMs on the same host, I can used bridged networking to maximize speed.

  2. With bridged networking, no physical network hardware is being used, instead the CPU is acting as a virtual NIC, which is faster because the limit is the speed of the system bus, rather than of ethernet/fibrechannel etc.

  3. For the reason stated above, there is no benefit to using something fancy like fibrechannel to share storage with VMs, except the elimination of a small amount of CPU overhead. Using a fibrechannel card would be needlessly creating an extra step for VM interaction with the hosted storage.

  4. The speed and complexity of setup of iSCSI vs NFS is something that varies a lot between hypervisors, so this choice should be made once I have everything else decided.

  • Don't expect the software switches to be faster than hardware. And I am not sure how guest networking relates to host storage in your case. – eckes May 20 '17 at 23:47
  • 1
    No I am just correcting your statement about the fast software switch, they are not so fast in all cases. However the access to the storage won't go over network anyway. – eckes May 21 '17 at 2:01
  • 1
    Yes NFS goes over the network, it depends however on who speaks NFS. Most often the storage protocols are spoken by the host. The guests see block devices. Especially if you plan to make them available to multiple guests. You can also make the guests speak the network protocols, but then you have for example no acceleration for iscsi in virtual NICs. When you share a block device the host has more potential for caching. – eckes May 21 '17 at 6:44
  • 1
    Oh now I understand, I was assuming you had a special reason to share a volume. So if you want to share a volume between multiple guest you need coordination by a Cluster filesystem or failover Cluster to manage this. If all you want is to share files between multiple guest then a network filesystem mounted inside multiple guests is the more leightweigt solution. (Iscsi and SAN/FC would all require coordination). It depends a bit on how you manage your hosts how you would actually serve the NFS filesystem. This can be done by the host but it's rather unusual. – eckes May 21 '17 at 12:35
  • 1
    If you have an NFS Server on your host and use a virtual switch/software bridge then no network hardware is involved. – eckes May 21 '17 at 18:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.