Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am attempting to share 2 USB drives over NFS on a linux machine to an ESXi host as 2 different datastores. When I connect to NFS server using either the ESXi host or another NFS client, I can mount the shares, but they both map to the same drive, however on the server the drives are independent.

/etc/fstab:

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    nodev,noexec,nosuid 0       0
/dev/mapper/Backdoor-root /               ext4    errors=remount-ro 0       1
# /boot was on /dev/sda1 during installation
UUID=02f6d7c8-43c1-4454-9283-c76a69fcbe15 /boot           ext2    defaults        0       2
/dev/mapper/Backdoor-swap_1 none            swap    sw              0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0

/dev/sdb1       /mnt/usb3       auto    rw      0       0
/dev/sdc1       /mnt/usbTest    auto    rw      0       0

/mnt/usb3       /export/usb3drive       none    bind    0       0
/mnt/usbTest    /export/usbTest         none    bind    0       0

The mounts are correct because when I navigate to the /export/* I get what I expect to be on each drive:

ls -l /export/usb3drive:

total 16
drwxrwxrwx 1 root root 12288 May  6 10:40 ghettoVCB-master
drwxrwxrwx 1 root root     0 May 18 19:30 Hera
drwxrwxrwx 1 root root     0 Feb 13 15:38 $RECYCLE.BIN
drwxrwxrwx 1 root root  4096 Mar 13 03:08 System Volume Information
drwxrwxrwx 1 root root     0 May 18 19:30 ZEUS

ls -l /export/usbTest:

total 0
drwxrwxrwx 1 root root 0 May 20 08:59 Hera
drwxrwxrwx 1 root root 0 May 20 14:59 Hera-Restore
drwxrwxrwx 1 root root 0 May 20 08:37 $RECYCLE.BIN
drwxrwxrwx 1 root root 0 Sep  2  2011 System Volume Information

The /export/users directory just has a text document in it, which I'm able to NFS mount and see fine.

/etc/exports:

# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#
/export 192.168.101.101(rw,sync,no_subtree_check,fsid=0)
/export/users 192.168.101.101(rw,nohide,insecure,no_subtree_check,sync)
/export/usb3drive 192.168.101.101(rw,nohide,insecure,no_subtree_check,sync,mp=/mnt/usb3)
/export/usbTest 192.168.101.101(rw,nohide,insecure,no_subtree_check,sync)

When I run exportfs -rav:

exporting 192.168.101.101:/export/usbTest
exporting 192.168.101.101:/export/usb3drive
exporting 192.168.101.101:/export/users
exporting 192.168.101.101:/export

Whenever I make changes I run service nfs-kernel-server restart and service idmapd restart

I then connect the ESXi host (or mount the NFS connections, both have the same result) datastores, the Datastores are:

BackdoorBackup -> 192.168.101.12:/export/usb3drive BackdoorTest -> 192.168.101.12:/export/usbTest BackdoorUser -> 192.168.101.12:/export/users

When I explore these Datastores I get:

ls -l BackdoorBackup:

/vmfs/volumes # ls -l BackdoorBackup/
drwxrwxrwx    1 root     root                  0 Feb 13 20:38 $RECYCLE.BIN
drwxrwxrwx    1 root     root                  0 May 18 23:30 Hera
drwxrwxrwx    1 root     root               4096 Mar 13 07:08 System Volume Information
drwxrwxrwx    1 root     root                  0 May 18 23:30 ZEUS
drwxrwxrwx    1 root     root              12288 May  6 14:40 ghettoVCB-master

ls -l BackdoorTest:

/vmfs/volumes # ls -l BackdoorTest/
drwxrwxrwx    1 root     root                  0 Feb 13 20:38 $RECYCLE.BIN
drwxrwxrwx    1 root     root                  0 May 18 23:30 Hera
drwxrwxrwx    1 root     root               4096 Mar 13 07:08 System Volume Information
drwxrwxrwx    1 root     root                  0 May 18 23:30 ZEUS
drwxrwxrwx    1 root     root              12288 May  6 14:40 ghettoVCB-master

ls -l BackdoorUsers:

/vmfs/volumes # ls -l BackdoorUsers/
-rwxrwxrwx    1 root     root                  0 May 21 13:36 doyouseethis.txt

As you can see, BackdoorBackup and BackdoorUsers mounted find and correctly, however BackdoorTest has the same export as BackdoorBackup.

After everything is mounted: showmount -d 192.168.101.12: (run on the NFS server)

Directories on 192.168.101.12:
/export/usb3drive
/export/usbTest
/export/users
/mnt/usb3
/mnt/usbTest

But /proc/fs/nfs/exports doesn't list usbTest:

# Version 1.1
# Path Client(Flags) # IPs
/export/users   192.168.101.101(rw,insecure,root_squash,sync,wdelay,nohide,no_subtree_check,uuid=293c3653:96cc461f:877f0a17:48565edd)
/export/usb3drive       192.168.101.101(rw,insecure,root_squash,sync,wdelay,nohide,no_subtree_check,uuid=f474b7aa:74b76dcc:00000000:00000000)

Now if I removed the /export/usb3drive line from /etc/exports then refresh the datastore list, BOTH BackdoorBackup and BackdoorTest point at /export/usbTest. So even though there is no assigned export for /export/usb3drive the NFS server throws no errors with a client trying to recieve it.

The NFS server is:

Linux version 3.2.0-23-generic (buildd@crested) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu4) ) #36-Ubuntu SMP 

Update:

I switched the nfs implementation over unfsd using unfs3 0.9.22. And now I see both drives fine using a standard linux mount, however another issue has arisen because of trying to mount them on ESXi 5.0.0 (I might try upgrading our test server's implementation of ESXi). All the drives mount correctly, the Capacity/Freespace report correctly in the datastore browser, but when I try to browse the datastores they return blank, and even through the CLI on the host, ls returns nothing. The really odd part is that I can access the files fine if I know the absolute path. None of these issues arise when using another linux machine.

Apparently this is an issue that has come up before, just not on our implementation stack. ESX-VMWare-on-NAS-NFS

share|improve this question
4  
I am attempting to share 2 USB drives over NFS ==> Because NFS itself is not unreliable and problematic enough, I've decided to make it worse by pointing it at removable storage in the hope someone will unplug it in the middle of a critical write. I BEG OF YOU - please reconsider this Bad Idea before you shoot yourself in the foot so hard you lose an entire leg. –  voretaq7 May 21 '13 at 17:40
    
The only other option for mounting remote datastores in ESXi that we found is iSCSI LUNs, which we cannot make accessible from multiple ESXi hosts. The drives are used as backups from our ghettoVCB runs, so then can be taken offsite, then if there is an issue spun up on the same drive without a copy operation. If you have a better solution for mounting removable storage on an ESXi host that we can take offsite, I'm all ears. There are also only 2 people in my department that work on the servers, and the operations are schedule d when the office is closed. –  JGamble May 21 '13 at 17:48
    
It's not so much the "NFS" that I object to as it is the "USB" -- you're setting up a situation where a plug getting jostled can knock out your ESXi host - that's not really a great situation to be in. If that's not a major concern for your environment that's OK, but I did feel morally obligated to point out the giant foot-gun :) –  voretaq7 May 21 '13 at 17:54
    
It's not a big concern for us because we're not going to be running production off of it for very long, if ever. It's our safety net in case our main host goes down, we can spin up quickly on our lurching backup server. The hope is to never need too, it also lets us "quickly" move the backup of our main servers to a test environment. –  JGamble May 21 '13 at 18:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.