Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am not sure how Linux kernel handles multiple file system drivers and if an inode lookup for each given mounting point in file system interferes with other mounting points, or is there something else that will make the system perform slower in case there is some slow file system driver in the kernel but mounted file system that is managed by that driver is almost not in use.

My concern is NFS. So does an NFS driver slow down a Linux kernel and/or user space assuming that operation that requires a fast disk I/O is being performed on a local file system?

I assume that having LD_LIBRARY_PATH or PATH may slow down the system as ld and/or bash will try to lookup stuff in directories on NFS. But does NFS really interfere somehow when I open a file on a local filesystem?

Thank you.

share|improve this question
add comment

3 Answers 3

up vote 3 down vote accepted

Each inode points to a single file system, and IO to/from that inode is handled solely by that filesystem driver, regardless of which other filesystem drivers are loaded into the kernel.

In particular, NFS does not per se slow down other IO. Of course, if the system is getting hammered over NFS, then also local IO for that filesystem will be slower, but that is because of the high IO rate to a shared resource (the disks underlying the filesystem), not anything specific to NFS. IOW, if the NFS subsystem is idle it should not in any way affect local IO.

And no, NFS will not interfere if you're looking up a file on a local filesystem.

share|improve this answer
add comment

I mount nfs from remote server to /nfs(50 directory and 100000 files). Then create,write and read files in other direcroty five times:

time for i in {1..1000};do touch $i;done
time for i in {1..1000};do echo "$i" > $i;done
time for i in {1..1000};do cat "$i" > /dev/null;done

create: average 7.903s
write: average 0.068s
read: average 7.682s

Then umount /nfs and test again

create: average 7.923s
write: average 0.048s
read: average 7.782s

Then restart server and test again(without /nfs)

create: average 7.901s
write: average 0.052s
read: average 7.582s

If the effect exists, it is very small. Server: HP ProLiant DL160 G6, Linux 2.6.32-24-generic-pae

share|improve this answer
    
So when you restart the server, NFS is not mounted and NFS kernel driver is not loaded. Is that correct? –  user57260 Oct 15 '10 at 15:14
add comment

The above answers, while correct, did not address the library searching that goes on with every system call that you asked about. In short, I ran a test on lightly loaded systems and the system calls against NFS took no additional, statistically significant time. I have an NFS mount point /data/media that I added to LD_LIBRARY_PATH and then I use strace to track system calls and their time to execute.

For those unfamiliar with starting an executable, every shared library must be found and loaded before execution passes to main(), which is where the probing of NFS could be a potential problem. By adding an NFS mounted directory to the library loader's path, we should be able to do simple timing tests. Obviously, if an NFS server is not responding, the execution of all new processes will likely halt, especially if the mount point is mounted as "hard," that is, without the "intr" option. This is partially because the shell will search the PATH for a new program's name and gets stuck on the NFS mount point. That is not what we are about to measure.

Here is the shell code to run a program and document the syscall execution time. Note again that I only have one NFS mount point, /data/media, in LD_LIBRARY_PATH just to keep it simple.

strace -T -e open java -version  2>&1 |\
perl -ne 'if (m/\/data\/media.*<(\d+\.\d+)>/) { \
             print "nfs,$1\n";
          } elsif (m/<(\d+\.\d+)>/) {
             print "local,$1\n";
          };'

The original strace output is similar to this

open("/usr/lib/jvm/java-6-sun-1.6.0.20/jre/bin/../lib/amd64/jli/tls/x86_64/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000041>

And the output is similar to this

local,0.000030
local,0.000031
nfs,0.000042
nfs,0.000029
nfs,0.000029
nfs,0.000030
local,0.000028
local,0.000027
local,0.000027

I chose perl because I'm comfortable with it, but any text filtering will do.

I threw these number in a spreadsheet and sorted them so that I could show you this table

               Avg (s)     Max (s)   Min (s)
NFS lag (s) -0.0000072   -0.000529  0.000007
NFS lag (%)    -17.84%     -90.74%    36.84%

Quite obviously, you should try throwing your system cache out the window before doing this test and see if it changes things, but that's not the normal condition for your system.

In case you have trouble interpreting the table, there were cases where NFS was actually faster than the local filesystem. This is counter-intuitive until you consider that VFS caching is probably going on.

Therefore, I conclude that for at least this little tiny test, having an NFS mount point in your LD_LIBRARY_PATH does not affect load time for executables. As long as the NFS server is up...

share|improve this answer
    
+1: Interesting! –  Steven Monday Oct 15 '10 at 22:56
    
+1, nice to know. Thank you. –  user57260 Oct 16 '10 at 2:52
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.