Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an ETL process that opens files from an NFS mount to process them. The number of files to be opened simultaneously is configurable.

I don't have shell access to the NFS server.

What are some good shell utilities I can use on the client to determine how effectively I'm using the NFS share? I want to maximize my throughput (balanced against flooding my ETL with too much data) and obviously, I don't want to put too great a strain on the NFS server.

How much can I do from the client vs. having to log on to the NFS server and run IO monitoring utilities there?

share|improve this question

2 Answers 2

up vote 2 down vote accepted

I would start with nfsstat and iostat.

For example, you might want to watch:

watch -n1 'nfsstat -c'
iostat -n 1 1
share|improve this answer
    
I'm digging for more information about the output of nfsstat so I can make sense of it. Once I can understand what the output is telling me, I'll know if this is the best answer to my question. :) –  deinspanjer May 27 '09 at 20:56
1  
Truth be told, you may have better luck with iostat for what you're doing. –  MikeyB May 27 '09 at 21:05

You might also find pipe viewer useful, depending on what your ETL process is doing with your data.

As per the following tutorial, it's basic usage is of the form:

pv access.log | gzip > access.log.gz

Where it would let you know how much data is completed vs how much is left to go.

You can also chain commands through it as so:

pv -cN source access.log | gzip | pv -cN gzip > access.log.gz
share|improve this answer
    
pipe viewer can be an incredibly useful tool when working with files on the command line. Unfortunately it doesn't help in this situation because the ETL process (a Java application) is responsible for opening the files and parsing through them. One way I recently used pv was to be able to tell how long it was going to take to bulk import a huge MySQL script file generated by mysqldump: pv dbdump.sql.gz | gzip -cd | mysql -u root -p –  deinspanjer May 27 '09 at 19:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.