Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to figure out the correct way to get the 'real world' write performance of my NFS cluster.

The NFS export is mount on a XEN hypervisor, and I'm running 'dd' from within a VM.

What I can't decide is should I use O_DIRECT/O_SYNC or not.

If I won't use it, the kernel will do page caching to RAM, and I'll will end up measuring RAM throughput, not disk.

On the other hand, most if not all server applications will use RAM caching anyway(am I wrong with that assumption?).

So I wonder where is the balance point, what will really happen in the 'real world'?

share|improve this question

First of all, I'd use an actual I/O benchmark utility such as 'iozone'. It'll test a variety of things and will even use O_DIRECT if flagged to do so. And even, as I recall, has some NFS-specific flags.

One thing I've found to be very reliable is to use a file-size that exceeds the RAM size for the device you're testing on. This busts the cache, and gives you reliable I/O numbers. If you have a 2GB RAM machine and test on a 4GB file, you'll get reliable numbers. If you use a 1GB file, but use the O_DIRECT flag instead, you'll get somewhat different numbers due to the 'only report comitted when told it is flushed' aspect of O_DIRECT.

This gets complicated when your network-file-system target also has a block-cache, so you may need to size your test to bust that cache rather than your Xen-local cache.

share|improve this answer

This only depend about what you want to measure.
You just forgot to define your goal.

If you want the NFS cluster raw performances, then you have to bypass your RAM.
If you want "real world" performances, then you have to use your RAM. But real world I/O are rarely those from dd. You will have better results with iozone for example.

So define your goal.
Then choose the appropriate tool and method.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.