Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an NFSv3 server and around 15 clients. I am looking for pros and cons enabling async on the server side. I have read about it, but it is still a bit unclear to me. I know it can lead to data corruption, if the server crashes in the middle of write operation. However, I also read that the client stores a cache of that same operation and can recover it, if needed. My questions are:

  • What exactly would happen if my server crashes (i.e. would it lose pending-to-be-written data, would it corrupt the underlying filesystem, etc)?;
  • What would happen if both the server and the client crash at the same time (i.e. power failure/fault and UPS failure to handle it)?;
  • What if the server crashes, but I have RAID BBU. Would the server recover safely?;
  • Is there any way to detect such a corruption (something similar to fsck maybe)?;
  • What if the server shutdown gracefully by UPS? Will I have chances of data corruption then?;
  • What do you guys use - sync or async?

All machines are Ubuntu OS 10.04.

I was trying to find similar question here to no available. I have read the NFS Home Page and took a quick look at Managing NFS and NIS, 2nd Edition book.

share|improve this question
add comment

3 Answers

up vote 5 down vote accepted

So what the NFSv3 spec says, is basically that for the following two NFS data operations

  • WRITE operation with the stable bit set
  • COMMIT

the server is allowed to return success to the client only after the data has hit stable storage. This is what the Linux NFS server implements with the default "sync" export option. With "async", the server can cheat and return success even though data is not on stable storage.

That is, the potential corruption issue with async is basically something along the following

  1. Server returns success for a WRITE or COMMIT operation
  2. Client sees the success, and at some point deletes the pages from its own cache (why waste space keeping them around since they are already on server storage, it thinks)
  3. Server crashes, thus losing the data which was not committed to stable storage
  4. Client reconnects to the server, but as there is no log of which data was written or not, it cannot know exactly which data was lost.

Now, the last point is the serious thing, in that there is no way to know which data was lost/corrupted or wasn't.

OTOH, if the client crashes, then any dirty data in the client cache (that hasn't been flushed) will be lost, but the client programmer can work around it (i.e. only after fsync() or close() returns success can the programmer assume data is on stable storage).

share|improve this answer
    
If I have understood correctly, using async for NFS is analogous to allowing disk cache (e.g. using nobarrier for local ext4 file system) and then losing power. In case of NFS "losing power" is technically server's kernel crash. The end result should be similar. –  Mikko Rantalainen Dec 11 '12 at 8:28
add comment

No. Exporting with async means the server tells the client "I've stored what you gave me on stable storage, you can stop worrying about it now" before it's even tried to write the data to your RAID.

To expand on this a bit, since you have a RAID card BBU, you'll get much quicker NFS performance by enabling the write cache. This is what the BBU is for, to keep data in this cache alive after power loss. I wouldn't enable async in production. as like the above author states, this is a separate part of the chain.

I recommend this ZFS article that includes a good bit of generic NFS and performance info:

https://blogs.oracle.com/roch/entry/nfs_and_zfs_a_fine

share|improve this answer
add comment

what exactly would happen if my server crashes (i.e. would it lose pending-to-be-written data, would it corrupt the underlying filesystem, etc)?;

Unrelated to the fact that the computer is an NFS server, if it crashes you will lose the data in your page cache (i.e. data that has been written but hasn't been flushed from RAM to disk yet). With a journaling filesystem the filesystem should be repaired automatically on next mount using the journal.

janneb has written a good explanation of what the crash means in the context of an NFS server.

what would happen if both the server and the client crash at the same time (i.e. power failure/fault and UPS failure to handle it)?;

You verify any data that matters.

what if the server crashes, but I have RAID BBU. Would the server recover safely?;

No. Exporting with async means the server tells the client "I've stored what you gave me on stable storage, you can stop worrying about it now" before it's even tried to write the data to your RAID.

is there any way to detect such a corruption (something similar to fsck maybe)?;

As janneb says, no.

what if the server shutdown gracefully by UPS? Will I have chances of data corruption then?;

No, because in this case the NFS server will write all the data to stable storage.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.