Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We several WebSphere application servers logging to a central NFS mount. Both the app servers and the NFS server are running in VMWare, the app servers are RHEL 5, and the NFS server is Ubuntu 8.04. All servers are connected via gigabit ethernet, at a minimum. We are experiencing random corruption of the logs, where there will be large amounts of white space in them. We used to mount directly to a NetApp server, but migrated to this setup. The NetApp doesn't have this problem.

On the NFS server, I've set the following sysctl commands:

sysctl -w net.ipv4.tcp_wmem="4096 87380 8388608"
sysctl -w net.ipv4.tcp_rmem="4096 87380 8388608"
sysctl -w net.ipv4.tcp_sack=0
sysctl -w net.ipv4.tcp_dsack=0
sysctl -w net.ipv4.tcp_max_syn_backlog=8192
sysctl -w net.ipv4.tcp_synack_retries=3
sysctl -w net.ipv4.tcp_retries2=5

The filesystem is exported with this command in /etc/exports:

/srv/export/logs        192.168.0.0/255.255.255.0(rw,no_root_squash,no_all_squash,sync,no_subtree_check)

And mounted with this command on the app servers:

mount -t nfs -o rsize=8192,wsize=8192 192.168.0.1:/srv/export/logs /webspherelogs/
share|improve this question
    
Is each application server logging to a different filename/set of files? –  MikeyB Oct 31 '09 at 3:39
    
Do you have an update on your issue? Did our suggestions give you a lead to solve that problem? –  Peter Schuetze Feb 11 '10 at 14:33

2 Answers 2

We had similar behavior with when two apps where logging to the same file. Especially if one application rolls the logfile while the other app tries to write to the logfile it might happen that they get the possition in the files mixed up.

So make sure that only one process writes to each file. Also avoid that two different threads within one process write directly to the file. It only asks for trouble if two apps write to the same file.

share|improve this answer

I'm not sure there is enough information to answer this question. Have you tried writing large amounts of data from your various servers to the NFS mount, do those files exhibit corruption? If not this could be a websphere specific issue or an interaction between websphere and NFS. Maybe something in the way it writes data to log files....

Either way I wouldn't recommend logging to an NFS mount if at all possible, you would be much better served with a centralized syslog-ng daemon with logging over tcp via the local syslog daemon. In general this would be a more flexible solution, would allow for greater redundancy and remove a lot of the complexity inherent in NFS.

If this architectural change isn't possible then I would first rule out the NFS server itself, before you bring Websphere into the equation.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.