1

I need some advice on troubleshooting an NFS server problem on Scientific Linux (RHEL) 6.1. The log on the server shows that an authenticated mount request was made:

Jan 13 16:30:02 ??? rpc.mountd[3996]: authenticated mount request from ????:784 for /shared-storage/cm/shared (/shared-storage/cm/shared)

But after that, it does not continue. On the client, it is also hanging. The interesting thing now is that I have two NFS servers, which should be identical, and the one is working perfectly, but the other exhibits the above mentioned behaviour. The problem is also not completely persistent, i. e. sometimes the mount request succeeds.

I assume that the problem must be related to the server rather than to the client, because it is working perfectly on the other server. My question is where I should search the problem. I have already re-created the exports using exportfs -r, I have restarted the NFS server, I have compared the rpcinfo outputs of both server - no success. The problem even survives a reboot. Any other ideas are appreciated.

As answer to Tim's question: I have sporadically the following in dmesg, but do not know whether it is related

e1000e 0000:0c:00.0: eth4: Detected Hardware Unit Hang:
  TDH                  <24>
  TDT                  <25>
  next_to_use          <25>
  next_to_clean        <24>
buffer_info[next_to_clean]:
  time_stamp           <1c3d12940>
  next_to_watch        <24>
  jiffies              <1c3d12940>
  next_to_watch.status <0>
MAC Status             <80383>
PHY Status             <792d>
PHY 1000BASE-T Status  <7800>
PHY Extended Status    <3000>
PCI Status             <10>

Further edit: The problem above does not occur on the machine that is working, so it probably is related.

Again an edit: The error is not on the (software) device that is used for NFS, but on another one. The NFS mount also does not trigger the message.

Improve this question
5
  • anything in syslog or dmesg that seems suspicious? wonder if there is hardware trouble on the misbehaving machine
    – Tim
    Jan 13, 2012 at 16:27
  • That's an idea. Possibly in dmesg: "e1000e 0000:0c:00.0: eth4: Detected Hardware Unit Hang:" - "e1000e" is the Intel NIC.
    – Christoph
    Jan 13, 2012 at 17:51
  • @Tim: Post an answer that I can vote for it if your are right...
    – Christoph
    Jan 13, 2012 at 17:56
  • Created an answer, good luck!
    – Tim
    Jan 13, 2012 at 19:00
  • Also added a hopefully relevant link, from our good friends at ServerFault ;)
    – Tim
    Jan 13, 2012 at 19:07

2 Answers 2

Reset to default
0

Anything in syslog or dmesg that seems suspicious? I am curious if there is hardware trouble on the misbehaving system.

Edit, curious about your error you saw in dmesg, and found the same error mentioned here: Linux e1000e (Intel networking driver) problems galore, where do I start?

From all of the debugging output the OP posted, I was SURE that his hardware was about dead, apparently there was a kernel parameter to fix the issue: pcie_aspm=off

You can try booting with that parameter and see if it fixes things up!

Improve this answer
4
  • I was able to further investigate the effect this morning. The error message is on another device than the one serving NFS and the message also cannot be provoked vis NFS use (actually, it works this morning). I will nevertheless try pcie_aspm=off asap, but I cannot reboot the machine in question without notifying the users some time in advance...
    – Christoph
    Jan 16, 2012 at 9:21
  • I now have added pcie_aspm=off, which fixes the issue in dmesg, but unfortunately not the actual problem...
    – Christoph
    Jan 16, 2012 at 14:14
  • As I did not have another idea, I finally replaced the NIC in the machine for testing purposes. Now it is working again, so I think the hardware was actually broken although the other NIC produced the output in dmesg...
    – Christoph
    Jan 18, 2012 at 9:27
  • Interesting... Good rule to follow though, if it is acting up, and there is no logical explanation, assume the hardware is failing / defective.
    – Tim
    Jan 18, 2012 at 14:00
1

Make sure portmap is running on both the server AND the client.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .