Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Just last night, ran a yum update on a couple of RHEL servers, I did manage to get a list of the packages but at least one of them seems to have caused a big problem.

The issue is that SSH to both servers seems to "drop" randomly. It doesn't actually disconnect, it just freezes. The SSH log on the server side looks normal, even with DEBUG LogLevel set. Sometimes it hangs just after the "Last login:" line appears, other times it will continue for a while but it eventually always hangs (even with constant activity happening). I initially thought this might be MTU or something, but nothing has changed, and other servers running CentOS as opposed to RHEL, but which didn't get a yum update yesterday are perfectly fine over the same VPN connection.

My problem is that I don't seem to have a way to fix this, I've tried to rollback the yum transaction, but SSH freezes during it and causes it to stop working. What's weird is that even if I manage to get screen open, and set yum running doing the rollback, the rollback will actually stop just like SSH has frozen, yet I can still detatch from the screen. This causes yum to appear as sleeping in the process list and I have to actually kill it after forcing the screen session to terminate. Any suggestions would be appreciated.

EDIT: Both servers are running RHEL 6.3

EDIT2: Some further information... It seems like the SSH connection is still active, because I still get messages from syslog and such on a connection I thought was dead:

[root@www www]# ls -la
^C^C
Message from syslogd@www at Feb 16 20:18:21 ...
   Message text
^C^C

There was about an hour between my first two Ctrl + Cs and the message popping up, and then you can see the Ctrl + Cs after didn't do anything either. It's just sort of... stuck.

EDIT3: I've tried reverting to the earlier Kernel by changing the GRUB default and the problem is not resolved.

EDIT4: Have some more information. After getting in via IPMI, I have the same problems, so it's not related to SSH, it is in fact another problem. Luckily though, I am now able to get the stack trace which is identical and hangs at the same place on both machines which are completely separate from eachother (although the hardware is the same).

https://dl.dropbox.com/u/28404486/bash_hang.jpg

share|improve this question
    
You've omitted the version(s) of RHEL involved here. –  ewwhite Feb 17 '13 at 1:26
    
Thanks, updated. –  user160357 Feb 17 '13 at 1:28
1  
This sounds like it's yum or your other processes getting hung up, rather than your connection. Can you suspend it with Ctrl-Z? –  Michael Hampton Feb 17 '13 at 3:00
    
Have you tried with tmux instead of screen? You never know. (I'm a tmux convert, having used screen for years until early last year). Have you tried debugging the yum transaction and looking for any particular calls where a child (uninstall?) process is hanging? –  Christopher Woods Feb 17 '13 at 5:51
    
Have you checked your server's average load? –  fboaventura Feb 17 '13 at 11:34

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.