Just last night, ran a yum update on a couple of RHEL servers, I did manage to get a list of the packages but at least one of them seems to have caused a big problem.
The issue is that SSH to both servers seems to "drop" randomly. It doesn't actually disconnect, it just freezes. The SSH log on the server side looks normal, even with DEBUG LogLevel set. Sometimes it hangs just after the "Last login:" line appears, other times it will continue for a while but it eventually always hangs (even with constant activity happening). I initially thought this might be MTU or something, but nothing has changed, and other servers running CentOS as opposed to RHEL, but which didn't get a yum update yesterday are perfectly fine over the same VPN connection.
My problem is that I don't seem to have a way to fix this, I've tried to rollback the yum transaction, but SSH freezes during it and causes it to stop working. What's weird is that even if I manage to get screen open, and set yum running doing the rollback, the rollback will actually stop just like SSH has frozen, yet I can still detatch from the screen. This causes yum to appear as sleeping in the process list and I have to actually kill it after forcing the screen session to terminate. Any suggestions would be appreciated.
EDIT: Both servers are running RHEL 6.3
EDIT2: Some further information... It seems like the SSH connection is still active, because I still get messages from syslog and such on a connection I thought was dead:
[root@www www]# ls -la ^C^C Message from syslogd@www at Feb 16 20:18:21 ... Message text ^C^C
There was about an hour between my first two Ctrl + Cs and the message popping up, and then you can see the Ctrl + Cs after didn't do anything either. It's just sort of... stuck.
EDIT3: I've tried reverting to the earlier Kernel by changing the GRUB default and the problem is not resolved.
EDIT4: Have some more information. After getting in via IPMI, I have the same problems, so it's not related to SSH, it is in fact another problem. Luckily though, I am now able to get the stack trace which is identical and hangs at the same place on both machines which are completely separate from eachother (although the hardware is the same).