Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

by mistake, i executed some applications that used all memory (and i think swap) on my ubuntu server and its now crashed, SSH doesn't work and freezes. Do you know any other options other than following solutions:

  1. physically restart the server.
  2. wait until a process ends.

is there any way to remotely restart the server when ssh not working? i can still ping the server, so wondering if any reserved memory is there for killing unfriendly processes or for basic commands such as restarting the Os.

*The commands executed with "nohup" so they didn't end by closing ssh sessions.

share|improve this question
add comment

2 Answers

up vote 5 down vote accepted

No, your system is... pretty much done. Your only hopes are waiting for one to end, or having one get so large that the OOM killer sweeps in and does something about it.

share|improve this answer
    
thank you, so the only practical option is physical restart, otherwise i might come up with 100* years waiting... –  Ehsan Sep 23 '11 at 17:19
add comment

It sounds like you system is doing what is called thrashing. What that means is your system is swapping pages in and out of memory for in use systems. That includes SSH.

You actually have a couple of options. The first option is of course to reboot the server, but it sounds like that is not much of an option for you because it's not local to you. If it's at a datacenter then you may be able to get them to walk up to it and reboot it. You have to check with your DataCenter.

Second option is also pretty simple. Your SSH session just hangs waiting for some stuff to be swapped out so it can allocate a tty for you. I have had this happen to me before, and you just have to wait for it to respond, it can take quite a while. Once you have a cli then you can kill or reboot the server.

The reason you can PING the server is actually pretty simple, the kernel's IP stack (IIRC) does not get swapped out that is why ICMP is able to respond.

A third option would require some setup before hand so I dont think it's a real option for you right now. You can setup a modem to allow you to dial in to a number and get a serial console. On the same note you can use IPMI Serial over LAN (SOL) to get a serial console. Also IPMI can allow you to reboot the server. Once again it sounds like those are not setup for you.

share|improve this answer
    
Thrashing can cause ssh to wait for so long that it times out before the password can be prompted for. Been there, done that. Waiting for it is not a reliable option. –  Ignacio Vazquez-Abrams Sep 23 '11 at 2:23
    
I didnt say it was a reliable option. Also a couple of changes in your ssh client config can deal with that as well. –  Squidly Sep 23 '11 at 2:41
    
Changes on the client won't alter how long the server waits for a password and I think that's the real stumbling block. –  John Gardeniers Sep 23 '11 at 5:17
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.