Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a lot of data (svn repository and postgresql database) in EC2 instance, yesterday the server doesn't respond to http requests, and I `sudo reboot', after that the server couldn't boot up again, what should I do?

The system log in the AWS console shows nothing output from the instance, and the most recent sys log is:

Ubuntu 9.10 domU-12-31-39-00-A0-06 tty1 

domU-12-31-39-00-A0-06 login: [1796075.232506] INFO: task rsyslogd:22423 blocked for more than 120 seconds. 

[1796075.232522] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.232509] INFO: task pdflush:17 blocked for more than 120 seconds. 

[1796195.232525] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.232639] INFO: task kjournald:296 blocked for more than 120 seconds. 

[1796195.232646] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.232728] INFO: task rsyslogd:22423 blocked for more than 120 seconds. 

[1796195.232735] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.232858] INFO: task amuled:874 blocked for more than 120 seconds. 

[1796195.232865] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.232962] INFO: task master:1716 blocked for more than 120 seconds. 

[1796195.232968] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.233065] INFO: task qmgr:1724 blocked for more than 120 seconds. 

[1796195.233071] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.233190] INFO: task java:4554 blocked for more than 120 seconds. 

[1796195.233196] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.233296] INFO: task java:4745 blocked for more than 120 seconds. 

[1796195.233302] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

[1796195.233353] INFO: task java:11614 blocked for more than 120 seconds. 

[1796195.233360] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 

It seems I can't do anything now, I can't SSH into it, and can't operate in single user mode, too. Any idea??

share|improve this question

2 Answers 2

up vote 5 down vote accepted

EC2 instances cannot be booted into safe mode.

What is the output of ec2-get-console-output [1] for that instance? It might contain clues as to what's preventing the instance from booting properly.

If this is an EBS-backed instance you'll be able to "stop" the instance, detach the root EBS volume from it, attach the root EBS volume to another instance and make changs to the contents, then reattach it to the original instance and "start" it running again.

If this is an instance-store instance then you're out of luck - there's no way to go in and fix one of those puppies.

[1] from the EC2 API Command Line Tools: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351

share|improve this answer

Actually... an EC2 Linux instance can be booted into safe mode, I've got one stuck there now. Its just not something you'd ever want to do. Apparently 14.04 doesn't play nice with the 'reboot now' command. This tells the server to restart in single user mode which requires console action. The AWS web gui is hardwired to use this command. Because you don't have access to the console you're effectively screwed. I'm adding this note so someone doesn't take your statement as a challenge and get stuck in the same hole I fell down.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.