I had problems accessing my instance via ssh client and web browser, after i Re-booted it, and then i created a new instance with the Ami of the previous instance but still i could not access my instance.

Then i read What to do with suddenly unreachable non-logging EC2 instance? post

But what i learned from there was that a new instance must be created in order to recover the files on the server and all installation of packages needed must be re-installed

Is there not a way for me to solve the issue without creating a fresh new instance and making fresh installation of packages, because that would really take a lot of time doing it all over again?

Thanks in advance

  • "created a new instance with the AMI of the previous instance" -- does this mean that you used the original AMI, the same that you used to originally create the other instance? Or does it mean that you made an AMI from the broken instance and then used that to launch a new instance? – Michael - sqlbot Dec 3 '13 at 18:36
  • Yes i made an Ami from the broken instance and then used to launch a new instance – Html Tosin Dec 4 '13 at 0:21

from comments:

i made an Ami from the broken instance and then used to launch a new instance

Whatever was broken with the original instance is likely to be broken in the replacement instance, so at least that part seems consistent. If you had made a new instance from the original AMI, that would be much more difficult to explain.

Note to self -- for the future -- once you get an instance up and tested and have verified that it's rebootable, stop it, and snapshot it while it's still working, make an AMI, make an instance from the AMI, and verify that the process is solid from end to end.

But, the short answer to your question should be yes, there should be another means of recovery other than throwing everything away and starting over... but cloning the broken machine should be pretty much expected not to work, depending on why the original machine isn't working.

If you still have the original instance available, that would be the best place to start troubleshooting, since there are additional things that could be broken on the new one.

Of course, if you have only "rebooted" the instance, not actually "stopped" and "started" the instance, you should also try that... because stopping an instance destroys it on the physical host where it's running and starts it up again on a new physical host. In the unlikely event that there was a problem there, this step could clear that problem, though I've not ever experienced this.

Otherwise, start up the instance, and give it a little bit of time to try to come up. Then, from "Instances" in the EC2 console, you should be able to select the instance, click the "Actions" drop-down, and choose "Get System Log" to retrieve the console output from the machine to see why it's not starting, or what it's doing.

The problem you are having almost has to be one of just a few things:

  • corruption of your root filesystem
  • root filesystem is fine but critical files have been inadvertently deleted
  • root filesystem is fine and files are fine but misconfiguration is preventing a full boot-up
  • instance is actually booting all the way up just fine, but something is misconfigured that's preventing you from accessing it -- like the configuration of sshd, or iptables, or the route table
  • incompatibility between the instance's low-level kernel and your installation (should be unlikely if the instance was initially working)

Assuming there's not a simple problem with your security group configuration that's preventing access, you should see something in the system log that explains or at least gives a hint as to why your instance isn't coming up... it's hard to speculate which of the above causes would be most likely.

You should be able to identify something from the console log, though... you didn't mention in the original question what the two status checks in the console were showing, or whether you've opened up "ICMP Echo Request" in the security group, which would allow you to ping the instance.

If you can find a problem in the console log, then the question becomes whether you would know what to do to fix this problem if you had the same problem on a physical server.

If so, then you should be able to fix the problem by connecting to, and editing/fixing files on, the actual disk volume that holds the root partition of the broken instance:

  • stop the broken instance
  • detach the root EBS volume in the console
  • create a new instance from one of the AWS-provided stock Linux AMIs.
  • allow it to boot, and verify connectivity (if no connectivity here, either, then you definitely have something else going on)
  • attach the broken instance's root volume to an available virtual device on the new instance, in the console
  • log in to the new instance
  • mount the old instance's root filesystem somewhere convenient, like /mnt
  • modify the configuration to fix whatever was broken, by editing /etc/fstab or /etc/ssh/sshd_config, or disabling or correcting the configuration of a misbehaving service, etc.
  • unmount the volume
  • detach the volume, in the console, and reattach it to the broken instance
  • restart the broken instance

Of course, this will involve having the expertise to find and fix the problem you've encountered... but I have successfully rescued a machine that failed to boot up due to a misconfiguration by using these steps... similar to what I would do if I had a physical machine and a rescue CD full of command line tools.

| improve this answer | |
  • Thks but i already tried stopping and starting the instance even before i made the post and it was still the same, i still couldn't connect :-( – Html Tosin Dec 4 '13 at 14:41
  • How about the rest? Anything interesting in the console log? – Michael - sqlbot Dec 4 '13 at 17:47
  • All i found in the error log was the errors generated from the some invalid mistyped linux commands i typed when the disk was working properly a week ago. No info about reasons for disk issue in the log – Html Tosin Dec 5 '13 at 4:39
  • Do you have a set /list of command line tools to rescue a linux machine, maybe that could help – Html Tosin Dec 5 '13 at 4:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.