from comments:
i made an Ami from the broken instance and then used to launch a new instance
Whatever was broken with the original instance is likely to be broken in the replacement instance, so at least that part seems consistent. If you had made a new instance from the original AMI, that would be much more difficult to explain.
Note to self -- for the future -- once you get an instance up and tested and have verified that it's rebootable, stop it, and snapshot it while it's still working, make an AMI, make an instance from the AMI, and verify that the process is solid from end to end.
But, the short answer to your question should be yes, there should be another means of recovery other than throwing everything away and starting over... but cloning the broken machine should be pretty much expected not to work, depending on why the original machine isn't working.
If you still have the original instance available, that would be the best place to start troubleshooting, since there are additional things that could be broken on the new one.
Of course, if you have only "rebooted" the instance, not actually "stopped" and "started" the instance, you should also try that... because stopping an instance destroys it on the physical host where it's running and starts it up again on a new physical host. In the unlikely event that there was a problem there, this step could clear that problem, though I've not ever experienced this.
Otherwise, start up the instance, and give it a little bit of time to try to come up. Then, from "Instances" in the EC2 console, you should be able to select the instance, click the "Actions" drop-down, and choose "Get System Log" to retrieve the console output from the machine to see why it's not starting, or what it's doing.
The problem you are having almost has to be one of just a few things:
- corruption of your root filesystem
- root filesystem is fine but critical files have been inadvertently deleted
- root filesystem is fine and files are fine but misconfiguration is preventing a full boot-up
- instance is actually booting all the way up just fine, but something is misconfigured that's preventing you from accessing it -- like the configuration of sshd, or iptables, or the route table
- incompatibility between the instance's low-level kernel and your installation (should be unlikely if the instance was initially working)
Assuming there's not a simple problem with your security group configuration that's preventing access, you should see something in the system log that explains or at least gives a hint as to why your instance isn't coming up... it's hard to speculate which of the above causes would be most likely.
You should be able to identify something from the console log, though... you didn't mention in the original question what the two status checks in the console were showing, or whether you've opened up "ICMP Echo Request" in the security group, which would allow you to ping the instance.
If you can find a problem in the console log, then the question becomes whether you would know what to do to fix this problem if you had the same problem on a physical server.
If so, then you should be able to fix the problem by connecting to, and editing/fixing files on, the actual disk volume that holds the root partition of the broken instance:
- stop the broken instance
- detach the root EBS volume in the console
- create a new instance from one of the AWS-provided stock Linux AMIs.
- allow it to boot, and verify connectivity (if no connectivity here, either, then you definitely have something else going on)
- attach the broken instance's root volume to an available virtual device on the new instance, in the console
- log in to the new instance
- mount the old instance's root filesystem somewhere convenient, like /mnt
- modify the configuration to fix whatever was broken, by editing /etc/fstab or /etc/ssh/sshd_config, or disabling or correcting the configuration of a misbehaving service, etc.
- unmount the volume
- detach the volume, in the console, and reattach it to the broken instance
- restart the broken instance
Of course, this will involve having the expertise to find and fix the problem you've encountered... but I have successfully rescued a machine that failed to boot up due to a misconfiguration by using these steps... similar to what I would do if I had a physical machine and a rescue CD full of command line tools.
