1

I've a Google Cloud VM instance which I restarted via SSH using the restart command. I waited for it to come back up, but it didn't seem to do so.

My GCC VM interface shows the VM is running, but it's not accessible even via webssh, or gcloud running locally, which does connect to the acccount successfully.

To test, I've created other VM instances.

Creating a VM instance using a snapshot of my disk puts me in the same situation with the new VM instance.

Creating a new VM instance from a fresh OS image and I have no connectivity issues at all. I can SSH between new VM instances, but can't SSH or even Telnet or ping to the instance I'm having trouble with.

Where can I go from here? Can I at least recover my data from the 'broken' VM instance's disk or snapshot?

Thanks.

EDIT: Here's the last few lines from the console, it seems there's a problem with the filesystem. It cannot mount ./ Can this be repaired?

 * Starting Mount filesystems on boot[74G[ OK ]

 * Starting Fix-up sensitive /proc filesystem entries[74G[ OK ]

 * Stopping Fix-up sensitive /proc filesystem entries[74G[ OK ]

 * Starting Populate /dev filesystem[74G[ OK ]

 * Starting Populate and link to /run filesystem[74G[ OK ]

 * Stopping Populate /dev filesystem[74G[ OK ]

 * Stopping Populate and link to /run filesystem[74G[ OK ]

 * Starting Initialize or finalize resolvconf[74G[ OK ]

[    9.452786] EXT4-fs (sda1): Couldn't remount RDWR because of unprocessed orphan inode list.  Please umount/remount instead
An error occurred while mounting /.

keys:Press S to skip mounting or M for manual recovery

 * Stopping Track if upstart is running in a container[74G[ OK ]

[    9.856747] random: landscape-sysin: uninitialized urandom read (32 bytes read, 30 bits of entropy available)
[    9.871893] random: landscape-sysin: uninitialized urandom read (32 bytes read, 30 bits of entropy available)
[   10.138464] random: lsb_release: uninitialized urandom read (24 bytes read, 33 bits of entropy available)
[   10.320672] random: mktemp: uninitialized urandom read (10 bytes read, 35 bits of entropy available)

EDIT: It seems the old VM's filesystem is corrupt. I've mounted the snapshot to a working VM and gotten this message: Thanks again. I've attached the disk and mounted it to another VM. I've ran fsck and gotten this message:

fsck /dev/sdb1 fsck from util-linux 2.27.1 e2fsck 1.42.13 (17-May-2015) cloudimg-rootfs: clean, 137997/1310720 files, 3602263/5242624 blocks

It was corrupt and I had to answer y to a lot of things. Hopefully it's repaired, I'll attempt to boot the vm from it.

0

Did you have the chance to interact with the instance through the serial console? Should this doesn't resolve your issue, you might want to attach the disk created from the snapshot to an accessible instance as secondary disk to retrieve your data or revert the latest changes on the affected disk.

6
  • Thanks for your response George. I wasn't aware the serial console was a two way street. I'll investigate. I did create a new VM using the snapshot, but that put me in the same situation, so it must be a config issue with my snapshot. I didn't know you could attach a 2nd disc to a working VM, I have a couple of working VMs now, so i'll try that route for data recovery. Thanks. Do you have any links to resources around attaching a secondary disk? – i-CONICA Oct 19 '16 at 18:38
  • I added the link in the answer, but this is the link again cloud.google.com/compute/docs/disks/add-persistent-disk. You can simply do it from the cloud console by editing your VM instance, access into the Vm and mount the disk. – George Oct 19 '16 at 18:55
  • Thanks, George. I seems the FS is corrupt. I've updated the question. I've mounted it as you suggested and ran fsck. I hope this is recoverable. – i-CONICA Oct 19 '16 at 19:57
  • If I make changes to the snapshot I've mounted, using fsck, parted or some other filesystem repair tool, will those changes reflect back if I try to boot the original VM from this now repaired snapshot? – i-CONICA Oct 19 '16 at 19:59
  • The changes will be reflected on the disk. Once it's repaired, you can detach it and recreate a new instance using the fixed disk as a Boot disk – George Oct 20 '16 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.