Alright if anyone wants to play god and work miracles i'm down.

So, i was given the task of making a script that found files that are older than 6 months, zipped them and then deleted them. On my way in making tihs script i ran this :

find / -type f -mtime -400 ! -mtime -180 | xargs gzip blablabla

And that gave EVERY SINGLE FILE a .gz extension. Now i undid it as soon as i noticed but it was kinda too late. Upon completion of the command none of my bash commands would work because the $PATH variable emptied itself. I tried lots of things before realising what was the problem.

So ater unzipping everything i'm still unable to boot. I've managed to make it to grub rescue, after wich i followed online instructions for :

root (hd0,0)
setup (hd0)
kernel (hd0,0)/boot/vml[...]
initrd (hd0,0)/boot/initrd.im[...]

After wich my linux partially boots but gives me the following errors :

Begin : Running /scripts/init-bottom ... mount : mounting /dev on /root/dev failed : No such file or directory
mount: mounting /sys/ on /root/sys failed: No such file or directory
mount: mounting /proc on /root/proc failed : No such file or directory
Target filesystem doesn't have requrested /sbin/init.
No init found. Try passing init= bootarg.

I've tried repairing the filesystem, i've booted from 3 different LiveCDs/ Rescue disks, i ran boot-repair from 2 different dicsc. I did force fscks...

I'm really out of ideas and i NEED to get this server to at least boot so i can recover my SQL databases. I'm desperate for help, i'll even pay if need be.

I've been lurking forums for 3 days straight all day to find a possible solution and i'm still at the same point... Help please?

  • 3
    if they're mysql db's you don't necessarily need to boot; in that case I'd try to mount the drive as a slave and copy over the /var/lib/mysql directory – user16081-JoeT Oct 15 '13 at 19:54
  • 8
    Clean install on new storage devices. Mount the old drive, transfer data as required. I would bet that repairing will not be worth the effort. – Zoredache Oct 15 '13 at 19:56
  • 7
    This is the point where you restore from backup. And remember not to run around doing non-privileged actions as the root user next time. – Magellan Oct 15 '13 at 20:18
  • 1
    because of version differences, re-install with the exact same version. we have corruption issues, your data may be corrupt. Repairing the system so it is bootable, will not help you if the data has been trashed. If your gzip command compressed your database files while the database was being used corrupt seems inevitable. – Zoredache Oct 15 '13 at 20:34
  • 5
    If your DB software was running while you ran those commands, you might not be able to recover the DBs. Gzip would have happily zipped up the file, then it unlinked it. But your DB software still had the file open and it was committing the changes to it. As soon as it stopped, the file was then removed. – toppledwagon Oct 15 '13 at 20:34

This will depend on whether the filesystems are repaired enough for you to be able to mount those partitions from a LiveCD. Don't bother trying to boot the system yet. First, mount the partitions and unzip all the .gz files. This will give you working copies of init and system binaries. Then you can use grub to repair the boot sector. Then boot to single user mode and fsck the filesystem again. If that works, you'll have a running system. You'll also have a bunch of unzipped files (like man pages) that really should be zipped, but it's better than having an unbootable system.

If you can't mount the partitions from a LiveCD, then you are unfortunately out of luck. Nothing will recover your system at that point.

| improve this answer | |
  • 1
    This actually worked like a charm... i can't thank you enough for this! MySQL wouldn't boot up but i didn't do a --force fsck quite yet, hopefully it'll fix it! THANKS – Dexirian Oct 16 '13 at 14:51
  • 1
    Awesome. Glad it helped. – Michael Martinez Oct 16 '13 at 17:31

The first thing I'd try is running a LiveCD environment and just attempt to ungzip everything, hoping that would return the system to a bootable state. Note: I'd be concerned about potential data corruption if the original gzip process was interrupted.

Otherwise I'd try to migrate the DB to a new system as others have suggested but as you've encountered there may be labor intensive dependency and configuration issues that will need to be resolved individually.

| improve this answer | |
  • Quick question : We are unsure of the old SQL database server, and the newer server is using a different Linux Distro. The newer server is running CentOS with WHM, and the old server was either Debian / Unbuntu. So my question is, how can i effectively migrate my SQL databases without corruption and whatnot? – Dexirian Oct 16 '13 at 14:18

The general consensus here, that you should just mount the disk in a working system and rescue your files, is not wrong. It's the sensible thing to do. But the other way is more fun, and very educational. I've learned a lot while fighting my way out of messy situations where other people would have just given up and reinstalled from scratch. (Not on a server that other people are depending on though...)

Anyway, so far you have an initramfs(initrd) that runs. That's a good start. But it can't complete the handoff to init because init is now init.gz maybe? To make any progress, it would help to know exactly what Linux distribution you have, so we can look up what tools are available in its initramfs for emergency use.

The error messages you presented look like they could have come from Debian's initramfs. If it is Debian, then you should have gotten an (initramfs) shell prompt on the next line after the last error. If you did, you should look into what's going on with those failed mounts. is /root/dev missing? (/root is where your normal root fs should be mounted during the running of the initramfs)

If you didn't get the shell prompt, then what came after No init found. Try passing init= bootarg. will be interesting. Even if it was nothing but a blinking cursor, that's a clue. If it seems totally frozen, try to get some information on what processes are still around using magic sysrq or Ctrl+ScrollLock.

The Debian initramfs also lets you request a shell at a few special landmarks by adding a break= parameter to the kernel command line. For example, to get a shell before the Running /scripts/init-bottom line, use break=bottom.

Aside: I don't know how the find command could have gzipped every file... it looks correct to me for the purpose of selecting files between 180 and 400 days old.

| improve this answer | |
  • When i do an ls under /root there is nothing found. So i can take it the fs mounting isn't okay on boot? Where can i change it? – Dexirian Oct 16 '13 at 12:59
  • 1
    @Dexirian so you did get a shell prompt (did you have to use break=bottom?)... yes, by the time it's trying to mount /root/dev and /root/proc and /root/sys, /root should be the real root filesystem. There must have been an error message earlier, about failing to mount it. Did you include a root= parameter in the kernel command line? My memory is a little fuzzy on this point but I think the root (hd0,0) just tells grub where to find its support files, and you still need to tell the kernel separately where the root is. – user193597 Oct 16 '13 at 13:10
  • Yes i did use root=, kernel= initrd=, and setup=, i did not have to use break=bottom. And i did not spot an earlier failed mount message, as it scrolls really quickly – Dexirian Oct 16 '13 at 14:15
  • @Dexirian Is console scrollback available? Shift+PgUp. And can you mount it fromt he (initramfs) prompt, something like mount -r /dev/sda1 /root? cat /proc/partitions to see what disks are available. – user193597 Oct 16 '13 at 14:34
  • 1
    Welcome to the world of system administration. – Michael Martinez Oct 16 '13 at 17:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.