223

Fact is? At this point, there's no simple/easy automatic fix for this. Data recovery is a science and even the basic, common tools need someone to sit down and ensure the data is there. If you're expecting to recover from this without massive amounts of downtime, you're going to be disappointed. I'd suggest using testdisk or some file system specific ...


95

Boot into the rescue system provided by Hetzner and check what damage you have done. Transfer out any files to a safe location and redeploy the server afterwards. I'm afraid that is the best solution in your case.


92

When you delete stuff with rm -rf --no-preserve-root, its nigh impossible to recover. It's very likely you've lost all the important files. As @faker said in his answer, the best course of action is to transfer the files to a safe location and redeploy the server afterwards. To avoid similar situations in future, I'd suggest you: Take backups weekly, or ...


47

I've had the same issue but just testing with a harddrive, I've lost everything. I don't know if it'll be useful but don't install anything, don't overwrite your data, you need to mount your hard drives and launch some forensics tools such us autopsy, photorec, Testdisk. I strongly recommend Testdisk, with some basics command you can recover your data if ...


39

SUCCESS! I was able to retrieve the private key. But it wasn't easy. Here's what you need to do: Make sure you do not restart the server or Apache. The game is over at that point. That also means making sure that no monitoring services restart Apache. Grab this file - source code for a tool named passe-partout. Extract the source code and adjust line 9 of ...


36

Easy enough. The // sequence isn't a comment in bash (# is). The statement OUT_DIR=x // text had no effect* except a cryptic error message. Thus, with the OUT_DIR being an empty string, one of the commands eventually executed was rm -rf /*. Some directories placed directly underneath / weren't removed due to user not having permissions, but it appears that ...


34

Sure. I've had battery-backed cache (BBWC) and later flash-backed write cache (FBWC) protect in-flight data following crashes and sudden power loss. On HP ProLiant servers, the typical message is: POST Error: 1792-Drive Array Reports Valid Data Found in Array Accelerator Which means, "Hey, there's data in the write cache that survived the reboot/power-...


33

The best way to fix a problem like this is to not have it in the first place. Do not manually enter an "rm -rf" command that has a slash in the argument list. (Putting such commands in a shell script with really good validation/sanity routines to protect you from doing something stupid is different.) Just don't do it. Ever. If you think you need ...


19

Recovering individual disks will only get you partial data. The "independent" means that the disk drives are not aware of each other, and can be exchanged separately. The best way to recover data from a RAID is to use the software or hardware that was used to build it, and if that is not an option, a tool that understands the superblock format ...


15

I would try to recover backup machine, where all copies were stored: 1st step - Make a backup of this erased "backup machine" drives with dd comand. 2nd step - Use testdisk to recover files. So lets say you want to recover 1TB, You will need extra 2TB, 1TB for backup (1st step) plus 1TB for recovery (2nd step). I did similar mistake with alias rm -fr [...


14

There is nothing special. Let's assume your@id.here is your ID.: Export keys and ownertrust: gpg --export --armor your@id.here > your@id.here.pub.asc gpg --export-secret-keys --armor your@id.here > your@id.here.priv.asc gpg --export-secret-subkeys --armor your@id.here > your@id.here.sub_priv.asc gpg --export-ownertrust > ownertrust.txt Import ...


13

The short answer is that it depends. In the situation you describe (a faulty disk + some unreadable sectors on another disk) some enterprise RAID controllers will nuke the entire array on the grounds that its integrity is compromised and so the only safe action is to restore from backup. Some other controllers (most notably from LSI) will instead puncture ...


11

How can I figure out how long it will take to create the new EBS volume? Create one. And then, try using it. Continue using it over a period of hours and days, and note what you observe. The first answer to your question is that it actually only takes a few seconds. The problem with that answer is that it doesn't tell the whole story: New volumes ...


10

Yes, had that case. Server "without UPS" in a data center (with the data center having a UPS). PDU failure - system crashed hard. No data loss. And that basically is it. The good thing about a BBWC is that it is in the machine. Have a UPS - believe me, sometimes someone does something stupid (like pulling the wrong cable). A UPS is external. Oh, THAT cable ...


8

Most likely it is storing the key in memory, which it does because it needs to keep a copy after it drops privileges and/or decrypts the key using a supplied passphrase. In theory, you could get it out of the process image if you attached a debugger, though if they are following best practices it will be encrypted against something in memory. That said, if ...


8

SAN to SAN replication is your best bet for bringing the file server back online as quickly as possible with a little loss as possible after declaring a disaster. Please note that this type of DR protection doesn't protect from the same things as local backups- you can't use a replicated SAN volume to, for example, undelete a file from last month. Corrupted ...


8

1) He erroneously assumed that // was a bash comment. It is not, only # is. The shell interpreted // text as a normal command, and did not find a binary called //, and did nothing. In bash, when you have a variable assignment (OUT_DIR=/data/backup/mongod/tmp) directly preceding a command (// text), it only sets the variable while running the command. ...


7

As mentioned in another answer, Hetzner has a rescue system. It includes both a netboot option with ssh access as well as a java applet to give you screen and keyboard on your vserver. If you want to recover as much as possible, reboot the server into the netboot system and then log in and download an image of the filesystem by reading from the appropriate ...


7

It might be wise to add the intr option to any /etc/fstab entries that might end up hanging or crashing. If you don't use the soft or intr options, then when the server hosting the NFS files goes down, the server on which the files are mounted (the client) may hang when booting up. According to man 5 nfs: soft / hard Determines the recovery behavior ...


6

Turned out this is a lot easier than I expected. Any Exchange 2013 database can be mounted on any Exchange 2013 server, regardless of the organization and/or Active Directory domain (although the same or higher CU level is probably required for the server). If a foreign database is mounted on a server, it will work perfectly and can even be used to store ...


6

This is an article where you can learn more about the main idea behind Storage Replica, all its prerequisites and features: https://technet.microsoft.com/en-us/windows-server-docs/storage/storage-replica/storage-replica-overview And here you can find the guide on how to implement volume replication with stretched cluster: https://www.starwindsoftware.com/...


6

Snapshots aren't backups! You really need to follow 3-2-1 backup rule to ensure your data is safe. To accelerate your tape backup and recovery process I'd suggest to go from Disk-to-Tape to Disk-to-Disk-to-Tape: deploy some VTL offloading "cold" backups to either physical tape or AWS S3/Glacier.


5

In addition to @serghei's answer, check the documentation of gnupg. It says that you should backup: ~/.gnupg/gpg.conf (standard configuration file) ~/.gnupg/pubring.gpg (legacy public keyring) ~/.gnupg/pubring.kbx (new public keyring using keybox format) ~/.gnupg/openpgp-revocs.d/ (revocation certificates) It suggests also to backup the ownertrust gpg --...


5

Is their any disaster recovery technique to make redundancy between blades? Replace "Blade" with "Server" and you get the answer. Blades are independent servers. Standard technologies do not become invalid just because you pack them dense into a blade center. Can it be possible- Memory and storage sharing between blades? Yes, it is possible - shared ...


4

You can and should restore it from backup.


4

This seems to necessitate a second answer to the question... I just had a standalone VMware ESXi host lose a drive in a RAID 5 array. The degraded array impacted performance at the VM and application level. Smart Array P410i in Slot 0 (Embedded) (sn: 5001438011138950) array A (SAS, Unused Space: 0 MB) logicaldrive 1 (1.6 TB, RAID 5, ...


4

I've had 2 cases where battery backed cache in HW RAID controllers failed completely (in 2 separate companies). BBC relies on the unsurprising idea that battery works. The catch is that at some point battery in controller fails and what's devastating is that in many HW raid controllers it fails silently. We thought we had a cache protected against power ...


4

A lot of people tend to over-think this. Just think of these servers as if they were deployed in a colo or in a corporate datacenter. In that case, how would you back them up? Likely it would be via a "legacy" backup product (Netbackup, Amanda, BareOS, etc.) that is connected to a tape library or VTL. This is something you should consider doing for your ...


4

I've just moved away from a DFS-R environment because of the very reason you described above. Locked files are impossible to deal with and causes all kinds of conflicts especially if both servers are being used like a proper failover (so users are hitting both servers at once). To me, DFS-R is decent for replicating over WAN/VPN connections to remote ...


4

I did extensive tests with scenerio's like you describe. I tried having a storage server with failover capability using DRBD, then using iSCSI to attach that storage to Debian machines running Xen. I quickly gave up on that, because I had too many problems. Part of those could be me, though. One of them was that the iSCSI block devices weren't created. Then ...


Only top voted, non community-wiki answers of a minimum length are eligible