Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [disaster-recovery]

Disaster recovery and preparedness is an unfortunate aspect of systems administration. This tag should be used for help with planning, implementation and best-practices related to recovering from a catastrophic event on a server or in a datacenter environment.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

Quorum at Scylla. DR orchestrator

I have a strech cluster with Scylla for 2 data centers with a balancer. The principle of operation is Every_Quirum. (After reading the documentation, I came to the conclusion that all DPCs should work)...
user avatar
1 vote
2 answers
76 views

GCP disk snapshots: Would losing the full snapshot render the incremental snapshots useless?

On GCP, when you do periodic snapshots of disks, the first snapshot is a full snapshot of the disk and the following snapshots consists of only the deltas of the previous snapshots. If a snapshot is ...
user avatar
  • 145
0 votes
1 answer
75 views

lvm recovery with raid reconfiguration

I have a Dell R510 with a H200 controller previously and I have two disks: a 2T SAS and a 250G SSD. I created two raid0s using those two disks on H200 and then created lvm on top of it. Here is my ...
user avatar
2 votes
1 answer
48 views

ESXi Host backup including extra vibs

I have a licensed ESXI Host with a couple of extra vibs (Chelsio, NVidia, dell) installed. I also have vcenter standard. More that I like to I discover that I have to reinstall the host, e.g. VMware ...
user avatar
  • 21
0 votes
1 answer
87 views

Disaster Recovery: PXE boot Windows Server 2019 from ISO

I have a dedicated physical Windows Server 2019 on the Ionos cloud platform. I need to implement a disaster recovery plan. I also have added a shared storage block on another Ionos server and mapped ...
user avatar
0 votes
0 answers
30 views

How handle chaos with service less pods across multi cluster

I’ve two cluster on different region, I want to deploy an app that doesn’t need service. How can I deploy this app on one cluster and if something goes wrong on this cluster (region offline, cluster ...
user avatar
  • 1
0 votes
0 answers
12 views

Recovering image using sysrcd after boot failure?

I have a dedicated server was running centOS with CentOS Web Panel, I've moved some /boot files and rebooted the server and it did not reboot. The server is unmanaged, sent a ticket for the hosting ...
user avatar
  • 1
2 votes
3 answers
1k views

Need to recover RAID 6 array

By definition RAID 6 is an array of independent hard drives equipped with two independent and distributed checksum schemes Does that mean I can recover each of the hard drives one by one using ...
user avatar
0 votes
0 answers
73 views

Rescue Tip: Wrong library file installation damaged Ubuntu 18.04 after upgrade

There was an old Ubuntu 14.04 linode.com instance which I attempted to upgrade to 18.04. Some Dependencies where somehow not upgraded anyway. So I attempted to install corresponding deb file from ...
user avatar
  • 33
8 votes
1 answer
2k views

Does one failed drive + one single bad sector destroy an entire RAID 5?

During planning my RAID setup on a Synology Disk Station I've done a lot of reading about various RAID types, being this a great reading: RAID levels and the importance of URE (Unrecoverable Read ...
user avatar
0 votes
1 answer
56 views

If I image a virtual machine with SQL server installed will the database work on a new instance?

On AWS I've always imaged machines that contained SQL Server installations and running databases. When I spin up new instances from the images, SQL Server and all database always returned without ...
user avatar
0 votes
1 answer
57 views

desperate attempt to head off RAID 0 failure, can i copy a disk with dd?

A colleague has been using a 10TB RAID 0 array to edit a film. He suddenly realized it's 98% full. The array has started seriously acting up, but cooperates grudgingly if you shut down all processes ...
user avatar
  • 506
2 votes
1 answer
90 views

Systemd and Disaster-Recovery Stand-By systems

We're using systemd to run various services in production. (Duh...) We're building out a matching "disaster-recovery" site, which will have the same application installed -- with the same ...
user avatar
  • 2,242
0 votes
2 answers
41 views

Cloud definition met? [closed]

note: question has been reviewed after suggestions it could be opinion based We recently suffered an outage due to a popular big provider's datacenter fire. Our public cloud instances hosted in that ...
user avatar
  • 1,679
0 votes
1 answer
26 views

Disaster recovery process document for Google cloud Virtual Machine

Hello we have been directed by auditors to provide disaster recovery document for Google Cloud Virtual Machine. Please Is this provided by Google? If so how can I get it. I would appreciate resources ...
user avatar
1 vote
1 answer
306 views

Safe to delete managed incremental snapshots on Azure?

Incremental snapshots of managed disks on Azure is a relatively new thing, and is not completely documented, so I have a crucial question: If I have incremental snapshots A, B, C and D of a disk (the ...
user avatar
0 votes
0 answers
23 views

How to make backup for the Local Server

I have critical server that running the local company system, that have system backup and company files and business documents with critical and important severity. I'm planning to create a Network ...
user avatar
1 vote
0 answers
56 views

Setting UP disaster recovery , one region to multi region in Google Cloud

I'm new to GCP and also learning about Multi-Regional architectures . We already have single region ( but multi zonal ) infrastructure setup. DNS names of the internal components do not have region/...
user avatar
  • 119
0 votes
1 answer
116 views

promox is blocked from fully booting by ifupdown2 start job

i have a promox server with ifupdown2 installed. recently when trying to reboot the entire server promox failed to get to console, it gets stuck on replication runner failing and the ifupdown2 service ...
user avatar
  • 111
1 vote
0 answers
129 views

Is it possible to backup and restore a Netbackup Master server using Windows Server Backup?

I have installed Netbackup 8.2 master server on Windows Server 2019 system. I read this document , which says you neet to re-install OS and Netbackup itself, then the catalog in order to restore ...
user avatar
0 votes
0 answers
123 views

Can't reconnect VPS Essential OVH after removing set_hostname from cloud-init

I have a VPS Essential in OVH with Ubuntu 18.04 LTS installed. I just removed the set_hostname module of the cloud_init_modules from /etc/cloud/cloud.cfg and reboot the system and now I cannot connect ...
user avatar
  • 113
1 vote
1 answer
126 views

How can I recover data from a five disk raid 5 on HP Proliant DL 380 [closed]

How can I recover data from a five disk RAID 5 on HP Proliant DL 380 where four of the five disks are reporting a SMART error. I have some critical data on the failed logical drive and checking online ...
user avatar
0 votes
1 answer
107 views

Azure VM off-site backups via CLI

I'm currently backing up Azure VMs via Azure CLI: create resource group: az group create -n backup-resource-group -l uksouth create recovery services vault: az backup vault create --resource-group ...
user avatar
  • 163
7 votes
2 answers
1k views

In 2020 - are there any viable Linux block-level replication alternatives for DRBD? [closed]

I'm researching how can we implement near-realtime replication from primary datacenter to a disaster recovery site. Data that would get replicated would be: Images of KVM VMs MySQL and PostgreSQL ...
user avatar
  • 29.4k
1 vote
1 answer
409 views

Windows 2019 backup: bare metal restore

I'm rehearsing a virtual machine bare metal restore. Backups are taken daily and saved on a secure external server. We are assuming a total disaster, so bad that no domain controller is available, so, ...
user avatar
  • 368
0 votes
2 answers
323 views

Copy binary file from other system with builtin shell commands only

I accidentally just have removed critical shared library and no one dynamically linked executable can't be run now. For example, even /bin/ls shows error while loading shared libraries: libunwind.so....
user avatar
0 votes
1 answer
108 views

AWS DR Setup and runbook

I'm working on creating a DR setup and runbook based on AWS. I don't have any experience with creating DR setups so it would be really helpful if the experienced veterans can guide me through it. ...
user avatar
  • 183
1 vote
0 answers
210 views

Zerto Failover - Win 2012/16 Servers Get Stuck on "Getting Devices Ready"

We are using Zerto to replicate our VMs to an offsite datacenter. During our failover tests the VMs have a tendency to "hang" from 15 minutes to several hours on "Getting Devices Ready." I have read ...
user avatar
  • 11
-1 votes
1 answer
848 views

Does Barman need ssh connection setup when installed on the same host as the PostgreSQL host?

I'm trying to configure Barman (pgbarman) to work alongside the PostgreSQL database on the same host, i don't want to install barman on a separated host, but I can't find any documentation about such ...
user avatar
  • 1
6 votes
1 answer
184 views

Recovering lost VHDX / VM (deleted by Veeam)

Earlier this week a VM on one of our hypervisors experienced extended downtime (~24 hours) due to some Windows updates going wrong. I ultimately was able to fix the issue, and noticed yesterday that ...
user avatar
0 votes
1 answer
629 views

Data recovery failure from Azure Data Recovery Vault using MARS Agent with on-premises computer

Despite trying several times since last night, the recovery of a volume from an Azure Data Recovery Vault to a new location on the original source computer fails after approximately (but not exactly) ...
user avatar
  • 141
0 votes
0 answers
129 views

moving raid 5 hard disks from ibm x3500 m1 to ibm x3650 m1

one of our server ibm x3500 m1 type 7977 dead yesterday. can we move its hard drive to access data on raid 5 logical drive on ibm x3650 m1 server with same serveRaid 8K raid controller? Usman
user avatar
9 votes
3 answers
3k views

Database accidentally deleted with a bash script [duplicate]

Edit: a follow-up question: Restore mongoDB by --repair and WiredTiger. My developer committed a huge mistake and we cannot find our Mongo database anywhere in the server. He logged into the server, ...
user avatar
  • 287
1 vote
2 answers
1k views

BTRFS unmountable after cold reboot (total_rw_bytes is twice too big)

One of my users in research environment invoked out-of-memory on a server which mounts a 52TB btrfs partition. I had to power cycle the server. After the reboot my btrfs partition cannot be mounted in ...
user avatar
  • 373
1 vote
1 answer
60 views

Azure VM Site Replication

I'm looking to set up site replication for our Azure VM's. My question is this, how often are the VM's backed up/replicated to the DR site? Is this replication automated or would I need to kick off ...
user avatar
  • 578
0 votes
1 answer
68 views

What happens to data in a GCP project if all owners or the project itself is suspended?

I have realized recently that a personal domain on google apps for business got suspended (the free for life version that was available in beta). This actually happened like 6 months ago and I didn't ...
user avatar
  • 103
2 votes
1 answer
2k views

Recover ZFS data

Context/Hardware: HP microserver gen8 1x1TB - standalone, 2x4TB Raid 1x16GB iLO SDCARD with Debian + OpenMediaVault Event: SDCARD failure restarted server and installed Ubuntu on 1TB drive ...
user avatar
1 vote
1 answer
155 views

btrfs recover: reliable or last-straw?

The chunk tree of a btrfs filesystem corrupted and I could not recreate it after hour-long reconstruction. As a last resort (besides restoring from backup) I could use btrfs restore -S -x -m -v to get ...
user avatar
1 vote
0 answers
60 views

Recovering VCS 2 node cluster in DR?

We are looking for a solution to recover the 2 node VCS Cluster at DR location for a DR Test. Prod: Hardware is Dell with 24 cores 64 GB RAM with Suse 11.4 Linux over SAN Boot & 4 NIC's of 1 Gig. ...
user avatar
  • 87
0 votes
1 answer
55 views

Can I shadowcopy/backup the AD search index?

We have a 1TB+ search index in our Server 2012 Active Directory (6.1) domain. This is located on a NAS, so rebuilding it takes "forever". For DR purposes, is there a way to correctly back up the ...
user avatar
  • 146
1 vote
0 answers
89 views

Tried to recover a Ext4 Superblock to a CEPH disk

We have a server with Proxmox installed with Debian, it's an dedicated server (sda). The VM's are being backed up every night on a CEPH disk (sdb). After an reboot of the dedicated, the ext4 drive ...
user avatar
  • 11
0 votes
0 answers
32 views

Possibility to recover from majority of C:\windows being deleted

So I have an interesting and somewhat fun scenario. A developer was working on some scripts that have worked in the past when run by him which deleted files in the working directory, but supposedly ...
user avatar
1 vote
1 answer
176 views

Can I restore a SQL Server instance from file backups only?

Our windows 2016 server failed to restart after a windows update yesterday. In order to avoid the update that crashed it issue (whatever it was), I'm in the process of rebuilding it clean. So I've ...
user avatar
  • 111
0 votes
2 answers
660 views

Recover a failing node on Percona 5.5.41-37.0-55

I have a 3 node percona cluster of where 1 of the nodes has, because of a server crash, InnoDB log sequence problems. 180425 17:32:53 InnoDB: Error: page 151555 log sequence number 12431827692 ...
user avatar
0 votes
0 answers
67 views

Recovering software raid after a hardware raid array was created on the disks

I had a server with 4 x 500GB SSDs in a software RAID10 array. I moved these drives to a new machine that had an HP P410i Hardware RAID controller. I didn't want to use the hardware raid controller, ...
user avatar
0 votes
1 answer
70 views

Azure Site recovery

Planning for ASR and migrating all our servers to Azure in distant future from VMware. Issue is that we do not have any VPN capable device in the network and have read that it is necessary. Can ...
user avatar
  • 1
1 vote
2 answers
468 views

What would cause mysqlcheck to incorrectly report a table as undamaged?

We are administering a MySQL server for one of our customers that has >100 databases with about 50 tables each, many of them InnoDB tables. The server crashed and I'm trying to find the culprit. When ...
user avatar
0 votes
0 answers
3k views

Synology: How to restore data from an accidentally deleted volumeX (BTRFS)?

The Synology in question has a RAID6 configured via mdadm which features a BTRFS filesystem mounted to /volume1. This path was mounted into a Debian chroot. The unmount was configured in the chroot ...
user avatar
1 vote
1 answer
607 views

Simulate power loss with force unmount?

I want to test disaster recovery of RDBMs after power loss under high load. My idea is to mount data directory under new mountpoint and then execute umount -f during the load and investigate outcome /...
user avatar
  • 228
4 votes
1 answer
3k views

Cannot login to restored domain controller unless NIC removed

We have four DCs (Windows 2008 R2) running on VMware, backed up with Veeam. Currently doing some DR testing. If I restore a DC (full VM) from backup (obviously isolated from our production ...
user avatar
  • 1,863

1
2 3 4 5
8