42

No, it's not. What happens when your filesystem or RAID volume gets corrupted? Or your server gets set on fire? Or someone accidentally formats the wrong array? You lose all your data and the not-real-backups you thought you had. That's why real backups are on a completely different system than the data you're backing up - because backups protect ...


26

This totally doesn't answer the question itself, but don't forget you can delete ranges of snapshots. zfs destroy zpool1/dataset@20160918%20161107 Would destroy all snapshots from "20160918" to "20161107" inclusive. Either end may be left blank, to mean "oldest" or "newest". So you could cook something up that figures out the "n" then destroy "...%n".. ...


23

More general case of getting most recent snapshot based on creation date, not by name. zfs list -H -t snapshot -o name -S creation | head -1 Scoped to a specific filesystem name TestOne zfs list -H -t snapshot -o name -S creation -d1 TestOne | head -1 -H:No header so that first line is a snapshot name -t snapshot: List snapshots (list can list other ...


20

The proper method was virsh snapshot-delete prod --metadata snap (This command can be found on the wiki. I tried it before asking here but it failed due to a typo that has been corrected since.) I don't know what it does that is not covered by removing the .xml file while libvirtd is down. Maybe the only difference is that stopping libvirtd is not needed. ...


18

The ResearvationId has nothing to do with Reserved Instances. It is a unique value indicating a request to launch EC2 instances. A launch request may come from the cli, AutoScaling, or the AWS Management Console. Each request to aws ec2 run-instances will return zero or more reservations of instances. Usually, it'll be one if it succeeds, but the docs say "...


17

Yes, there are performance implications for long-running snapshots. There are even greater implications for consolidating delta VMDKs back to the original disk file. This can cause unresponsiveness in your VM's operating system or other undesirable behavior. VMware has templating and cloning functionality built into vCenter. You need a $600 vSphere ...


15

Backups serve two functions. First and foremost, they're there to allow you to recover your data if it becomes unavailable. In this sense, snapshots are not backups. If you lose data on the filer (volume deletion, storage corruption, firmware error, etc.), all snapshots for that data are gone as well. Secondly, and far more commonly, backups are used to ...


15

The amount of time it takes EC2 to build a snapshot is directly proportional to the number of modified blocks that volume has seen since the last snapshot. In the case of your journal, that sees a lot of write/delete operations, so it's not surprising that it would take a long time to snap. Keep in mind that these are truly block devices. They know nothing ...


15

I can literally delete all but the most recent with impunity Assuming you don't need any data that was already deleted or overwritten on the volume when you took the most recent snapshot, that's true. EBS snapshots are logically incremental -- not physically incremental. Here's the cleverness that explains the difference: Snapshots of EBS volumes don't ...


12

The -F switch comes in handy if you have messed with the destination dataset after it has been received. Once you do any changes to it (including doing something as innocent as a directory listing as this would change atimes), it is no longer in the state it was in after the initial transfer. Trying to run a plain zfs receive from an incremental data stream ...


12

The only thing you are missing is to quiesce the guest filesystem before taking the snapshot, to ensure that it is consistent. This can be done with virsh domfsfreeze if you are using libvirtd. For example, the order of operations is: # Freeze guest filesystems virsh domfsfreeze $VM_NAME # Create snapshot qemu-img create -f qcow2 -b $VM_NAME.qcow2 ...


11

There's no meaningful difference between the two, and it's only the OpenStack dashboard that makes a distinction between between images and snaphots. I submitted this issue as a UI bug a while back, and it has been fixed in the upcoming (havana) release of OpenStack.


11

For on-site backup, snapshot might be good enough, provided that you regularly 'export' your snapshot somewhere else, where it exists as passive data. And, regularly test if your 'shipped snapshot' can be restored. This is how I implemented a quick backup of some of my servers: store the data on ZFS, take a ZFS snapshot, send the delta to another server, ...


11

How can I figure out how long it will take to create the new EBS volume? Create one. And then, try using it. Continue using it over a period of hours and days, and note what you observe. The first answer to your question is that it actually only takes a few seconds. The problem with that answer is that it doesn't tell the whole story: New volumes ...


9

Things have changed in the years since this question was posted: 1: ZFS now supports compressed replication, just add the -c flag to the zfs send command, and blocks what were compressed on disk will remain compressed as they pass through the pipe to the other end. There may still be more compression to be gained, because the default compression in ZFS is ...


9

From the output, I am guessing that the subvolume of which you have snapshots is actually an ecryptfs private directory. If that is correct, can you try unmounting the private directory and then try to delete the snapshots? The error message states that the device is in use. Assuming you are not using that directory as a current working directory, and no ...


8

Is this accurate, and is it fundamentally the same for LVM and KVM snapshots? Yes and yes.


8

What HopelessN00b said. No. Proper backups are on a separate device than the device being backed up. What happens when you lose two or more drives? What happens when your server room burns down? What happens when someone accidentally destroys your array? (Anecdote alert: I once heard of someone who had PXE set to auto-install the latest Fedora. His ...


8

They are a backup, yes. I've personally used them in place of daily incrementals before, but we still did weekly fulls to tape. They protect quite well from any non-netapp (systems accessing volumes) user or admin errors or problems. They do not protect from catastrophic hardware failures of the netapp itself. My understanding is that SnapMirror does copy ...


8

Most NAS snapshots implement copy-on-write, so the snapshot itself initially takes up no space (or next-to-none, there is some overhead). But any block that is changed whilst the file it's part of is snapshotted must be copied. If an entire file is deleted whilst it's still in a snapshot, all those blocks must still be kept. So in your case above, as long ...


8

How do other organizations deal with this? I do more or less what you're doing. Keeping snapshots around for a few days for a specific reason isn't bad. You just need to make sure to not forget about them and let them "rot" for any longer than is needed. I have my monitoring system set to alert me if snapshots stay around for longer than three days. Is ...


7

The Snapshot is part of a chain of images and requires the availability of all snapshots. You can boot off the snapshot, but you must have all the previous images intact as well Having a snapshot chain does degrade performance. Highly loaded server VMs should not be running off of snapshots at all To manage snapshots, you simply try to keep ...


7

No you can't! The process of deleting the snapshot actually consists of committing the changes that were made in it. ESXi won't let you change power state during this process (forcing it off by taking drastic measures could corrupt your disks) You can start the process with the VM off but this will only make it quicker if the VM has heavy IO. The downside ...


7

In my opinion, the simplest way will be just to backup VMs regularly, as @EEAA mentioned before. We have quite large amount of VMs running in our environment. So we configured backup Every 1 week. Our setup is StarWind + Veeam. Also, we considered other solutions, e.g. MS DPM, but encountered some issues with it. Veeam just works great.


7

You may have to look in CloudTrail to find out what IAM Role was used to create the snapshots. That should take you in the right direction - whether it's a CloudWatch role, EC2 role, or some external script using IAM User keys. Also have a look at AWS Backup - perhaps that is being used to create the images. In that case you can set the lifecycle policy ...


6

Properly implemented snapshots MUST be supported by your storage as decent backups do use them as a very first stage of creating a backup job. It's however a bad idea to use snapshots for primary backup. Reasons: 1) Snapshots and backend storage CAN fail. So real backups must be using separate spindle set or there's a great chance to lose both primary ...


6

Yes, it is. It is a perfect way to store backups. Nothing else is needed, heck, even doing ingtegrity checks are just wasted time. Just to confirm - before I give more advice... you work for a competitor of mine, right? You really do, sure? No? Oh. Sorry, NUTS. No, not at all. Sorry, dude. Problem is that you are totally open to any error that happens in (...


6

Snapshots aren't backups! You really need to follow 3-2-1 backup rule to ensure your data is safe. To accelerate your tape backup and recovery process I'd suggest to go from Disk-to-Tape to Disk-to-Disk-to-Tape: deploy some VTL offloading "cold" backups to either physical tape or AWS S3/Glacier.


5

I know this post is a little old, but the question and points raised are still very relevant. VMware snapshots are definitely not backups. Worse side effect that I had happen to me and numerous customers, they keep a 6 month old VMware snapshot, the ESXi host experiences an unplanned disruption causing the ESXi to reboot or a reboot is needed to clear the ...


5

The number of files and directories involved in a zfs send/recv stream should have no direct impact on its transfer speed. Indirectly, it might, because it is usually true to say that the 'spread' of the dataset across your disks will be higher with more directories/files, depending on the workload that generated them. This matters, because it's far easier ...


Only top voted, non community-wiki answers of a minimum length are eligible