11

I started a snapshot of 1TB volume that carries 750GB of data in AWS EC2 without shutting the instance down. It gave completed status, progress 100% when I noted after 10hrs. I can see the started time. But how to know the exact completion time of the snapshot?

3 Answers 3

17

The completion time is generally not important, as the snapshot is of the volume as at the time you request it, even if you change it while the snapshot is being made. It can take several hours, but it generally doesn't matter how long it takes. My experience is snapshots can take from a few minutes to a few hours, depending on how much data is changed, if there's an existing snapshot, and probably due to internal AWS factors as well.

As pointed out by Joseph below in the comments, the completion time matters if you're doing a backup before you make a change and need to roll that change back.

If you need a completely consistent snapshot you shut the server down, trigger the snapshot, then start the server immediately. This primarily ensures all data is flushed to disk and in a consistent state, with nothing in RAM. It's similar to Windows shadow copy.

In most cases a snapshot of a running system is fine, but sometimes it will be inconsistent. If an application is writing to the disk when the snapshot is requested that data may be corrupt.

EC2 Snapshot documentation is fairly good, like most AWS documentation.

Tip: the first snapshot of any volume is the slowest as it needs to back up every block. Also, the longer the interval between snapshots the slow they tend to be as there's more data to snapshot. If you need a really fast snapshot at any particular time, take a snapshot a couple of hour earlier as that will make the next snapshot faster.

10
  • 1
    "If an application is writing to the disk when the snapshot is requested that data may be corrupt" is not, in the strictest sense, an accurate description, since it implies that the problem lies in the snapshot. The data in the snapshot won't be inconsistent with the data that was on the volume, but the data on the volume may have been in an unusable state, depending on how well the filesystem/application can recover from any partial and unflushed writes. A snapshot is the same thing you'd have if you pulled the power cord on a physical server, then made an image of the disk. (also, +1) Jun 23, 2018 at 14:24
  • 1
    Making a "throwaway" snapshot, first, is a useful strategy. Start the snapshot, then wait. Come back later. Once that is done, do whatever is necessary to make the "real" snapshot (e.g. fsfreeze until the snapshot begins) and it will typically complete much more quickly, because it can integrate the unchanged blocks captured by the throwaway. Then you can safely delete the throwaway, because EBS snapshots can share identical data blocks yet no snapshot depends on the continuing existence of the others. Jun 23, 2018 at 14:32
  • This doesn't answer the question. Jan 16, 2022 at 2:55
  • 1
    @CrescentFresh you're right, but I'm not sure there's any way to find out the completion time, and it's usually not particularly important. The start time is the important part as that's the time the snapshot is taken.
    – Tim
    Jan 16, 2022 at 8:01
  • 1
    @Tim: yes I get it, it's impossible to know currently. It would be useful in the case of trying to gauge how long a particular system might be impacted during the snapshot, in order to provide as much information to people. We ended up timing it on a staging environment (start snapshot, polling until done, measure how long it took). Feb 1, 2022 at 22:56
3

There is no pre-defined completion time. The bigger the data is, the more time it takes to backup. It is recommended to turn off the instance for data consistency or may be not do any read/writes while the backup is in progress.

2
  • Does snap complete faster if I do it after shutting instance down? If so, atleast can I see the 100% progress manually?
    – krkart
    Jun 23, 2018 at 8:16
  • 1
    No, it doesn't snapshot faster if the volume is shut down. See my answer for more details.
    – Tim
    Jun 23, 2018 at 9:15
0

You can use AWS Backup to take the snapshot (On-Demand backup OR using Backup Plans), then either run an On-Demand report which will have the "Completion Date" column OR you can go to AWS Backup console -> My Account -> Jobs, filter by volume ID, click on the JOB ID and check the same field, next you need to go to the EC2 console -> Elastic Block Store -> Snapshots and get the "Started" value. Having these 2 values you can calculate the actual run time.

Do not use "Job Run Time" from the AWS Backup report since it is calculated based on the "Creation Date" of the job and not when the job actually started.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .