5

Is it possible via the AWS CLI or via some third-party tool to easily download all contents of a versioned bucket as it was on a particular datetime?

I.E. I would like to retrieve the version of each object in the bucket which would have been regarded as the 'latest' version on the given date.

6

Looking at the aws cli, I only see the versioning functionality exposed in the s3api section, not in the s3 section.

From what I've understood it's only possible to fetch previous versions of objects based on their version id.

By default, the GET operation returns the current version of an object. To return a different version, use the versionId subresource.

See http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html for examples and additional details

To solve what you're asking for my understanding is that, unless you have been tracking the version ids yourself, you will need to fetch a list of the versions first to determine which ones to get.

You can use the versions subresource to list metadata about all of the versions of objects in a bucket. You can also use request parameters as selection criteria to return metadata about a subset of all the object versions.

See http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETVersion.html for details.

That response would have data like the following for each version that matches your query:

    <Version>
        <Key>my-image.jpg</Key>
        <VersionId>3/L4kqtJl40Nr8X8gdRQBpUMLUo</VersionId>
        <IsLatest>true</IsLatest>
         <LastModified>2009-10-12T17:50:30.000Z</LastModified>
        <ETag>&quot;fba9dede5f27731c9771645a39863328&quot;</ETag>
        <Size>434234</Size>
        <StorageClass>STANDARD</StorageClass>
        <Owner>
            <ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID>
            <DisplayName>mtd@amazon.com</DisplayName>
        </Owner>
    </Version>

In terms of the aws cli this would be aws s3api get-object --version-id ... and aws s3api list-object-versions ... respectively for the API calls mentioned above.

| improve this answer | |
  • Thank you, while not an quick easy solution, this will help me write a script to perform the task I need to complete. Thanks for taking the time to reply. – Nada_Surf Jul 4 '14 at 8:34
  • hard to believe Amazon haven't allowed for this – mcmillab Jun 26 '19 at 0:45
3

Using Duplicity you can do a restore from a point in time by including the parameter --time. You can refer to this link for some examples of use with Amazon S3 service.

duplicity --time 2012-10-01T01:00:00 s3+http://backup_bucket_url /local_restore_path
| improve this answer | |
  • 1
    Thanks for your comment. Though I'd prefer to not have to use Duplicity, given that I will need to deal with the hassle of having a huge set of Duplicity files (my data set is about 900gb). – Nada_Surf Nov 3 '14 at 12:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.