7

Objects larger than 5GB are supported with multi-part uploading. We have 5gb+ files in one bucket already that I'd like to move to another, under the same AWS account. When I issue the command using s3cmd:

s3cmd mv s3://BUCKET1/OBJECT1 s3://BUCKET2[/OBJECT2]

I comes back with the error:

ERROR: S3 error: 400 (InvalidRequest): The specified copy source is larger than the maximum allowable size for a copy source: 5368709120

If it's at all possible, I presume it will not be able to be moved over without extra bandwidth/cost. Even so, I'm trying to figure out if you can move large files in some sort of multi-part method.

4

Currently, what you're trying to do can't be done in a single operation. Move in S3cmd to the API is essentially a copy and delete all in one and it's a restriction of the copy operation.

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html

You can store individual objects of up to 5 TB in Amazon S3. You create a copy of your object up to 5 GB in size in a single atomic operation using this API. However, for copying an object greater than 5 GB, you must use the multipart upload Upload Part - Copy API

http://docs.aws.amazon.com/AmazonS3/latest/dev/CopyingObjctsMPUapi.html

The examples in this section show you how to copy objects greater than 5 GB using the multipart upload API. You can copy objects less than 5 GB in a single operation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.