0

Currently we need to migrate a bucket of about 1 TB but it contains a lot of files split into many layers of subdirectories. If I understand the pricing correctly moving data between buckets in same region should be free according to this:

"Transfers between S3 buckets or from S3 to any service(s) within the same region are free."

From looking at the AWS docs the way the recommend is to use the aws s3 cli to sync the files across.

In this case I would run aws s3 sync s3://oldbucket s3://newbucket. On a ec2 instance running in the same region as the buckets. But wouldn't I still be charged for GET / PUT requests?

The storage cost is not a problem in this case, what I'm just worried about is the huge amount of small files that would inflict a huge cost of requests.

If anyone would have any better insights in this I would be very relieved.

  • 1
    When you have complicated billing questions like this, it might be best to contact AWS support first via email, so you have an audit trail of their response in case things go awry; save yourself the headache! – jski Jul 6 '17 at 12:29
  • Great point, @jski. Being able to go "I did what you told me to, and it should've cost $5 based on what you said" is a really nice thing to be able to do if you get an unexpected $10k AWS bill. – ceejayoz Jul 6 '17 at 12:49
2

I find AWS pricing really complicated. As you say "Transfers between S3 buckets or from S3 to any service(s) within the same region are free." as stated here on the AWS Pricing page: https://aws.amazon.com/s3/pricing/

However on the Simple Monthly calculator on the S3 section it does suggest there is a small cost for copy actions as well as Get and Put http://calculator.s3.amazonaws.com/index.html

Try using the calculator to get an estimated cost for you copy operations.

  • According to the calculator this operation of getting and putting 1 million files should be less than $40. Almost seems too cheap, but 'll do what @jski wrote in the comment above and contact AWS just to be safe. – Carl Jul 6 '17 at 13:09
  • That makes sense, I don't think it is going to cost a fortune but better to be on the safe side. if you are happy with the answers can you mark the answers as accepted. good luck. – Michael Brown Jul 6 '17 at 13:16
0

Leverage an EC2 instance in the same region rather than local machine to avoid egress charges and likely reduce transfer speed

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.