4

We have 85TB of data in AWS S3 bucket. What is the best option to move it to Azure?

I see two options right now:

  1. To buy AWS Snowball Edge. Will I be able to import data to Azure from it?
  2. To buy Azure Data Box. Will I be able to export data from AWS to it?

One more difficulty is that the data is a gazillion of small files (images), so will AWS charge for GET requests and Azure for PUT requests if we choose their respective devices?

  • 2
    I'd think you'd need one of each, and do the transfer locally between the two devices (potentially with a server in between). – ceejayoz Nov 11 '17 at 22:12
3

I would call both Amazon and Azure to discuss the latest services for moving this data.

From my experience you have two choices:

  1. Over the wire data transfer using Azure. Azure offers a service called Azure Data Factory which supports moving data from a number of vendors and services into Azure. You will have to pay for gets, puts, data transfer, etc.
  2. Data Transfer Devices. Both Amazon and Azure support using physical devices to export / import data into their cloud. You would order an export service from Amazon which will be delivered on Snowball. You would then copy the data from Snowball to physical disk drives that you send to Azure for import.

85 TB is not a huge amount of data today (85 PB would be), so the data transfer times will probably be less for over the wire versus export from Amazon, copy data, then import into Azure. Given the real world this will probably take a week for over the wire and a month for export / import. This does not consider the time to actually verify the data was transferred without error. This phase would probably take another week or two.

2

For a strictly over the wire transfer you could you a product like Cloud Berry Labs’ Drive which will let you map S3 and Azure storage as a lettered drives. Then you can simply copy between them. I believe the client version would do all of what you need since you don’t need to share the drive on the network.

Cloud Berry Labs

One note on this software is that it buffers the files as it transfers so windows may show its done transferring however in the queue it may still be transferring, especially with large files that it has to do a multi part upload.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.