0

I have large amounts of data (500+ GB) stored in azure blob storage that I need to transfer to rackspace cloud files. I know it is possible to perform such a migration using the SDKs from both services, but is there a free, standard, 1-step process for doing this? I've built a POC utility but would like to avoid having to optimize it to perform the transfer within a reasonable amount of time.

Thanks.

closed as off-topic by EEAA, user62491, Ward, squillman, colealtdelete Oct 31 '13 at 16:13

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions seeking product, service, or learning material recommendations are off-topic because they tend to become obsolete quickly. Instead, describe your situation and the specific problem you're trying to solve." – EEAA, Community, Ward, squillman, colealtdelete
If this question can be reworded to fit the rules in the help center, please edit the question.

  • If you do decide to do it via API, a couple of pieces of advice on the Rackspace side: multithread it and remember the 100 uploads/sec/container limit. – phoebus Oct 30 '13 at 21:51
  • Also, if you decide to do it via the SDKs I would recommend running it from a server on Azure or Rackspace's side (don't make a clientside tool be an intermediary). – Kyle Kelley Oct 31 '13 at 2:36
1

This is not an official endorsement, but CloudBerry Cloud Migrator does support Azure and Cloud Files. It's in beta today, which also means it is free.

http://www.cloudberrylab.com/cloud-migrator.aspx

I don't know of any other utilities that can perform this for you that are out right now. I'm checking with our migration team to find out if there are any utilities they can recommend.

  • Thanks for this suggestion and for looking around. Due to licensing concerns with my client, I cannot allow their data to be copied via this service. I've already put significant effort in a home-grown solution since I haven't been able to find anything viable, so I might just end up using that, unfortunately. – S.C. Oct 31 '13 at 20:15
  • S.C. - I asked around internally and it doesn't seem like we have a tool for this right now. Sorry! I'm going to add this to our list so we can begin developing a solution for this. – Chris Rasco Nov 1 '13 at 3:18
  • Over the last few days I've developed a robust .net based solution that averages a Gig or so transferred every nine minutes depending on the overhead involved. It's still a work in progress, but I'll talk with my superiors about open-sourcing it and posting it here. Thanks again. – S.C. Nov 5 '13 at 0:05
  • Great! Feel free to email me too. Email is in my profile – Chris Rasco Nov 5 '13 at 1:53
1

Posthaste is a Gevent-based uploader for Cloud Files that you may be able to leverage to speed up writes to Cloud Files if you do decide to script it. Cyberduck will do GUI uploads to Cloud Files, although I can't speak for its speed.

  • Posthaste looks interesting. I have used cyberduck in the past, but since it requires the files to be on disk before uploading them to the cloud service, it is of limited use in this scenario. I've also tried the cloudberry alternative, but it has the same issue. Apart from disk space, disk IO would be a real bottleneck with a two-step approach. – S.C. Oct 31 '13 at 20:17

Not the answer you're looking for? Browse other questions tagged or ask your own question.