0

I have large amounts of data (500+ GB) stored in azure blob storage that I need to transfer to rackspace cloud files. I know it is possible to perform such a migration using the SDKs from both services, but is there a free, standard, 1-step process for doing this? I've built a POC utility but would like to avoid having to optimize it to perform the transfer within a reasonable amount of time.

Thanks.

Improve this question
2
  • If you do decide to do it via API, a couple of pieces of advice on the Rackspace side: multithread it and remember the 100 uploads/sec/container limit.
    – phoebus
    Commented Oct 30, 2013 at 21:51
  • Also, if you decide to do it via the SDKs I would recommend running it from a server on Azure or Rackspace's side (don't make a clientside tool be an intermediary).
    – Kyle Kelley
    Commented Oct 31, 2013 at 2:36

2 Answers 2

Reset to default
1

This is not an official endorsement, but CloudBerry Cloud Migrator does support Azure and Cloud Files. It's in beta today, which also means it is free.

http://www.cloudberrylab.com/cloud-migrator.aspx

I don't know of any other utilities that can perform this for you that are out right now. I'm checking with our migration team to find out if there are any utilities they can recommend.

Improve this answer
4
  • Thanks for this suggestion and for looking around. Due to licensing concerns with my client, I cannot allow their data to be copied via this service. I've already put significant effort in a home-grown solution since I haven't been able to find anything viable, so I might just end up using that, unfortunately.
    – S.C.
    Commented Oct 31, 2013 at 20:15
  • S.C. - I asked around internally and it doesn't seem like we have a tool for this right now. Sorry! I'm going to add this to our list so we can begin developing a solution for this.
    – Chris Rasco
    Commented Nov 1, 2013 at 3:18
  • Over the last few days I've developed a robust .net based solution that averages a Gig or so transferred every nine minutes depending on the overhead involved. It's still a work in progress, but I'll talk with my superiors about open-sourcing it and posting it here. Thanks again.
    – S.C.
    Commented Nov 5, 2013 at 0:05
  • Great! Feel free to email me too. Email is in my profile
    – Chris Rasco
    Commented Nov 5, 2013 at 1:53
1

Posthaste is a Gevent-based uploader for Cloud Files that you may be able to leverage to speed up writes to Cloud Files if you do decide to script it. Cyberduck will do GUI uploads to Cloud Files, although I can't speak for its speed.

Improve this answer
1
  • Posthaste looks interesting. I have used cyberduck in the past, but since it requires the files to be on disk before uploading them to the cloud service, it is of limited use in this scenario. I've also tried the cloudberry alternative, but it has the same issue. Apart from disk space, disk IO would be a real bottleneck with a two-step approach.
    – S.C.
    Commented Oct 31, 2013 at 20:17

Not the answer you're looking for? Browse other questions tagged .