0

I have large amounts of data (500+ GB) stored in azure blob storage that I need to transfer to rackspace cloud files. I know it is possible to perform such a migration using the SDKs from both services, but is there a free, standard, 1-step process for doing this? I've built a POC utility but would like to avoid having to optimize it to perform the transfer within a reasonable amount of time.

Thanks.

S.C.
  • 111
  • 3
  • If you do decide to do it via API, a couple of pieces of advice on the Rackspace side: multithread it and remember the 100 uploads/sec/container limit. – phoebus Oct 30 '13 at 21:51
  • Also, if you decide to do it via the SDKs I would recommend running it from a server on Azure or Rackspace's side (don't make a clientside tool be an intermediary). – Kyle Kelley Oct 31 '13 at 02:36

2 Answers2

1

Posthaste is a Gevent-based uploader for Cloud Files that you may be able to leverage to speed up writes to Cloud Files if you do decide to script it. Cyberduck will do GUI uploads to Cloud Files, although I can't speak for its speed.

phoebus
  • 8,380
  • 1
  • 31
  • 30
  • Posthaste looks interesting. I have used cyberduck in the past, but since it requires the files to be on disk before uploading them to the cloud service, it is of limited use in this scenario. I've also tried the cloudberry alternative, but it has the same issue. Apart from disk space, disk IO would be a real bottleneck with a two-step approach. – S.C. Oct 31 '13 at 20:17
1

This is not an official endorsement, but CloudBerry Cloud Migrator does support Azure and Cloud Files. It's in beta today, which also means it is free.

http://www.cloudberrylab.com/cloud-migrator.aspx

I don't know of any other utilities that can perform this for you that are out right now. I'm checking with our migration team to find out if there are any utilities they can recommend.

Chris Rasco
  • 206
  • 1
  • 3
  • 4
  • Thanks for this suggestion and for looking around. Due to licensing concerns with my client, I cannot allow their data to be copied via this service. I've already put significant effort in a home-grown solution since I haven't been able to find anything viable, so I might just end up using that, unfortunately. – S.C. Oct 31 '13 at 20:15
  • S.C. - I asked around internally and it doesn't seem like we have a tool for this right now. Sorry! I'm going to add this to our list so we can begin developing a solution for this. – Chris Rasco Nov 01 '13 at 03:18
  • Over the last few days I've developed a robust .net based solution that averages a Gig or so transferred every nine minutes depending on the overhead involved. It's still a work in progress, but I'll talk with my superiors about open-sourcing it and posting it here. Thanks again. – S.C. Nov 05 '13 at 00:05
  • Great! Feel free to email me too. Email is in my profile – Chris Rasco Nov 05 '13 at 01:53