3

I'm currently using Amazon S3 to host all static content on my site. The site has a lot of static files, so I need an automated way of syncing the files on my localhost with the remote files. I currently do this with s3cmd's sync feature, which works wonderfully. Whenever I run my deploy script, only the files that have changed are uploaded, and any files that have been deleted are also deleted in S3.

I'd like to try Rackspace CloudFiles; however, I can't seem to find anything that offers the same functionality. Is there anyway to accomplish this on Rackspace Cloud Files short of writing my own syncing utility? It needs to have a command-line interface and work on OS X.

James Simpson
  • 13,488
  • 26
  • 83
  • 108

2 Answers2

2

The pyrax SDK for the Rackspace Cloud has the sync_folder_to_container() method for cloudfiles that sounds like what you're looking for. It will only upload new/changed files, and will optionally remove files from the cloud that are deleted locally.

As far as the initial upload is concerned, I usually use eventlet to upload files in as asynchronous a manner as possible. The total time will still be limited by your upload speeds (I don't know of any SDK that can get around that), but the non-blocking code will certainly help overall performance.

If you have any other questions, feel free to ask here on on the GitHub page.

-- Ed Leafe

Ed Leafe
  • 130
  • 4
0

The Rackspace Python SDK can do that for you. There's a script called cf_pyrax.py that does, more or less, what I think you're trying to do. There's a write up on it in this blog post.

Everett Toews
  • 10,337
  • 10
  • 44
  • 45
  • Hmm, I'm really hoping this isn't the best option. It is going to take well over 24 hours to upload 100,000 files, while the same operation takes less than 30 minutes with s3cmd. Not yet sure if the difference is Rackspace, the SDK, or a both. – James Simpson Feb 19 '13 at 05:19
  • Hmmmm...that doesn't sound right. Let's try get to the bottom of this. Open an issue at https://github.com/rackspace/pyrax/issues with as much detail as you can (bandwidth, file copy times, type of files, etc.). Include the code if you're able to. – Everett Toews Feb 20 '13 at 01:45