Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

What tool can I use to synchronize the content of a local directory with that of a Rackspace Cloud Files bucket?

I need a rsync-like application to do a mirror, not a backup application. What I want is the ability to do

rsync --update --delete /local/dir/ cloudfiles://bucket

so that the remote bucket contains an exact mirror of the files that are in /local/dir/.

The files' metadata (owner, permissions) is not important and there are no directories inside the main directory.

share|improve this question

migrated from superuser.com Aug 27 '12 at 14:24

This question came from our site for computer enthusiasts and power users.

5 Answers 5

The easiest way to sync a local directory to Rackspace cloud files is through the console tools provided by the openstack/swift project. On ubuntu, the tools can be installed with apt-get install python-swift

Then, assuming you are in the directory you want to upload, run the following command in the terminal:

$ swift -A https://auth.api.rackspacecloud.com/v1.0 -U <username> -K <api-key> upload <containername> . --changed

This will recursively upload the files from your current directory to the <containername> container, saving time by uploading only changed files. You need to supply the <username> you use to log in to the Cloud Control Panel and the <api-key> available under Account / Account Settings in the same control panel.

Attention: If you use relative or absolute paths, swift will upload them with the pseudo-path provided on the command line into the container. So if instead of syncing . you sync /var/www/test, then files will end up under /var/www/test pseudo-path of the container - most likely, this is not what you want.

share|improve this answer

For Linux I found this old project: http://code.google.com/p/cloudfiles-sync/wiki/Instructions

For Windows there is this GUI tool: http://www.cloudberrylab.com/free-openstack-storage-explorer.aspx

I also just found this tool that might let you mount cloud files storage: http://smestorage.com/?p=static&page=LinuxDrive

share|improve this answer

You can use the Rackspace Cloud Files FUSE module (http://www.rackspace.com/knowledge_center/article/mounting-rackspace-cloud-files-to-linux-using-cloudfuse) to create a mountable file system, but beware of the following caveats:

  • use something like --size-only to determine if the file was fully written, not -a or anything like that, since setting permissions and times is not supported
  • using --bwlimit is not going to work, because the module caches writes in a temp file in memory, then eats up all the bandwidth when uploading; i'm conducting an experiment using the trickle utility to see if that helps
share|improve this answer

Another potential option, as an alternative to CloudFuse for mounting Rackspace Cloud Files as a volume against which you run rsync, you could run Caimito as a Cloud-Files-to_WebDAV bridge.

http://caimito.ngasi.com/

Then, while you could conceivably try to employ the Fuse DAV2 filesystem plugin with rsync and a bunch of special command-line options to get files up to Cloud Files via the bridge, I instead recommend "sitecopy", which at its core isn't terribly dissimilar to Unison.

https://www.howtoforge.com/maintaining-remote-web-sites-with-sitecopy-debian-squeeze-ubuntu-11.10

Sitecopy does a GREAT job of pushing files via WebDAV to its target (even if our target is a fronted emulation layer to Cloud Files). This is because "sitecopy" maintains a local database of remote-end file metadata that makes for speedy batch comparisons vs rsync.

Caimito was surprisingly stable and easy to install and configure despite its Java roots.

You might conclude that using "swift" (mentioned above) might be a more direct vector to the solution, but this solution gives you a few more places to probe, dissect, debug and control the data flow.

share|improve this answer

I think you can use rsync for that but it that doesn't work for you for whatever reasson also try: Unison File Synchronizer, it really works great!

share|improve this answer
    
Neither rsync nor unison support Rackspace Cloud files (nor any other non-ssh based storage). –  gioele Aug 27 '12 at 17:03
    
if that's the case then I don't think you can use any standard tools for non-standard webapp, the best thing to do is to reach out to rackspace and ask them what can be used for that, you may need to use their API in order to access it.. –  alexus Aug 27 '12 at 17:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.