5

I need to backup/sync few hundred GB of files in S3 buckets to openstack swift containers.

I have thousands of files in several S3 buckets I want to copy/sync over openstack swift containers but I don't want to copy them over on the disk server then copy them to the swift container ( I can't store them). I also would like to avoid using the server bandwidth by mounting the s3 buckets and the openstack swift container using fuse.

The ideal setup would be using the api from one end to the other so when one file is copied over I could get the url of the file from openstack swift (example from aws to ovh cloud storage)

source : https://s3.amazonaws.com/bucket/file.txt
destination : https://storage.bhs1.cloud.ovh.net/v1/AUTH_398y32kue/container/file.txt
ict4ngo
  • 51
  • 2
  • Welcome to Server Fault! Your question is incomplete and, as written, suggests your current knowledge level on the tool may not meet the site's standard for asking questions. Please review the [Welcome to Server Fault tour page](http://serverfault.com/tour) and read [How do I ask a good question?](http://serverfault.com/help/how-to-ask) and consider deleting your current question, reviewing the tool's documentation, then coming back to ask more detailed and specific questions. – Paul Nov 24 '17 at 15:33
  • Hey man! I believe it can be done with assistance of S3 CLI: https://stackoverflow.com/questions/32427274/sync-command-for-openstack-object-storage-like-s3-sync But it would be really helpful if you could provide some details on your infrastructure. – batistuta09 Nov 28 '17 at 19:34
  • thanks @batistuta09 I just edited the question, I'm aware of s3cmd, s3cli, duplicity but all of them are locally syncing/copying the file. I cannot send a file from one cloud to the other through those tool – ict4ngo Nov 30 '17 at 19:12

1 Answers1

0

I think https://rclone.org would provide what you need.

John Dickinson
  • 1
  • 1
    This is a very low quality answer ... imagine some day the link stops working, what value would this answer have then? – Pierre.Vriens Sep 29 '18 at 09:42
  • 1
    Thanks @John-Dickinson I'm using rclone already however It works only if I use a local target or a local source. It means I absolutely need to mount S3 container or mount the swift container. – ict4ngo Oct 03 '18 at 15:47