0

I've been using AWS S3, and I'm not happy that I can't do things like an rsync from the Linux server to do a backup on S3. It's because S3 stores objects I'm told and not files. Is this something all cloud storage services do including those like Google and Microsoft?

I'm not seeing the benefit of object storage (AWS S3), since it doesn't emulate a Linux file system. Because if I go to restore from an S3 backup to the Linux file server, it isn't going to look like it did originally because it ignores information like Linux permissions, for example.

Edward_178118
  • 955
  • 4
  • 15
  • 33
  • 1
    The benefit is it's cheap (relatively), reliable, and has unlimited space. Either they didn't think to add rsync compatibility, or they tried and it was too hard. – user253751 Jul 03 '20 at 17:18

1 Answers1

1

Direct Question

To answer your direct question: No. There are many cloud storage services, they are all different.

If you need rsync then suggest you rent a server to rsync your files to. If you want them backed up to the cloud you can use the AWS CLI and "aws s3 sync" on a cron job.

Implied Question

The wider question is difficult to answer because this is more a criticism than a actual question. If the question was "how do I back up my on-premise Linux server to AWS" I'd say

  • "Use the aws s3 sync command along with an s3 bucket with version control and encryption enabled", or

  • "use software such as restic or MSP360 to backup to AWS S3, B2, Google Cloud, etc". But that's not what you asked.

S3 Background

S3 is generic storage with its own API, and many other features such as versioning, encryption, etc. I imagine it was designed generic to make it easily usable by many operating system, devices, or anything else that needs it.

S3 has both an API and a command line tool, making it fairly easy to use.

Whether you call something an object or a file is largely irrelevant. S3 can be viewed as a file system by any OS, and you can even use tools like s3fs to make S3 look like a file system. That's not really what it's designed for, but it can do it. If S3 doesn't suit your use case, don't use it.

S3 General

AWS has multiple storage solutions. EC2 with EBS or EFS volumes is more likely suitable for your needs.

Tim
  • 31,888
  • 7
  • 52
  • 78
  • There is a reasonable expectation that AWS S3 should support Linux since it is very popular and they support it by having their own version of Linux. Using AWS S3 CLI is not going to preserve and restore the Linux file permissions. Renting a Linux server isn't going to provide the redundancy that cloud computing is famous for. You said S3 can emulate a Linux file system, how can it do this? – Edward_178118 Jun 20 '20 at 11:24
  • S3 is not interested in emulating a block device or file system. Use one of the many backup systems that can store their archives in S3. Or Amazon's other storage offerings like EFS, a file share offering NFS with most of what you expect from POSIX file operations. – John Mahowald Jun 20 '20 at 12:07
  • Have a read of my updated answer. – Tim Jun 20 '20 at 19:24