Note: Backing up data stored on Amazon S3 is similar but quite old, and doesn't address common practice.
Our service involves each user upload a couple of files. There are hundreds of thousands of users which will keep increasing.
We use AWS for our infrastructure, and currently all files are uploaded and stored directly in S3. The top-level folder is about 75 GB and growing.
For a long time we've had a script that does a nightly backup by copying each file via the S3 API into another folder. Last few months the storage cost became huge, so we switched to storing just bi-weekly backups.
The reason for keeping backups is that the business depends on the user uploaded data. Think of Dropbox or Flickr or Giphy.
We're not worried about S3 losing the data but rather due to human error, which is more likely no matter how many precautions we take. In such cases, we should be able to restore data to a certain point in time.
However, the S3 backup strategy seems dubious since the backups don't seem to be happening correctly. The size of backups seems to be very low sometimes, implying that S3 copy operations don't behave - even using the S3 web console to copy-paste a folder with a large number of files doesn't work properly and hangs after partially completing, makes us even more suspicious of large copy operations on S3.
We are aware of S3 object versioning but it doesn't seem like the right solution for a backup to mitigate the effects of human error like deleting the bucket.
What backup strategy for files do businesses that depend on user uploaded files take?