I'm wondering if there is an efficient way to backup files from an (S)FTP Server (Exavault), to Google Cloud Storage (GCS). We are using Exavault a managed FTP service and want to regularly back up files to GCS, then purge files older than 30 days from Exavault.
Current Implementation (Bloody Slow):
- Mount FTP to filesystem on Google Compute Instance using
gsutil rsyncto sync files from mounted storage to GCS Bucket
- Delete files based on mtime +30
This method is running really slow and I don't think it will be a reasonable solution at this point.
Are there any solutions that can do something like this, with ~ 500GB of data, more efficiently?