Currently I create an NFS and mount it on a special server that does the actual backups and uploads it to s3. My main concern is I feel like overcomplicated this simple task. I schedule cron job to run the backup off hours but still would like to optimize and improve my strategy.
Currently the script on backup server will just find files on the NFS share within specific dates, tar them all up and use aws sync command to upload to s3 glacier. On the next run, the script gets a list of aws s3 objects and skips finding files for those dates (the date is part of archive file name, so its a very fast custom sync mechanism)
Checking network usage during the tar command over NFS is never more than 200Mbps and that is over lan through a 1Gbit switch. Upload to s3 is capped in aws cli configuration. So the script is very low key which is what I want. Yet something tells me maybe I should have just simply used aws sync to backup the data everyday and not try to reinvent the wheel. The extra complexity in find and tar and extra sync logic might lead to some unforseen bugs simply due to overcomplexity.
My question is what are the drawbacks of doing it my way (my main concern is is using NFS for heavy read backup operations bad practice?) And adding all that extra logic.
But if I just brute force it for simplicity and use aws sync for all files will take tremendously long for terabytes of data and also spike the CPU on the backup server and due to huge amount of read, spike CPU on production with the NFS share. I suppose I could try sync for specific dates and store a local file with what dates I already backed up.
Would appreciate some wisdom here :)