I've got a site-to-site vpn connection between two data centers (one in San Jose, the other in Toronto).
I need to send a 32GB file from one dc to another - FAST AS POSSIBLE.
I've found a shell script that splices up the 32GB file smaller files and then uses scp to transfer over in parallel fashion.
The question is how do I determine the optimal file size for sending over the various little files across the site-to-site vpn connection (I'd like to try to maximize the bandwidth).
Obviously the more scp process I run on the server, I guess there's more load put on that server.