I have to handle many (1'000 - 30'000) big files ranging from 200MB up to 2GB. The demand for these files is variable (0 - 300 downloads / file). This is why a single file must saved on 2 or more servers.
My servers are placed in different datacenters (France), with different size HDDs (750GB to 4TB).
Currently I share the files using PHP and ncftpget / ncftpput, but it's very slow.
I need a solution to handle balancing these files across 7+ servers.
Example: File A: 300 downloads/day File B: 100 downloads/day File C: 1 download/day
Every file is saved on a different server. File A has a "high demand" and must copied to another Server.
"high demand" = Many users want this file...