Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have to handle many (1'000 - 30'000) big files ranging from 200MB up to 2GB. The demand for these files is variable (0 - 300 downloads / file). This is why a single file must saved on 2 or more servers.

My servers are placed in different datacenters (France), with different size HDDs (750GB to 4TB).

Currently I share the files using PHP and ncftpget / ncftpput, but it's very slow.

I need a solution to handle balancing these files across 7+ servers.

Example: File A: 300 downloads/day File B: 100 downloads/day File C: 1 download/day

Every file is saved on a different server. File A has a "high demand" and must copied to another Server.

"high demand" = Many users want this file...

share|improve this question
What OS are they hosted on? – MDMarra Jun 2 '12 at 11:40
Thats all Debian Server. – Imises Jun 2 '12 at 11:41
300 downloads a day is not high demand. – womble Jun 2 '12 at 17:54
it´s a example for one file ;) We host 2.000 or more files – Imises Jun 3 '12 at 8:44

I'd use something like DRBD if you need real-time replication, or rsync in a cron from a master server if you don't need real-time mirroring.

share|improve this answer
i dont want to sync alls my servers. I want to save different files on the servers, but files with a high demand should be on multiple server. – Imises Jun 2 '12 at 14:14
That's really unclear. You should edit your question and define things like "high demand". – MDMarra Jun 2 '12 at 16:14
Sorry :) "high demand" = many users want do download a single file. Example: File A: 300 downloads per day // File B: 100 downloads per day // File C: 1 download per Day. File A need to share on 2 or more server – Imises Jun 2 '12 at 16:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.