Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Im currently building a small CDN topology system, and im looking for the best way to sync the files within certain folder in all the servers.

I need to be able to upload a file to a certain SINGLE FTP, and to get it to duplicate to all the other servers.

The size of the folder that needs to be synced is around 50GB, and i need it to be synced in Up-To 5 minutes difference.

Basically, there are 2 options:
1) have some kind of folder syncing between all servers, and sync that single folder.
2) have some kind of one to many FTP upload mechanism

The O/S on all servers is windows 2008 R2.

Which is the better method? Any other ideas?

share|improve this question

1 Answer 1

Folder syncing is basic. On Windows Server, the best suggestion is to use DFS-Replication. It watches the USN journal (file system change journal) for changes and replicates them.

Here is a simple overview (I am currently analyzing the underlying I/O involved):

Read access against a location on a Windows system

  1. Readfile() [win32 api]
  2. NTReadFile() [kernel32.dll]
  3. Ntdll.dll [pass to kernel mode drivers]
  4. ntoskrnl.exe used to call subsystem
  5. driver.sys is used to call drivers as:
  6. ntfs.sys (USN journal affected) <----------
  7. ftdisk.sys / dmio.sys
  8. disk.sys
  9. RAID controller
  10. hard disks

What happens when you append data to a file on a DFS-R member server ("primary"):

  1. on primary, append data to file
  2. on target, inspection of existing Staging file
  3. on primary, checksum generated and copied to Staging
  4. on target, inspection of existing file
  5. on target, inspection of existing Staging file
  6. on target, generates file in Install from checksum
  7. on target, moves file into target folder
  8. on target, checksum generated and copied to Staging

It is very efficient and very effective for replicating between members.

share|improve this answer
    
Sounds good. Is it possible to use over the internet? with completly different servers? not same domain? Does it have a primary server and the rest copy from it? Do you have any guide for this? –  DeaglinG Jul 21 '11 at 23:11
    
What do you mean over the internet? Publicly? I suppose so. The two machines would have to be trust each other in some way (like domain trusts, forest etc). You can configure one way or two way replication. Search google for guides. You might be better off with Windows ports of rsync or Unison. –  mbrownnyc Jul 22 '11 at 20:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.