We have a very large Windows file store (several terabytes and tens of millions of files) that I want to keep in continuous replication over to another server, in as near to real-time as possible. I'm looking for options on tools that will make this happen.
So far I've come up with:
- Move it to a NAS or SAN and share the files. Won't work, not an option for us for at least six months.
- Use robocopy with /MON. I'm worried about the drain on replication source resources as it rescans the entire tree every cycle. (While /MON watches for file changes before triggering a new cycle, each run performs a full scan, and does not actually use the file change data.)
- Rsync. No better than robocopy.
- DFS. I've heard very bad things from people at MS about DFS for stores with millions of small files. Probably half our files are very small.
- Hybrid. Write my own tool or find one online that uses file watchers to target what files need to be copied. Though the buffer can overflow, so still run a nightly robocopy to pick up anything that was missed.
- Backup-based. Some kind of crazy scripted backup/restore thing where the backup software can do very fast incremental snapshots.
Any ideas would be greatly appreciated, thanks!