For hosting our web sites in a Windows load-balanced environment we've traditionally used DFSR to sync folders between servers with our hosting company.

We're looking to migrate a lot of our sites to Amazon EC2 and I'm playing around with load balanced setups at the moment. That's all going fine, but now I'm onto trying to get folders/drives to sync between web servers.

Ideally I'd like to use DFSR, but it relies on having a domain controller in order to sync folders. After some research into setting up a DC on EC2 it appears that this is quite tricky (and somewhat outside my area of expertise as a developer) but I've seen it said once or twice that it's not necessarily that reliable?

So far alternatives I've looked into:

  • Network share from www1 to www2 - if this is on a single web server then it provides a single point of failure (plus we've had performance issues with this).
  • Network share on a separate "micro" Windows instance - again, provides a single point of failure as if this instance goes down then the whole site
  • Cached network drive - I investigated using a cached network drive to get around the "single point of failure" issue, however, it's tricky to get a persistent networked drive that IIS will be able to see, and even then, I couldn't see a way to cache it (plus I'm not sure how quickly changes would be replicated between machines).
  • Robocopy - from what I've seen this this could be useful if I ran it every minute, but it looks like it won't copy locked files? Also, there would be a lag in sync-times

Sorry if that's not explained very well or I've missed something obvious. Has anyone come up with a good, reliable method of syncing folders between web servers on Amazon EC2 which is as good as DFSR (or close)?

  • 1
    windows 2012 has the ability to support file server scale out, I'm just not sure if amazon support this process and if its economical. – tony roth Aug 12 '13 at 15:01
  • Amazon says that 2012 pricing is the same as any other version of windows, and that you can create a Storage Space on EBS volumes. So, this could easily be the winner. – mfinni Aug 12 '13 at 15:29
  • Interesting - I'll take a look at that today - thanks guys :) – tristankoffee Aug 13 '13 at 8:44
  • How often do your files really change? – Dusan Bajic Aug 13 '13 at 8:49
  • @dusan.bajic, it depends on the client and website, we generally need it for CMS sites where the client will be regularly uploading content and assets. – tristankoffee Aug 13 '13 at 16:38

Use S3 for user generated content (one bucket that all webservers will have access to), and keep application on EC2 webservers?

  • Thanks for the suggestion, but we are moving existing applications onto EC2, some of which would require a fair amount of dev to integrate with S3. Do you know of a reliable way of mapping S3 to a drive on Windows server? I'd be looking for something that works like Dropbox but for S3 (i.e. content is stored locally, but synced between S3 as it changes locally/on S3) – tristankoffee Aug 14 '13 at 8:51
  • You can find some third party tools recommendations on AWS forums but I believe none of them is officially supported by Amazon. I've successfully used s3.exe from s3.codeplex.com to sync windows folder with s3 bucket but not for real-time sync. – Dusan Bajic Aug 14 '13 at 10:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.