HA for Resque & Redis

Question

Trying to avoid SPOFs for Resque and Redis. Ultimately the client is going to be PHP via (https://github.com/chrisboulton/php-resque). After going through and finding some workable HA for nginx+php-fpm and MySQL (mysql master-master setup as a way to simply master-slave promotion), next up is Resque+Redis.

Standard install of Resque uses localhost Redis (at DigitalOcean). I am heavily depending on Amazon Route 53 DNS failover to try to solve this.

resque1.domain.com points to localhost redis (redis1.domain.com) => same server resque2.domain.com points to localhost redis (redis2.domain.com) => same server

Do resque.domain.com with FAILOVER resque1 as primary and resque2 as secondary. What this means is that most of the time (99%), resque1 should be getting hit with resque2 as just a hot backup.

This lets me just have to get 2 servers and makes sure that any hits to resque.domain.com goes somewhere

---

The other way to do this is to break out resque and redis into 4 servers and do it as follows

resque1.domain.com -> redis.domain.com resque2.domain.com -> redis.domain.com redis1.domain.com redis2.domain.com

Then setup DNS Failover

resque.domain.com -> primary: resque1 and secondary: resque2 redis.domain.com -> primary: redis1 and secondary: redis2

I'd like to get away for 2 servers if I can but is this 2nd setup much better or negligible?

Thanks, Chris

Answer 1

If you have one Redis server, that’s still a SPOF. You might mitigate the risks of that SPOF by automating the provisioning and deployment in addition to performing regular Redis backups and testing of restore procedures. That route would require you to either watch round-the-clock for failures or automatically dispatch the provisioning.

Two servers provides you with a known-working, hot failover. That’s a much better disaster planning scenario.

I’m not familiar with DNS failover, but it appears functionally similar to load balancer VIPs and virtual interfaces. That model of failover is easy to configure, simple to test, and very reliable.