I've developed interest in VPS clusters lately and it seemed like a great idea to try to set up a multi-VPS setup, which I will describe below. It isn't really meant for production use, but more as an experiment to improve relevant skills and knowledge of multi-server systems. However since I lack the knowhow, I need some general information.
Since I deal daily with Node.js applications that use Redis as datastore, it would serve as the basis of the setup. What I had in mind was basically a minimum of 2 VPS setup. Each of the servers would be running the same Node services (let's say 5 different services at once on both of the servers), also each VPS runs one instance of Redis, which is used by the Node services to store data. The aim of this setup is to enable the data mirroring between both servers (if Node service #1 in the first server added something to Redis, the change should also be reflected in the second server). This would also mean that uploaded files etc should be mirrored on both servers (in a sense that changes in both datastores and filesystems must be reflected in the other).
Ideally, this would enable simple load-balancers that share the load between servers and in case of a single server failure, the other servers would keep running and therefore keeping Node services online. The speed at which the changes need to be mirrored is not high and even a minute-long delay wouldn't really matter. However if a user is actively updating data, he must be presented with the data he just changed (in the sense that he must somehow be forced to communicating with the server he updated the data in, since the changes might not have been mirrored on the other servers yet).
What are the reasonable ways to achieve load balancing? I've heard about using some DNS magic, but don't really understand it. Simply updating DNS records would be too slow since they are cached in multiple places. Also I read about using one "main proxy" server that would handle balancing between other servers. This seems a bit risky because if the main server failed, everything would be offline.
How to mirror parts of filesystem on different VPS's so that uploaded images etc would be present on both servers. Are there any wide-spread software options, or would a simple script that detects uploads and then replicates those files on different servers work just fine?
- Does Redis even support the kind of mirroring I described? I only found information about master-slave replication, which, if I understood correctly, means that the updates are one-way, in the sense that master can update the slave, but the slave cannot update the master.