if blog_id > 50000 use 2nd server

Question

I am trying to understand possibilities if i need in future. I am preparing to open a network on WordPress MultiSite with HyperDB. I want to let users open subdomain sites with registration and i need a solution for using a 2nd server when need.

There is 1 main site and subdomains are not connecting each other. I need somehow forward requests to other server if blog_id > 50000 There is some "old" duscussions about it: http://codex.wordpress.org/How_to_Scale_WPMU

The idea is to receive all incoming requests on a single HTTP server. This server, using mod_proxy and mod_rewrite, will route requests to X backend servers, acting as a reverse proxy.

Well, I am barely managing a server and really newbie on this area. How can i do this?

Answer 1

The 'How to Scale WPMU' page you link to proposes a solution which seems like a nice solution but I question how well it will scale. What they do is fairly straight forward. They use the host name header to let the reverse proxy decide to which server to route the request. Basically they look at the first letter of the blog name and then route the request to 1 of 26 servers, one for each letter of the alphabet.

This potentially doesn't scale very well. First of all because some letters in the alphabet will be more popular than others. There will be far less blogs starting with the letter q or x than for example with the letter e or m. The second problem is that if one or a few blogs starting with the same letter are very popular that one server will get a much higher load than the 25 other servers. The other servers might be at a very low utilization while that one server can not cope with the traffic. Not very good use of the available resources.

Load balancing at such a scale is not a very easy task and requires a lot more intelligent logic at the reverse proxy. This also is true for the solution you are trying to build. If you need to forward requests based on the blog_id you need some custom logic at the reverse proxy that can look up the host name to find the blog_id and then decide to which server to forward it to. I don't know if this kind of logic would be possible at all using Apache. I'm more of an IIS specialist and to build such a solution with IIS it would be necessary to write a custom rewrite provider. Not that hard to do but also doesn't tackle the scaling problem.

To solve the scaling problem you need additional logic that can dynamically add servers to blogs or groups of blogs that at that time receive a lot of traffic. This requires synchronization between the reverse proxy and a provisioning/deployment system that can dynamically add/move servers to a pool of blogs. Blog platform Squarespace has made this one of their unique selling points.

Answer 2

You should scale this the simple way:

1. Start of on a single server with local MySQL service.
2. Later move your database service to a dedicated instance.

Cloud computing can be nice for this kind of thing – start off slow and have computing resources always available. You could also look into a provider who manages servers for you and try to build a relationship with them so they can help you along the way.

Either way there are plenty of things to do (and learn):

• How to scale PHP (e.g. you probably use Apache first, then later something like nginx and php-fpm).
• How to scale MySQL – the defaults usually don't utilize the available resources very well, etc.. Down the road you could set up master-master MySQL setups and read-only slaves to scale this further.
• Implementing a cache – e.g. WP Supercache or something more aggressive like Varnish which could cache and serve requests for blogs without any PHP or database interaction required. This can work really well when updates are not as frequent. Of course measures have to be implemented to invalidate the cache when an update (new blog post, edited blog post) happened.

All of this will work for a while, but there is no ultimate fix. Or no "one fits all solution".

I'm not sure how much logic is required here and doing it automatically doesn't work really well.

The problem you will most likely have is, that some blogs get more busy then others. Something to think about is how you can easily move a blog from one server to another, e.g. to a new server (dedicated to it or one that is more idle).

Same applies for your database service, you can scale it out with replication only so far. One of the things you might need is to setup a dedicated instance for more busy clients, etc.. I'm not sure how WP MU works on the backend to answer how much of this can be automated or what exactly is required to get this done.

Bottom line: I wouldn't overdo it and I'd play it by the ear. I'm not sure if you can predict your success, but I wouldn't invest money in infrastructure unless it is really required.

Last but not least, you sound like a beginner from what I gather. So don't take me wrong, but I suggest you either take a lot of time and take some courses for system administration and become a PHP developer (and database administrator) along the way, or you figure out a way to hire people who can solve these problems for you.

HTH