the right Servers architecture for a large facebook application?

Question

We are a group of 3 students, we created a facebook application with more than 753,320 active users right now, application hosted on LAMP 1&1 server :

- AMD Opteron 1352 4 x 2,1 GHz
- 4 GB RAM.
- 2 x 750 Go (RAID 1 Hardware).
- Connection : 100 Mbps.

This application work very well, without any problem.

We are preparing a new application, and we expect millions of active users after few months.

Application information :

• Created with PHP / MySQL.
• Each user can run a minimum of 25 queries per usage.
• Serve many static files (images, flash files, css, js).
• This application contain 8 sections, for example games, gifts etc...

We want to know the right architecture for this application server.

• How many servers we need to host it ?

• If we host php files on this server :

  • Intel® Core™ i7-920 Processor 4x2.66 GHz
  • 12GB RAM

MySQL remote Server, and Static files on each Server with the same configuration.

Can the application handle millions of requests daily ?

• What is your suggestion for this kind of applications ?, can anyone tell me details of the suggested architecture ?

Thanks in advance.

Answer 1

Regarding MySQL,

1. mysqltuner is a must for any prod box.
2. Slow Query Log will get you a long way down the road to a more performant app.
3. Turning on the General log (BRIEFLY) can be a good thing, then go run EXPLAIN on all your queries to make sure you have proper indexing (no coverage, good cardinality, etc.)
4. Are you keeping session in the database? Don't do it if you can avoid it, but if not, consider a MEMORY table.
5. While on the subject of table types, consider the actual usage of each table. Transactional tables with high read/write needs might be better in the InnoDB storage engine. Tables that are predominately write or read might best be served as MyISAM. Are you logging to the database too? Consider the ARCHIVE engine for those tables.

Answer 2

Any answers you get will be wild-ass guesses. You really need to do proper load testing of your application, with realistic data and usage patterns all the way through the hardware and software stack. Use the numbers from your load test to come up with a scalabilty plan, and cost estimates Nothing scales up perfectly linearly, especially at the database layer, so there is some guesswork even when you have hard numbers, and you can hit a "wall" in a particular component (such as the database). Start with JMeter, which can capture HTTP sessions for generating load. Commercial tools are much more capable, but are also very expensive.

Answer 3

Without knowing exactly what you're doing, it's impossible to give iron-clad advice. However, I will say this:

Spend some time planning to scale now. Consider virtualization, the benefits of which are manifold.

For not a heck of a lot of money, you can rent a bunch of little VPSes from Slicehost, Linode, Rackspace Cloud, etc. Six months from now, when you're fabulously successful, you can rent/buy your own hardware, and run your own virtualization, or stick with your vendor, or move to "real" servers, or whatever. But if you think you'll ever need more than one server, code your application against a cluster setup. For the cost of your dedicated box, you can run 10 "toy" servers.

By leveraging inexpensive virtualization providers now, you can make sure your architecture can scale outwards.

If you look at your requirements, and decide you might one day need to run segmented mysql databases, you can model it now with inexpensive VPSes.

If you think you're going to want some number of load-balancers in front of a bunch of PHP boxes, you can do it now.

The same goes for setting up a few servers/balancers to serve static content (or you could go the s3/cloudfront route).

But if you're really expecting high-load and lots of traffic, it's wise to break things apart now, as opposed to buying the biggest dedicated box you can and praying that you'll be able to scale later.

Answer 4

I would say with the size of a user base the most important part of the architecture here is caching and the ability to easily scale on demand. Just look at Facebook and think about this for a second, in 2009 they had a daily data growth of 12TB! yes daily.