Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am developing a web application which will require scaling to multiple servers. I'll need multiple web servers running my application code, and there will be heavy reads and heavy writes to the database. Clearly I cannot just use one database.

I've dealt with database replication where there was a single master and multiple slaves. With that, we had multiple read-only API servers, each with a dedicated server and a dedicated (replicated) slave database.

But how would I deal with writes? It seems like it would be a bad idea to have all writes happen to one single database due to bottnecking, and instead I'd want write to multiple separate databases. But then I'd have multiple master databases, and I can imagine it would be a disaster if the masters became out of sync. And, I could imagine there would be primary key collisions.

Another option may be to have multiple databases with the same schema, but each is not a copy of the other. This sounds scary though, as one could crash, and I would lose data. This seems like a maintenance nightmare, but maybe I am wrong.

What would be a good, tested, practical, scalable architecture for the needs I described? Links to articles or just posts here would be great.

share|improve this question
3  
How big is your application going to be? Stack Overflow has gotten a lot of mileage out of using a single database instance (in their case Microsoft SQL Server). I wouldn't discount the possibility of using a single-master RDBMS. –  Evan Anderson Nov 11 '10 at 23:08
1  
Good to know. A co-worker told me a very similar thing: if you optimize and have proper hardware, you can go a long way with a single database. The application will be pretty large, and it will potentially have a ton of users. See gnymb.us for an idea of what I'm talking about. –  Chad Johnson Nov 12 '10 at 5:51
add comment

3 Answers

There's nothing remotely close to one-size-fits-all here, but if you're looking to move beyond traditional relational database solutions, then I suggest you Google "NoSQL" and start reading.

share|improve this answer
add comment

short answer: sharding

long answer: you either develop your own "nosql" layer on top of a sql db (usualy mysql) or you use one of the many hot topic nosql tools (mongodb, cassandra, redis, etc).

however since your post included zero metrics its extremely unlikely you actually need to do this, if you did you'd be in numbers-mode. Most likely you'll get by with a healthy dose of ram and some time spent tuning sql queries, or, a couple grand worth of SSD/high-end-storage.

share|improve this answer
1  
Dont shard unless "you must" Sharding is effective and it works.. However its a pain in the *** to administrate and maintain.. You should never plan to shard, or look to shard.. you should only ever shard when you MUST... –  Arenstar Nov 12 '10 at 0:05
    
Good information. I wasn't aware of sharding. Research time... –  Chad Johnson Nov 12 '10 at 5:50
add comment

If you can fit your data differently to standard relational tables than i would consider NoSQL ..

A heavy write style cluster data store would be cassandra ( NoSQL ) It is designed to be write non-blocking... :D

Otherwise.. Attempt sharding with SQL and as the bigger companies sometimes do, run mysql in a master<---> master configuration..

What type of data is written and read?

share|improve this answer
    
Files are being written in a web IDE for my project at gnymb.us. I'll have another look at NoSQL... –  Chad Johnson Nov 12 '10 at 5:49
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.