The ability of a system, network, or process, to handle growing amounts of work in a graceful manner or its ability to be enlarged to accommodate that growth.

learn more… | top users | synonyms

1
vote
0answers
109 views

How can I scale my RoR applications?

I am developing a Ruby On Rails application and would like to deploy in a production environment. I have multiple identically configured Ubuntu web servers I can use but I don't know how to scale ...
1
vote
3answers
179 views

What kind of server is used by sites like Pandora or Last.fm to serve media files?

What kind of server is used by sites like Pandora or Last.fm to serve media files? I am assuming they would have something other than webserver for storing and serving media files... what are some of ...
2
votes
3answers
2k views

What are low, medium and high traffic websites in terms of visits? [closed]

We often read on the internet about big famous website such as Facebook, Google, Yahoo... having hundred of million of unique visitors each month. But what is the order of magnitude for smaller (or ...
1
vote
1answer
470 views

Tomcat scalability - how many servers

I'm trying to size a tomcat cluster. The key factors will most likely be: The size of clustered in-memory cache we decide to use. So for example, if we wanted to support 100GB in-memory cache, then ...
2
votes
2answers
190 views

Designing a web application to scale

While designing a web application facebook application to be precise. Which can spike and increase rapidly because of it vitality and is right intensive. What point should one keep in mind while ...
2
votes
1answer
1k views

Does PHP *have* to serialize/unserialize session data between each HTTP request? Or is there a setting just to keep sessions in memory?

I think I understand why sessions are evil but for snappy client user experience I don't want to have to re-query the database on each HTTP request. (As a comparision, Java servlets can effortlessly ...
0
votes
2answers
978 views

NFS or GFS for LVS 10 Server Setup

Currently we have a 10 servers setup as a linux virtual server. We have: 2 Load Balancers (master / slave) 3 Web Servers 2 MySQL Servers (master / slave) 2 Central File Storage Servers (master / ...
1
vote
3answers
786 views

google search engine architecture- how do so many concurrent users do a search on it [duplicate]

With millions of users searching for so many things on google, yahoo and so on. How can the server handle so many concurrent searches? I have no clue as to how they made it so scalable. Any insight ...
3
votes
2answers
3k views

Server Clustering (Django, Apache, Nginx, Postgres)

I have a project deployed with django, Apache, Nginx and Postgres. The project has requirement of live data viewable to customers. The projects main points are: 1. Devices in field send data to ...
7
votes
4answers
8k views

postgreSQL vs Cassandra vs MongoDB vs Voldemort?

Which database to decide upon? Any comparisions? Existing: postgresql Issues Not easily scalable horizontal. Needs sharding etc Clustering does not solve the data growth problem ...
1
vote
2answers
406 views

What techniques do you use to design and scale a web app horizontally (in EC2)?

Specifically I'm looking for techniques for scaling a web application which has no central database server, in the cloud, but general advice is great. I have come across GlusterFS, which looks great, ...
2
votes
1answer
206 views

What database is easy to maintain and manage in a cluster?

I'm looking for a database (DBMS) that is easy to scale out. I would like to have high availability so I need a multi-master cluster, where the data is replicated to two or more physical computers. I ...
0
votes
5answers
1k views

I have to shard a mysql database. I want to start with 12 shards on 2 machines. What is the best way to move some of them when I want to add a new server?

All tables are InnoDb. I would rather not use mysqldump, because the shard sizes will be about 200 GB (about 700 million rows), and that will take too long. I was hoping to just stop mysql for an ...
1
vote
3answers
1k views

Sharing / replicating EBS across AWS nodes

I would like to use single EBS storage across multiple EC2 nodes (web/app servers). I've read some articles on snapshot sharing, but that doesn't suit well for what we need. We use filesystem for ...
3
votes
1answer
2k views

scalability: when to use CDN?

i've read about CDN but dont know exactly what it is for. lets say i've got an international social network (text and image content), and it's growing in traffic from different countries, do i have ...
0
votes
2answers
679 views

Linux/bsd tcp load balancing with 10 gigabit ethernet

Okay, I've been looking at layer 4 load balancing solutions for 10 gigabit links. I need the following properties Works at 10Gig ethernet speeds. Can support long live tcp connections. up to 1mil ...
4
votes
1answer
699 views

how does a web application cope with thousands of requests?

I went to a few websites and noticed that they all use AJAX technology for many tasks such as chat, messages, and so forth. They use a lot of httprequests obviously. My question is if you build a ...
3
votes
2answers
995 views

What is optimal hardware configuration for heavy load LAMP application

I need to run Linux-Apache-PHP-MySQL application (Moodle e-learning platform) for a large number of concurrent users - I am aiming 5000 users. By concurrent I mean that 5000 people should be able to ...
0
votes
1answer
286 views

How to manage static content with dynamic

I have a lot of static pages, which include html files + js (SCORM packages). I have dynamic pages, also with JS which talks to the static pages JS. Static pages are being opened in an iframe, and we ...
4
votes
2answers
319 views

10k-20k TCP connections on windows server(s)

What kind of configuration do I need for a Windows based server(s) to support 10K-20K TCP connections simultaneously. 99% of these connection will only send keep alive (but will remain active) signals ...
30
votes
9answers
2k views

What's the piece of hardware listening on Facebook's or Wikipedia's IP address?

I am trying to understand how massive sites like Facebook or Wikipedia work, for my intellectual curiosity. I read about various techniques for building scalable sites, but I am still puzzled about ...
1
vote
4answers
2k views

How to scale beyond 150 page views per minute?

I have a Facebook app written in PHP. It has 150 page views per minute and will have up to 300 page views per minute till end of this year. While getting more PV I start to have problems with ...
4
votes
2answers
231 views

When do I need more than shared hosting?

What are the guidelines? While the answer could be "when your webpage or queries return slowly -- that answer doesn't quite work for me. What are your reasonable expectations for shared hosting? How ...
4
votes
8answers
3k views

Multiple servers vs 1 big server performance

My team of developers has suggested a server structure for an upcoming project we are developing. Our structure is "logical", meaning that the various logical components of the application (it is a ...
0
votes
1answer
330 views

How can I measure the load capacity of my video encoding server and plan for scaling?

I have a server running on Amazon EC2 that acts as a video encoder (LAMP CentOS + FFMPEG). How can I load test it with video files and get an idea of how many encodings it can handle. I can't seem to ...
4
votes
6answers
2k views

SVN limitations - Number of users and repositories?

I would like to know if SVN has a specified limitations of how many users and repositories it can sustain? Thank you.
0
votes
2answers
837 views

how many databases can a single server handle?

I heard that in general, you don't see a single server running sqlserver having more than 100-150 databases on it. Is this the same case with mysql or postgres?
1
vote
4answers
256 views

file share system architecture asking for advice

I am using Windows platform to setup a web based file share system. In more details, individual users could upload and share documents from web interface (content may be big document, like video ...
1
vote
3answers
99 views

Should I start hosting my web server in more that one single server machine?

I'm developing an advertising site for the real estate business. The thing is that it is a startup business and I'm not too positive in thinking that I will get a lot of users very quickly. So, my ...
0
votes
2answers
138 views

Efficient large file delivery over web

In the last few weeks we started serving more and more videos by http, ranging from 50 to 500mb each, and as we didn't had a new server to use it just for that i set it up on our live streaming ...
0
votes
2answers
352 views

Where could I find some network architecture courses or good books? [closed]

The best source that I found is not really a course, is most like a blog with short stories about succed cases of architecture strategies in highscalability.com My concern is if is there any good ...
5
votes
4answers
1k views

Best way to improve resilience?

One of the major projects I have lined up for 2010 is attempting to mitigate some of the Single Point of Failures (SPOFs) in a network I currently manage. We currently have a single datacentre rack ...
5
votes
6answers
1k views

(*nix) Cloud/cluster solutions for scalable web-services

I'm going to build a high-performance web service. It should use a database (or any other storage system), some processing language (either scripting or not), and a web-server daemon. The system ...
2
votes
6answers
12k views

Where does youtube store so many videos?

If we are to make something like youtube, then what should be the approach with respect to storing the videos? Any idea about what youtube does ?
1
vote
1answer
162 views

Create a database for scalability

I know this sounds like a programming question which means it should be on stackoverflow but i believe server admins worry about this more and i am not looking for just the programming answer to this ...
0
votes
2answers
425 views

I have one database (mysql). What is the best “failover” strategy?

Does MySQL come with a built-in automatic failover? Or, are there load-balancers out there that can "detect" whether the database is alive...and if not...shoot it to the failover database? I know ...
10
votes
9answers
1k views

Scalable (> 24 TB) NAS for research department

I have been asked to provide a new fileserver to store our medical images (raw format, not dicom). As we have no IT staff, I have to find a solution, and I hardly know anything about this topic. The ...
1
vote
1answer
445 views

Which requests cause a w3wp process to grow considerably?

On a production environment, how can one discover which Asp.Net http requests, whether aspx or asmx or custom, are causing the most memory pressure within a w3wp.exe process? I don't mean memory leaks ...
3
votes
4answers
276 views

Is it normal to get bandwidth as the first scalability bottleneck with a 100 Mbps connection?

I'm stress testing a site that we're making, and we're finding a very surprising result compared to my expectations: Our site starts to load very slow with a few hundred simultaneous people, even ...
5
votes
3answers
2k views

need advice on building a scalable architecture for moodle

I'm looking into designing an architecture for a moodle based education site, it will serve several thousand users at first but needs to be able to grow to support hundreds of thousands to millions of ...
6
votes
2answers
2k views

Scalable CouchDB setup

I want to migrate an existing application that has approx. 10 million records stored in a relational database to CouchDB. The thing that I love about CouchDB is easy replication and fast cached views. ...
8
votes
9answers
10k views

How to solve linux subdirectories number limit?

I have a website which will store user profile images. Each image is stored in a directory (Linux) specific to the user. Currently I have a customer base of 30+, which means I will have 30+ folders. ...
2
votes
4answers
8k views

How many databases can SQL server express handle

I'm running a SQL EXPRESS 2005 server currently hosting ~50 databases. The databases serve clients' CMS/eCommerce websites. The connections are to a single instance, no user attached instances are ...
1
vote
3answers
241 views

Ad server software for high traffic site

The company I work for runs a web site that gets a little over fifty million hits a month. We are looking at replacing our existing custom-written ad server with a prepackaged solution. It ...
0
votes
1answer
177 views

Avg AntiVirus Professional Scalability?

With the Network editions of AVG Anti virus for business, how well does it scale? How many users can it support on one network? I have heard a few stories of business's with about 3000 computers under ...
18
votes
10answers
33k views

Running 100 virtual machines on a single VMWare host server

I've been using VMWare for many years, running dozens of production servers with very few issues. But I never tried hosting more than 20 VMs on a single physical host. Here is the idea: A stripped ...
0
votes
7answers
511 views

High performance web server with no database interactivity

I'm getting ready to setup a server that will be responsible for tracking statistical data from a high volume traffic source. It will be handling requests at about 6-7mil/hour on average, all of ...
3
votes
6answers
3k views

How do I reduce RAM usage on my server?

I have recently launched a site that is very popular but I am having trouble with scalability. My site makes heavy use of FFmpeg and at peak times RAM usage hits the 2 GB point quickly and the swap ...
1
vote
2answers
199 views

Enterprise scaling - hardware & server tips

Things change when you are delivering to a large enterprise audience. What are the most important things to think about in relation to your server or server farm when scaling to a very large user ...
23
votes
6answers
3k views

Scaling databases with cheap SSD hard drives

I hope that many of you are working with high traffic database-driven websites, and chances are that your main scalability issues are in the database. I noticed a couple of things lately: Most large ...