The tag has no wiki summary.

learn more… | top users | synonyms

51
votes
8answers
100k views

What limits the maximum number of connections on a Linux server?

What kernel parameter or other settings control the maximum number of TCP sockets that can be open on a Linux server? What are the tradeoffs of allowing more connections? I noticed while load testing ...
16
votes
5answers
25k views

How many requests should my webserver be able to handle?

Not going into specifics on the specs since I know there is no real answer for this. But I've been doing load testing today with the ab command in apache. And got to the number of 70 requests per ...
15
votes
4answers
5k views

Which database server scales better: PostgreSQL or MySQL?

I'm working on a website that could eventually have a very large database. The website will be an art community not until deviantART. I'll be running a lot of queries for analyzing how the site is ...
13
votes
5answers
727 views

How do services with large ingest rates install enough hard disks fast enough?

Amazon, Facebook and Google must take in TBs if not PBs per day, so does that mean they have people continuously installing new hard disks and cables in new racks in the same way you might build a ...
10
votes
2answers
11k views

Tips for maximizing Nginx requests/sec?

I'm building an analytics package, and project requirements state that I need to support 1 billion hits per day. Yep, "billion". In other words, no less than 12,000 hits per second sustained, and ...
9
votes
8answers
520 views

More RAM vs. more servers

I was recently asked "Do you know when to decide between going for more RAM or more servers?" (in the context of scaling data mining applications). I had no idea, so what are some ways to decide? I ...
9
votes
1answer
4k views

Recommendations for Elastic Search hardware [closed]

Are there any good guides for hardware level to support ElasticSearch? Are recommendations for Lucene or Solr a good place to start? We're looking at rolling out a deployment starting with 27 ...
8
votes
2answers
3k views

Is there a way to add more backend server to haproxy without restarting haproxy?

We want to be able to add more backend servers on demand. Right now I don't see a way to add more backend servers to the config file without restarting haproxy.
8
votes
2answers
986 views

Large VPN network (~600 servers) with OpenVPN

I'm doing a preliminary study for a contract to build a VPN network between ~600 remote servers running Linux CentOS 6 (+ their 600 private LANs). The network is supposed to be star-based, so that ...
7
votes
10answers
15k views

Where to purchase a server capable of 512GB of RAM? [closed]

Where can I purchase a single server capable of 512 GB, yes GB, of RAM? The most I've seen is 256 GB (32 DIMM x 8 GB per stick)
7
votes
3answers
4k views

Postgres vs MySQL: Which one is more difficult to scale out?

From your experience, which database serve is more difficult to scale? MySQL or Postgres? I know MySQL has several out-of-the-box scaling/clustering features, but with Postgres there is none that ...
7
votes
1answer
2k views

How do I scale my EC2 instance?

I'm very new to server configurations and Linux in general, so please bear with me: I have a small EC2 instance running a LAMP server on Ubuntu 10.04 LTS. I'm completely baffled as to how I would ...
7
votes
3answers
1k views

The best way to share a file uploads directory across horizontally scaling web servers

I'm currently trying to spec up a horizontally scalable cluster for a drupal based web app, that looks something like the colorful diagram below: The load balancer implements sticky sessions, so a ...
6
votes
5answers
3k views

How to identify heavy write to disk?

I have this problem with server running CakePHP application. The server is insanely slow, I first thought that it's application problem, but then I found constant 5-6MB/s write to disk. What is the ...
6
votes
3answers
2k views

How to allow elastic load balancer through port 80 in security groups?

I want to temporarily block port 80 for the outside world, but want the load balancer to pass through the firewall (via security group) so it won't see the instance as unhealthy. How can I do that? ...
6
votes
2answers
286 views

AWS - how to serve faster to oversea users

We have our infras relying on AWS in the us-east-1 region. (EC2, CloudFront, RDS, ElastiCache) We are now having more and more users from the APAC. Users start to complain about network speed to our ...
6
votes
3answers
460 views

how to go about scaling a web-application?

I am primarily a web-application developer, and I do not know much about scaling/scalability techniques. My application is written in Python, using Django; a fairly standard setup. I currently use ...
5
votes
5answers
3k views

AWS: Multi-region setup using single RDS instance

I'm trying to scale our web application (PHP, MySQL, memcache) in a multi-region scheme. Currently we are using a setup with two EC2 instances behind an ELB and an RDS instance, all of them in US-EAST ...
5
votes
5answers
3k views

When is the right time to add a separate database server, add more web servers?

Usually web projects start small, everything is on the one server. But if site becomes popular one server won't be enough. So, since I don't have any experience in this, I'd like to get an idea of: 1. ...
5
votes
2answers
205 views

HAProxy not balancing server load fairly

HAProxy doesn't seem to keep the connections to the servers balanced. Keep this in mind: using HAProxy v1.3.26 5 equally balanced server specs algorithm is round robin, but no weights are applied ...
4
votes
4answers
3k views

Load balanced Drupal servers: a good way to share data among the servers?

I'm trying to get an overview of how i would go about creating a load balanced web server setup. Setting up the actual load balancer and adding two or more web servers seems fair enough. However, I ...
4
votes
4answers
266 views

Multiple web servers sharing a db server

I am contemplating migrating to mosso to have several cheap app servers. The goal is to have a dedicated database server, however, does it make sense to have one database server that hosts the ...
4
votes
5answers
1k views

HAProxy and “sharding”

I am cross posting this from stack overflow since it was suggested I might get a better answer here ... I was wondering if anyone has used HAProxy for sharding. Specifically, I would like to be able ...
4
votes
1answer
221 views

Sharding MongoDB internationally with data stored based on location

We're looking to set up data centers globally and switch to a NoSQL database like MongoDB to support having our data distributed over these data centers (and as a pleasant side effect, introduce more ...
4
votes
3answers
381 views

I know how to scale my software, but how to prevent downtime because of network outages? [duplicate]

We are running rather large LAMP sites which scale well software wise. We use redundant load balancers in front of a bunch of webservers using MySQL via a proxy in master-slave-slave-slave. We are ...
4
votes
3answers
187 views

First steps in scaling servers up for a Ruby on Rails application

I'm running a RoR web application on a dual core AMD Athlon 4450e server w/ 8GB RAM and Ubuntu. It's doing alright, getting spikes up to 320 dynamic requests per minute, it's also handling all the ...
3
votes
6answers
2k views

How can I have 2 database servers for my website?

I have a site that has heavy traffic and my database server (mysql) is getting heavy traffic at peek hours. Instead of upgrading to a better server, I am thinking of having 2 database servers working ...
3
votes
3answers
5k views

Suggestions on large scale web applications architecture?

It's a big question:) We are running a website with LAMP that is not big, 5 web servers with LVS load balancing, 3 MySQL servers with replications and separation of reading and writing, and we use ...
3
votes
1answer
2k views

How can I make Microsoft Management Console high-DPI aware in Windows 8.1?

Since Windows 8.1 doesn't allow system-wide "Windows XP style" high DPI support, how can I make the Microsoft Management Console apps (mmc.exe) high-DPI aware? There is no "Troubleshoot compatibility" ...
3
votes
4answers
1k views

How to choose size for a cloud server (rackspace)

We're going to test the rackspace cloud next week to see how it's working with our web app. It's a LAMP environment with a lot of MySQL databases. How do I choose the "right" server size? On ...
3
votes
1answer
414 views

Firewall as a guest VM?

Out of curiosity, would it be pointless/wasteful/silly to put a firewall as a VM guest (regardless of VM host - ESX, Xen, Hyper-V, etc. etc.) and redirect all traffic from other VM guests through the ...
3
votes
1answer
6k views

How to horizontally scale SSL termination behind HAProxy load balancing?

I've been looking around and no one seems to be trying to scale SSL termination the way I am and I'm curious as to why my approach seems so uncommon. Here's what I want to do followed by why: ...
3
votes
2answers
363 views

Memcached scaling strategy

Currently I am running a production environment with 4 dedicated memcached servers, each of them having 48Gb of RAM (42 dedicated to memcache). Right now they are doing fine, but traffic and content ...
3
votes
2answers
239 views

Virtualization as a way for scaling

I'm asking this question to find out what are you best practices to prepare your system for scaling, I know that basically you don't have to over optimized before really having to, and that's the best ...
3
votes
2answers
242 views

RT database scaling

Recently I heard someone suggest that RT request tracker may have scalability issues due to its non-normalized database (someone at a Perl meeting I went to referred to it in a positive light as ...
2
votes
4answers
861 views

How to scale down for low memory server?

I'm having problem with one of my Rails applications, that runs on 365MB VPS, which seems to be not very much. When there are more than 3-5 users at the same time it runs out of memory and starts to ...
2
votes
3answers
2k views

MySQL Hardware Specs

Currently we have a database that is 35gb in size and is running on a Virtual Machine with 2GB of RAM. The database is expected to increase at a rate of 3 to 4Gb per day continuously. What are the ...
2
votes
3answers
738 views

MySQL : table organisation for very large sets with high update frequency

I'm facing a dilemma in the choice of my MySQL schema application. So before I start here is a picture extremely simplified of my database : Schema here : http://i43.tinypic.com/2wp5lxz.png In one ...
2
votes
1answer
979 views

Can I use MongoDB with Amazon EC2 small instance?

I want to move my MongoDB from a virtual server to amazon web services. I know MongoDB provides a preconfigured AMI, with three provisioned EBS volumes. But this is only available for m1.large ...
2
votes
4answers
552 views

What should I do to scale OUT a Sql2008 database?

I'm about to scale UP our sql2008 database. Simple. But I might need to scale OUT our sql databases. For a simple scale out situation (ie distributing the processing load), are there some good ...
2
votes
3answers
5k views

Maximizing TCP connections on HAProxy load balancer

I am currently using HAProxy in order to load balance tcp connections from clients to my Erlang app server. The connection is persistent, which means I'm limited to roughly 64K clients on an optimized ...
2
votes
1answer
308 views

Scaling by moving Database to Amazon RDS

We have a situation with an Application which is hosted on a dedicated server where we are anticipating a potential surge in traffic. The major bottleneck on the app is MySQL which is heavy on read ...
2
votes
3answers
2k views

Best way to scale up an IIS/SQL Server application

We have an application that we developed in ASP and SQL Server. We use Rackspace to host it. Each of our "customers" has their own IIS site and SQL database. Each customer could have a dozen or ...
2
votes
1answer
95 views

File storage for replicated front-end web servers?

We are a company running a website with ~800k unique visitors each month with a constant growth rate of 35-40%. We have just gone from having one server to a dedicated mysql database and an ...
2
votes
1answer
180 views

Scaling advice LAMP + Load balancers

My current setup: 1 Http Load Balancer 3 Apache Frontend Server 1 Mysql Load Balancer (Balancing reads across all three, writes go to master only) 3 Mysql Database Server (1 master, 2 slave) All ...
2
votes
1answer
552 views

How to properly loadbalance separate fastcgi servers

We are running bigger deployment with multiple servers running Django applications under apache2 and mod_wsgi. We are considering switch to apache2 + fastcgi and moving fcgi processes to separate ...
2
votes
2answers
2k views

Why do we get TCP disconnects when using AWS Elastic Load Balancer?

We are running our servers on AWS EC2 instances. The servers have to provide HTTP on port 80 and RTMP (TCP) on port 1935. Running a single instance works fine. But as soon as I use Elastic Load ...
2
votes
3answers
2k views

Legacy non-dpi-aware application resolution scaling?

Our environment prominently featuers an outdated but absolutely mission-critical Win32 application that is not dpi-aware. It is optimized for an 800x600 display. Most of our users now have 17"-20" ...
2
votes
0answers
46 views

scaling website to perform equally fast worldwide [closed]

Our website is growing and we get more and more customers all around the world. We use Nginx, PHP and MySQL Our servers are in UK, so it is fast in Europe, but the further we got away from servers - ...
2
votes
1answer
96 views

MongoDB PHP EC2 Setup Configuration

I am new to web development and server set up. I am looking for some advice or a link to a tutorial on setting up a production system up. Right now, I have a server (Ubuntu, Apache, MongoDB, and ...