Tag Info

Hot answers tagged

33

Since instant updates are also acceptable, you could use lsyncd. It watches directories (inotify) and will rsync changes to slaves. At startup it will do a full rsync, so that will take some time, but after that only changes are transmitted. Recursive watching of directories is possible, if a slave server is down the sync will be retried until it comes back. ...


22

I'm sure there will be some interesting answers to this, as there is a lot of disagreement on what metrics to look at. I wrote DBCC INDEXDEFRAG, SHOWCONTIG and designed their replacements for 2005, plus wrote the Books Online content, so I'll give you my view, and explain the numbers in Books Online and the maintenance plan wizard for 2005, which I chose. ...


18

Here are my recommendations (your millage may vary) Use hardware RAID. This goes contrary to my recommendations to use software RAID in other posts, however this is a specific situation where you want the hardware RAID card. Specifically you want the battery backed NVRAM on the RAID card to reduce the time to takes the fsync the log file to disk. Use ONLY ...


18

NFS is a sharing protocol, while Rsync is optimized for file transfers; there are lots of optimizations which can be done when you know a priori that your goal is to copy files around as fast as possible instead of providing shared access to them. This should help: http://en.wikipedia.org/wiki/Rsync


17

For PHP there are 2 important things that will increase capacity: Advanced PHP Caching (APC) as was mentioned. This is what we use at Yahoo!. There are other similar projects, but this one is Rasmus' baby. FastCGI instead of mod_php. There is debate on this issue as mod_php is usually the fastest. However, I would assume that you have a single Apache ...


16

Your MaxClients is WAY WAY WAY too high. What is the current size of your apache process? Multiply that x 900. Is that greater than 4GB? If so, the machine is likely going into swap. I usually start with MaxClients = 2x vCPUs in the box (grep -c processor /proc/cpuinfo). Which in this case would be about 8. Then make sure that MaxClients x apache ...


14

In my view the less you rely on hosts file entries the better. It is far preferable to use automated systems like DNS. Your con is spot on - it reduces manageability. It also becomes error prone when things change. More than once I've known people to spent considerable time trying to debug what they thought was a DNS issue, only to discover that there was a ...


14

There are several considerations in this, some which are handled on IIS (HTTP compression, caching headers fx), and some which are handled during the build process / before deployment (such as Javascript and CSS file concatenation & whitespace minification). As such, it's a bit hard to give you a complete rundown in one answer, as some of it will depend ...


13

Maybe it's not slower transfer speed, but increased write latency. Try mounting the NFS share async instead of sync and see if that closes the speed gap. When you rsync over ssh, the remote rsync process writes asynchronously (quickly). But when writing to the synchronously mounted nfs share, the writes aren't confirmed immediately: the NFS server waits ...


13

As Chris S says the optimum block size is hardware dependent. In my experience it is always greater than the default 512 bytes. If your working with raw devices then the overlying file system geometry will have no effect. I've used the script below to help 'optimise' the block size of dd. #!/bin/bash # #create a file to work with # echo "creating a file to ...


11

Cache as much as you can. Any pages that are dynamically created should be cached so that users will get a static version. In page components that query the db should also be cached. Try using an external service like Amazon S3 to serve images and multimedia (or have it ready to use if the site suddenly gets hit with a ton of traffic). Going live ...


11

Aside from increasing the speed of light? Google isn't a good measure of network speed. The response times that you're seeing indicate clearly that you're hitting Google servers in the same country (or at least on the same continent) from both locations. Routing to the other side of the world isn't a simple affair - start with a traceroute between the ...


9

There's a very interesting write up here where someone is using IIS to serve static files. It mainly concentrates on tweaking the IIS file caching settings to limit disk activity (which was his bottleneck). He says he's seen a 20x increase in performance.


9

Once again Dave Cheney really knocked it out of the park here. I really can't add anything to his answer to your question. However, I'd like to point out what you didn't ask. As Jeremy Zawodny and Peter Zaitsev taught me years ago, your ROI for time spent tracking down and optimizing bad queries is going to out perform your ROI for time spent making ...


8

Well when your profiling anything like this to find the bottleneck you need to rule things out one by one. You'll need a baseline to get comparisons against. If you have the "ab" tool installed (it comes with apache) you can use this. To get your baseline I recommend getting the average of at least a couple of hundred requests. Here's an example: $ ab -n ...


8

The main argument I have seen against them is that files that are common to many servers will have to be grabbed over and over again by the client. Better is to include a version string in the name of the file itself. So for example, if you round robin load balance your web servers each server will generate its own ETag causing the client to grab it ...


8

Consider using a distributed filesystem, such as GlusterFS. Being designed with replication and parallelism in mind, GlusterFS may scale up to 10 servers much more smoothly than ad-hoc solutions involving inotify and rsync. For this particular use-case, one could build a 10-server GlusterFS volume of 10 replicas (i.e. 1 replica/brick per server), so that ...


8

I can't really answer the "who should I use" part of the question (as it's off-topic), but given that I do have significant experience in making sites/applications scale for high-traffic loads, I can definitely suggest that you look at getting a Reverse-Proxy CDN. The last company I worked for used Yottaa for this, and were able to use their services to ...


7

I doubt rsync would work for this in the normal way, because scanning a million files and comparing it to the remote system 10 times would take to long. I would try to implement a system with something like inotify that keeps a list of modified files and pushes them to the remote servers (if these changes don't get logged in another way anyway). You can ...


7

Application security is a sticky, many-armed tentacle monster of a problem. If you're concerned that your web service will be a target for attack due to the subject matter, then by all means use Apache. ModSecurity acts as a web application firewall, analyzing incoming traffic for malicious patterns and blocking as necessary. This is no replacement for ...


7

I think you are misunderstanding how TCP works. Each packet sent will always advertise a receiver window (aka. RWIN) and an optional scaling factor, see RFC 1323 The sender is not allowed to send more than the amount of data specified in the RWIN without it getting acknowledged. Depending on the congestion window, the sender may decide to fill up the RWIN ...


7

"if you have a quad core, your load average is usually around 4.0" Incorrect, load averages is how long it takes each process to get a slot at the CPU (It's a bit more complicated than that, it actually involves the CPU 'queue' but saying time is a lot easier than trying to explain that). As a highly simplistic example, when running Apache, the load average ...


7

Make the swap / paging file a fixed size. This will eliminate the overhead of auto-growing the paging file. It also reduces fragmentation. Control Panel -> System -> Advanced -> Performance Settings -> Advanced -> Change. In the windows performance settings, Control Panel -> System -> Advanced -> Performance Settings, turn off all of the cute effects, ...


6

I disagree wholeheartedly with Kyle. If it is not necessary, it should be removed. It's a best practice to not install unnecessary software. The person undertaking the task, such as yourself, should be confident in the implications of the decisions they are making. Removing standard system utilities and libraries is generally frowned upon but that will ...


6

Move your PHP session files to a tmpfs, use APC ( or other ) and remove all PHP modules that you do not need. Remove all Apache modules you do not need/use. To create a tmpfs (a directory in RAM! ) mkdir /tmpfs; chmod 777 /tmpfs mount -t tmpfs -o size=256M tmpfs /tmpfs In /etc/fstab add the line below to create it on reboot! tmpfs /tmpfs tmpfs ...


6

You can't have a server which is "too optimized". With load average of 0.8, it means some of your server is idle. For a quad core your load isn't "usually around 4.0". However with a quad core, you usually don't have to worry about performance if your load is under 4.


6

That's actually remarkably good - the higher the better. The MySQL performance is good if the value of Key Read Efficiency is 90 percent and above http://www.webnms.com/mysql_agent/help/mysql_agent/performance.html


5

My recommendation: forget minify and use the gzip module. It will work better and accomplish the same goal. But, as of course, you can do it. There's a 3rd party module named strip, especially for this topic.


5

I don't believe you will notice any hit in performance. Log writes will likely be buffered and then flushed out to disk. So unless you have high disk load you will be fine. If you do have high disk load than you probably want more memory (if this is only a webserver) so web data is served from a memory cache. Nginx is programmed with an event driven ...


5

So, you are kind of there. Although it only tangentially relates to IP. What you are really looking for is a second domain. What happens is your web browser only opens 2 (IIRC) connections to a domain at a time to pull down content. So what happens is if you have a bunch of static content, images, css, javascript files, etc at most you can pull down 2 at a ...



Only top voted, non community-wiki answers of a minimum length are eligible