Questions tagged [optimization]
Optimization is the process of modifying a system to make some aspect of it work more efficiently or use fewer resources.
340
questions
46
votes
7
answers
56k
views
Why rsync is faster than NFS?
Few days ago I noticed something rather odd (at least for me). I ran rsync copying the same data and deleting it afterwards to NFS mount, called /nfs_mount/TEST. This /nfs_mount/TEST is hosted/...
30
votes
3
answers
79k
views
Optimal values for ServerLimit, MaxClients, MaxRequestsPerChild directives
I'm running a traffic intense site plenty of dynamic content, mostly user-generated.
The server is a dedicated one and has a total of 4 Intel(R) Xeon(R) CPU X3210 @ 2.13GHz proccesors. I need to know ...
29
votes
5
answers
4k
views
What configuration options for MySQL provide the biggest speed improvements?
What configuration options for MySQL provide the biggest speed improvements?
I'm wondering about actual configuration file improvements, table types, hardware setups, replication, etc. Anything other ...
28
votes
3
answers
89k
views
Maximizing rsync performance and throughput - directly-connected gigabit servers
I have two Dell R515 servers running CentOS 6.5, with one of the broadcom NICs in each directly attached to the other.
I use the direct link to push backups from the main server in the pair to the ...
27
votes
7
answers
12k
views
Options to efficiently synchronize 1 million files with remote servers?
At a company I work for we have such a thing called "playlists" which are small files ~100-300 bytes each. There's about a million of them. About 100,000 of them get changed every hour. These ...
27
votes
3
answers
35k
views
How to determine the best byte size for the dd command
I know that doing a dd if=/dev/hda of=/dev/hdb does a deep hard drive copy. I've heard that people have been able to speed up the process by increasing the number of bytes that are read and written at ...
25
votes
7
answers
56k
views
How many files in a directory is too many? (Downloading data from net)
Greetings,
I'm writing some scripts to process images from various photo websites. Right now I'm storing all this data in individual text files in the same directory.
The directory is web ...
20
votes
2
answers
13k
views
IIS 7.5 optimizations for a site serving only static content
I'm looking to setup a cookie-free domain intended to serve static content for a web application, similar to the http://sstatic.net/ site that the stack exchange sites use.
My question is, what ...
18
votes
3
answers
9k
views
Building a 1,000M row MySQL table
This question is reposted from Stack Overflow based on a suggestion in the comments, apologies for the duplication.
Questions
Question 1: as the size of the database table gets larger how can I tune ...
17
votes
6
answers
18k
views
Optimize apache/php/mysql running on VPS for heavy load
Question about optimizing an apache/mysql server on a VPS with 512m of RAM. Under normal load everything runs fast, no connection lag. However when we get our heavy traffic days (50k+ visits) the ...
15
votes
5
answers
16k
views
How to minimise SpamAssassin (spamd) memory use
I'm using SpamAssassin on Debian (the default configuration with Pyzor, AWL and Bayes disabled, and sa-compile enabled), and each of the spamd child processes consume around 100 to 150MB of memory (...
14
votes
6
answers
551
views
First days online : how not to kill your site
Suppose you have this fancy new site, with lots of data (like big images), and you're about to put it online. If you do "too much" publicity, during the first days, the site will be overwhelmed with ...
14
votes
1
answer
18k
views
How to disable keys in MySQL InnoDB tables?
Is it possible to disable keys in InnoDB tables?
If yes, how?
If not, why?
12
votes
2
answers
25k
views
Nginx auto minify html output
does anyone know of a way to get nginx (or any other method) to minify html output on the fly? It seems straightforward to me and can shave a few kb off and help speed up the site.
10
votes
2
answers
4k
views
Linux Slow Start: Changing ip route does not have any effect on initial window
I changed the tcp initial window in my machine to 10 as shown below
[user@site etc]$ sudo ip route change default via 17.255.209.1 dev eth0 proto static initcwnd 10
And changed ...
10
votes
1
answer
4k
views
When should you use and not use Etags?
I was just looking at our site on WebPageTest.org and one of their recommendations for speeding up a website is:
ETag headers should generally not be used unless you have an explicit reason to need ...
10
votes
2
answers
43k
views
What are typical NFS read/write rates
I have just installed a server that is sending HTTP video streaming files to a Web server over NFS and I want to make sure that I have the transfer speed optimized. But I don't know what sort of read/...
9
votes
1
answer
1k
views
How can I tell when it is time to defrag my SQLServer database?
What is the best practice for measuring SQLServer database fragmentation/performance rolloff and determining when to defrag a SQLServer database table?
I'm most interested in learning what the useful ...
9
votes
1
answer
25k
views
Apache prefork optimization - Choosing the right `MaxRequestsPerChild` value
I'm trying to optimize our web servers to handle as much connections as possible. I read a many posts and the Apache notes. I'm trying to understand which value should I choose for MaxRequestsPerChild....
9
votes
2
answers
2k
views
Speed up MySQL inserts with partitions on MyISAM with unique key
I have a large MyISAM table (~30M rows). At some point I've switched it to fixed row format, so now table takes ~40Gb on disk and 2Gb for indexes. Table has a unique index and there are 100 'insert on ...
8
votes
6
answers
4k
views
At what point is a server considered idle?
Conceptually, what is the definition of an idle server?
What resource metrics would you look at to make an assumption on whether or not a server was idle?
Would you look at:
CPU utilization
Disk ...
8
votes
4
answers
11k
views
Profiling Apache+Mysql+Php server - which is the bottleneck?
How do I profile an Linux + Apache + Mysql + Php server for speed?
I have a server with a heavily modified MediaWiki instance running on Ubuntu 8.04. It's a bit sluggish - I haven't done anything to ...
8
votes
4
answers
2k
views
IIS Optimization
What kind of optimization (performance or whatever) do you use when going live with IIS (on windows 2003) ?
Currently, I use 1 application pool per web site, but I think I can do more :)
8
votes
2
answers
18k
views
Fixing window scaling problems with Linux TCP
I'm trying to improve the throughput of one of my servers overseas and after monitoring a transfer between the server and my home computer with wireshark I'm pretty sure I have a problem with the ...
8
votes
2
answers
1k
views
what is the IOPS behavior when partitions of single disk are used in an LVM?
I have an ubuntu 14.04.1 LTS server which have LVM(backed by hardware RAID5) with logical volume and a volume group named "dbstore-lv" and "dbstore-vg" which have sdb1 sdb2 sdb3 created from same sdb ...
7
votes
1
answer
4k
views
How to select the best Stripe Size when configuring a RAID Array
There are various "references" on the Internet that delve into some discussion about what factors to consider when selecting a particular RAID stripe size (4KB to 128KB or more) for an array of disks, ...
6
votes
3
answers
57k
views
Evaluating the CPU I/O wait on Linux
Doing a top to check the io wait, I get these figures:
Cpu(s): 6.7%us, 1.4%sy, 1.2%ni, 85.5%id, 5.0%wa, 0.0%hi, 0.3%si, 0.0%st
Looking at these figures (%us ~= %wa), do they mean that:
there ...
6
votes
2
answers
4k
views
Does it makes sense to run "optimize table" when the mysql database is stored on SSD?
The manual says about "optimize table":
"Deleted rows are maintained in a
linked list and subsequent INSERT
operations reuse old row positions.
You can use OPTIMIZE TABLE to reclaim
the ...
6
votes
1
answer
29k
views
Nginx "ssl_stapling" ignored, no OCSP responder URL in the certificate
I'm trying to setup OCSP stapling on Nginx because I ran a test and it suggested the idea and well, you know.
https://sslcheck.globalsign.com/en_GB/sslcheck?host=aj2jewellers.co.uk#176.58.103.165
I'...
6
votes
3
answers
3k
views
Optimizing VPS Server Memory
WHAT I'M TRYING TO DO
The server resource limits sometimes run tight; to help prevent memory exhaustion, I've had to limit server processes. I'm needing a little expert help to know if I'm on the ...
6
votes
5
answers
11k
views
CPU's on Hyper-V host system is just idling, even though VM's are at full throttle
I have a server that is running Windows 2008 64 bit Hyper-V, with 8 gigs of RAM and Intel Xeon X3440 @ 2.53 Ghz, which gives me 8 logical cores in the performance monitor on the host system.
I have ...
6
votes
2
answers
539
views
Do you remove unwanted packages (*NIX), or do you disable them?
Im currently building a cobbler and puppet setup. One thing I do is disabling unwanted services with puppet, but then I thought: Should I remove them with cobbler, rather than disabling them after ...
6
votes
3
answers
4k
views
Guest optimization tips for Ubuntu on VirtualBox [closed]
I was wondering if anyone could shed light into "Guest optimization tips for Ubuntu (guest)"
I came across Virtual Machine guest optimization tips, for Windows.
(serverfault.com/questions/4647/ubuntu-...
6
votes
1
answer
476
views
Red Hat Linux tuning for a single application with a few TCP clients [closed]
I know Linux is built for multi-user / concurrent-process, but at the corporate level, many of them are used for single (but big) applications to support an internal service, of which serving only ...
6
votes
3
answers
5k
views
Can I optimize these APC settings more?
I would like to optimize APC some more but I am not sure where I could do something. First here is the screenshot after one week of running with the current configuration:
I have now the following ...
6
votes
3
answers
2k
views
Very slow write speed (NVME drive) on 10G network
Setting up an experimental lab cluster, the write speed for data received via 10G fiber connection is 10% of the local write speeds.
Testing transfer speed between two identical machines; iperf3 shows ...
6
votes
2
answers
2k
views
Cron job for database repair and optimization
I would like to repair + optimize my MySQL database once a week!
How could I do this?
6
votes
1
answer
4k
views
Nginx performance when running multiple instances on the same machine
I'm trying to understand the performance impact of having multiple nginx instances (masters) running on the same machine, rather than having them all load into a single instance using different server ...
6
votes
2
answers
4k
views
How can I determine the memory used by a forked process (Linux)?
Following on from this question. Debian, if that matters.
I understand that some memory is shared between forked processes. How then can I determine how much memory is used by a process/set of ...
5
votes
6
answers
3k
views
Method for finding unnecessary indexes of an operational SQL Server database?
Besides wasting space on the database, unnecessary indexes on SQL Server can slow down insert and update operations. Developers who lack experience in database principles sometimes tend to create ...
5
votes
6
answers
684
views
Capacity Optimization / Deduplication Options for Primary Storage
I'm exploring options for making more efficient use of our primary storage.
Our current NAS is an HP ProLiant DL380 G5 with an HP Storageworks MSA20, and one other disk shelf which I'm not sure what ...
5
votes
2
answers
2k
views
How can I handle big traffic spikes on my dedicated server?
I have an E-Commerce site, which gains a sudden peak of very high traffic when I turn on different ads campaigns.
From this moment on, my current dedicated server (Hostgator, 4GB Memoray, 100mbps ...
5
votes
2
answers
8k
views
Disk I/O utilization up to 100% after tuning InnoDB-related MySQL configuration
I have added following lines to "my.conf":
query_cache_size=128M
innodb_buffer_pool_size=512M
innodb_flush_method=O_DIRECT
After that Disk utilization started raising, reached 100% and holds steady, ...
5
votes
2
answers
2k
views
How do I optimize TCP stack for HTTP server?
I have a HTTP server that serves only two kinds of page: about 10 KB and about 16 KB (both compressed, other files are from CDN). As the latency is quite high (ping takes more than 300 ms), I want to ...
5
votes
2
answers
5k
views
Optimizing MySQL for small VPS
I'm trying to optimize my MySQL config for a verrry small VPS.
The VPS is also running NGINX/PHP-FPM and Magento; all with a limit of 250MB of RAM.
This is an output of MySQL Tuner...
-------- ...
5
votes
0
answers
2k
views
`nginx reload` and `nginx -t` very slow - how to investigate
I have a quite small VPS on AWS (EC2 t2.nano) that is running an nginx server. while sometimes a sudo service nginx reload or nginx -t is blazing fast, most of the time it is horribly slow (>60 ...
5
votes
2
answers
6k
views
Optimize php-fpm and varnish for a powerful server
My setup is Intel® Core™ i7-2600 and RAM 16 GB DDR3 RAM
varnish+nginx+php-fpm+apc for a not very heavy WordPress blog with W3 Total Cache and CDN
My problem is that after 55 hits per second ...
4
votes
5
answers
32k
views
How to Best Manage High PostgreSQL CPU Use?
I am trying to fix a high PostgreSQL CPU use problem. We are using PostgreSQL 8.0.9 and when our JEE web application (in JBoss) is used in certain load increase conditions, top shows a slow increase ...
4
votes
2
answers
5k
views
Do access logs (from nginx) slow down a website?
Does it slow down a website if your webserver (in my case, nginx) is writing to an access log?
I'm mostly wondering if it's noticeable.
Plus, what's the point of knowing the user agents / ips that ...
4
votes
8
answers
6k
views
Optimize a fresh Ubuntu server installation
After installing a fresh Ubuntu server (I personally use 8.04 LTS) what do you do to optimize it in terms of memory consumption, disk usage and speed? What services and pre-installed packages do you ...