Questions tagged [optimization]
Optimization is the process of modifying a system to make some aspect of it work more efficiently or use fewer resources.
339
questions
0
votes
0
answers
32
views
Optimzing Docker Builds with docker-compose.yml when using PostGres DB's, performance, etc
I just have a general about how best to build and manage DB servers when using Docker builds with docker-compose.yml. Historically, I've sometimes bound the Postgres DB's to a bound folder on the ...
- 141
1
vote
1
answer
32
views
What is the fastest method of getting a small amount of data from whitelisted servers?
I have a bunch of servers which build various different programs for various different systems.
Once a build has completed, it gets archived into a single file and compressed, then an md5sum is ...
- 944
1
vote
1
answer
228
views
How to reduce MySQL Server CPU usage or load?
How to reduce MySQL Server CPU usage or load ?
Here is my.cnf :
skip-name-resolve
disable_log_bin
innodb_buffer_pool_size=200GB
innodb_log_file_size=25GB
innodb_log_buffer_size=8M
...
- 11
0
votes
1
answer
276
views
optimize Mysql 8.0 performance for millions of small tables
my data structure is around 0.2 million Innodb tables. each table is not longer than 300 rows. And the number grows by ~3000 tables per day. I'm afraid that this may bottleneck my I/O operations ...
0
votes
0
answers
1k
views
What is the ideal value for threads on thread_pool in nginx config?
I have a machine with 2x CPU 24 Cores, 48 Threads
Total: 48 Cores, 96 Threads.
My nginx default configuration came configured with threads=32
thread_pool pool_name threads=32 max_queue=0;
Should i ...
- 1
2
votes
1
answer
279
views
NGINX Redirect to Cached File in Subfolder
I have a program that creates optimized versions of pictures that are uploaded to the /images/ folder on the web server. It traverses subfolders and in each one, creates a .optimized folder that holds ...
6
votes
3
answers
2k
views
Very slow write speed (NVME drive) on 10G network
Setting up an experimental lab cluster, the write speed for data received via 10G fiber connection is 10% of the local write speeds.
Testing transfer speed between two identical machines; iperf3 shows ...
- 191
0
votes
1
answer
162
views
Tightvnc Server using only 1 core?
I have a server setup with 8 cores and 32gb of RAM. I am using it to launch as many java applications as I can and hope to make the most efficient use of my resources. Upon launching I notice the ...
- 131
0
votes
1
answer
110
views
product import is broke behind cloudflare
Im having trouble to resolve my database errors and how to optimize it. Coz of the slowness of my server (mostly querries i believe), my imports turns with a cloudflare error code. (Cloudflare blocks ...
- 3
1
vote
0
answers
113
views
Wordpress UPDATE queries on MySQL database stuck
I have an Amazon 24XL server
96 Cores
378 GB RAM
Database size 5.7G
Debian GNU/Linux 9 (stretch)
PHP 7.3.16
mysql Ver 15.1 Distrib 10.3.22-MariaDB
I have only one WordPress site where users read ...
- 21
0
votes
2
answers
2k
views
MariaDB 10.3 per thread memory usage and low CPU utilization
I am a bit unsure why we are seeing these memory numbers while the cpu usage is actually very low.
-------- Storage Engine Statistics -----------------------------------------------------------------
...
0
votes
1
answer
72
views
mysql thread and tomcat thread should be the same for a better perfoming?
my tomcat instance on ubuntu server running with mysql and only the application has access to mysql (only the service-user tomcat). I have small experience level with optimizing with mysql and tomcat. ...
- 67
0
votes
1
answer
860
views
Where is my bottleneck: RAM, CPU or disk?
I have a 32 core, 64 thread, 128GB RAM system with internal SSD.
I'm running 64 data mining/simulation jobs in parallel that each load 1GB from disk and run for about 5-10 minutes .
I want to know ...
0
votes
3
answers
520
views
Fastest possible configuration for a local MySQL database, no security needed?
We have a local Optiplex PC setup to run our shared MySQL MariaDB database with no way to access it from the outside world. It doesn't have any critical data that needs to be protected.
Basically, no ...
- 105
0
votes
1
answer
550
views
Mysql Tuner output : Need help analyzing
-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 75d 21h 28m 49s (441M q [67.390 qps], 1M conn, TX: 5795G, RX: 281G)
[--] Reads / ...
4
votes
2
answers
9k
views
XFS with No Barrier Mount option deprecated
I tried to perform SQL Server database restore with xfs filesystem and barrier=0 in Azure cloud environment. It performs restore of some 15GB data in around 8 minutes.
I heard this option is getting ...
3
votes
1
answer
1k
views
Debugging and fine-tuning the linux process scheduler
When debugging sporadic web application slowdowns, I've found that all the processes have quite high maximum scheduling delay.
# perf sched record -- sleep 10 && perf sched latency --sort max
...
- 338
1
vote
1
answer
2k
views
How to convert swap partition from RAID1 to RAID0 using mdadm
How to convert linux SWAP memory from RAID1 to RAID0?
Start with:2 exact size swap formatted partitions (/dev/sdc5 and /dev/sdd5) on 2 physically separate disks that are used via Software RAID(named ...
- 166
3
votes
4
answers
13k
views
Joins performed without indexes: 5568
I was trying to do mysql optimization. Yesterday the CPU usage was 100%.
I am trying to optimize with mysqltuner but this time I gave a mysqltuner warning.
I have a high-traffic db. 500-600 ...
- 39
8
votes
2
answers
840
views
what is the IOPS behavior when partitions of single disk are used in an LVM?
I have an ubuntu 14.04.1 LTS server which have LVM(backed by hardware RAID5) with logical volume and a volume group named "dbstore-lv" and "dbstore-vg" which have sdb1 sdb2 sdb3 created from same sdb ...
- 616
0
votes
0
answers
2k
views
CloudFront HTTP/2 slower (high TTFB) than HTTP/1.1
My site was running on HTTP/1.1 with CloudFront at front, Webpagetest reports the load time usually at 2 seconds. Yesterday I enabled HTTP/2 (and SSL) on CloudFront, expecting to see an improvement. ...
- 101
0
votes
1
answer
38
views
Which parameters and setting affect network throughput when connecting two machines together without a switch?
I have a situation where I need best possible throughput (TCP/IP sockets & SMB3) between two Windows Server 2016 connect with a 1m patch cable.
I have to use existing hardware.
When you connect ...
- 103
1
vote
2
answers
9k
views
mysqld taking 800% CPU since traffic increased on my site, how to fix
i have centos 7 x64 64 GB RAM and 500 GB of SSD dedicated server.,
And recently cpu usage was really high,
$ uptime
13:50:30 up 169 days, 1:53, 1 user, load average: 37.48, 37.64, 33.60
so i ...
user138870
0
votes
0
answers
263
views
Should I worry about TTFB?
I have read differing opinions on the subject, but Im also wondering how much of this is affected by server location.
I have a VPS in NY, USA and im an Australian developer, VPS' are pretty pricey ...
- 133
8
votes
6
answers
4k
views
At what point is a server considered idle?
Conceptually, what is the definition of an idle server?
What resource metrics would you look at to make an assumption on whether or not a server was idle?
Would you look at:
CPU utilization
Disk ...
- 203
0
votes
0
answers
106
views
Optimizing nginx with SSL for bot traffic without sessions
The service I run, cronitor.io, has a telemetry collection API that sees bursts of traffic around 1000rps. (overall baseline is much lower, call it 100rps)
Most of our requests come from things like ...
- 267
1
vote
1
answer
3k
views
How to speed up connection with a server in China
I have a EC2 server in China using Amazon AWS China.
It's a server with : Apache 2 and PHP The database is a RDS instance running MySQL (also in china)
The problem: Access from out of China is very ...
- 21
3
votes
1
answer
23k
views
Optimizing nginx client_body_buffer_size for web app accepting file uploads
When configuring nginx, what considerations affect client_body_buffer_size?
I've set it to 10m for a website where users upload photos and image-memes (kind of like 9gag). Note that ...
- 2,335
0
votes
1
answer
382
views
How to simplify my nginx reverse proxy config
I have finally gotten around to setting up a Nginx reverse proxy to handle the multiple websites I host at my house since I only have one external IP address and everything is working, but what I want ...
2
votes
2
answers
910
views
winbind authentication through apache slow
I have a linux machine setup to authenticate users using Active Directory with samba/winbind.
Apache is then setup to use that to authenticate the users. This is done using:
<IfModule ...
- 794
2
votes
1
answer
441
views
Speed up backup of just ACLs on large file systems
I'm trying to make quick backups of just the ACLs on large GNU/Linux file systems. Extended permissions are not really necessary.
I run 4 little benchmarks on a small partition just to make an ...
- 338
2
votes
2
answers
124
views
Minimal upgrade from one minor CentOS version to the next
I would like to update our remote systems from CentOS 7.2 to 7.3. Is there an option that would only install the minimum number of packages required? The standard upgrade would involve downloading ...
- 115
1
vote
1
answer
337
views
Should a KVM host be doing things other than hosting VMs? (such as ZFS)?
What are the performance and stability implications of running resource intensive services in a hypervisor vs guest OS?
I want to set up a workstation with both Linux and Windows, hosted by (probably)...
- 313
0
votes
0
answers
29
views
Optimize webserver to handle huge load spikes [duplicate]
Our application is growing fast so now we need to scale the infrastructure to don't slow down the entire website without worrying on how many users we have
The backend system is based on many curl ...
- 11
2
votes
1
answer
2k
views
Tune when sssd goes into offline mode
I have a linux server with sssd using an ldap server for the auth backend. The connection to the ldap server is flakey, so I get authentications which are fine and then suddenly one will take forever ...
- 548
-2
votes
2
answers
124
views
Server bandwidth is going crazy, after reboot all ok
I have a streaming server with 10gbps card on it. I'm using ~2.5gbps.
My server is running perfect for 8-9 hours / day, and after that is going crazy, and everyone is freezing. At exactly the same ...
- 131
4
votes
0
answers
2k
views
`nginx reload` and `nginx -t` very slow - how to investigate
I have a quite small VPS on AWS (EC2 t2.nano) that is running an nginx server. while sometimes a sudo service nginx reload or nginx -t is blazing fast, most of the time it is horribly slow (>60 ...
- 237
4
votes
2
answers
6k
views
LVM cache in writeback mode is equivalent to external ext4 journal?
I am running ext4 filesystems on LVM. The two big speedup options I'm looking at are, LVM cache and the external ext4 journal.
It sounds like if I'm using a single SSD for this, LVM cache in ...
- 143
1
vote
2
answers
2k
views
nginx multiple paths (too many)
i got this working nginx config:
server {
listen 80;
server_name mydomain.com;
root /var/www/mydomain/wordpress;
index index.html index.php;
location /customer1 {
alias /var/www/...
0
votes
1
answer
702
views
NGINX + PHP-FPM multidomain optimization
I am trying to optimize the parameters of PHP-FPM on a VPS with multiple domains.
Each domain is configured with its own user/group, NGINX uses different PHP-FPM sockets. For each domain I have ...
- 21
1
vote
0
answers
167
views
To which value set ulimit
I'm setting up nginx from what I've understood the worker_connections value must not exceed the maximum number of open file descriptors retruned by ulimit -n.
On my fresh 16.04 ubuntu installation ...
- 11
0
votes
1
answer
5k
views
nginx: configuration file test failed [closed]
I using DirectAdmin and Nginx installed,
i have huge clients at the moment and i see "Nginx 503 Error" sometimes,
and i have 3Gb ram with 790 used ram...
And now i want increase Worker Connections, ...
- 1
2
votes
1
answer
479
views
Move htaccess rules to apache configuration files
I'm running a website with Joomla on a dedicated server (Debian), and i've read that moving .htaccess rules to apache2 configuration files may result in a good performance improvement (Apache HTTP ...
- 21
-2
votes
1
answer
260
views
How to choose database and backend server locations?
I could not find a tutorial for beginners for how to locate my servers.
For instance I use Compose.io for my MongoDB and DigitalOcean for my Meteor.js app.
Lets say I want to have fast service in ...
- 99
4
votes
0
answers
1k
views
Identifying bottleneck with nginx VPS load testing
I'm trying to optimize a Digital Ocean droplet (512mb), testing using loader.io
I'm testing on my homepage, which is HTTPS / PHP. I setup FastCGI page cache, which got me from 100 req/sec to 2,000 ...
- 41
0
votes
0
answers
29
views
Way to know what source is reading the most from MySQL Server(RDS)?
I have a mysql server which is registering very large bandwidth usage. I want to know what options I have for:
Knowing what source is using the MySQL server most. (server ip?)
What tables or queries ...
- 273
1
vote
2
answers
824
views
Track what is causing AWS RDS to use insanely high bandwidth
We are having a high traffic (100K hit/day) Drupal news website hosted on AWS. Its behind cloudflare & 2 Load Balanced Varnish servers. For some reason the bandwidth usage is very high for RDS. ...
- 273
-3
votes
1
answer
108
views
My website is superslow but the %CPU is only 25%
I have 2 servers for my website :
One server 256GO RAM 20c/40t for MySQL
One server 256GO RAM 20c/40t for Apache
The MySQL server is always ok (30-50% of CPU is used).
But the Apache ...
1
vote
1
answer
196
views
Ubuntu LAMP server busy at peak hours
I'm running a Ubuntu 14 server with LAMP on virtual cloud hosting (CityCloud).
The server is usually doing very well with response time<0.5s. However, the problems start at peak hours, mainly ...
- 23
0
votes
1
answer
2k
views
nginx time to first byte
I running several project on one machine, below with all the info.
on nginx config if PHP project use fastcgi cache, Node.js project use proxy_pass , and all project use gzip.
I'm new to maintain ...
- 147