About my setup:
- I've got an HAProxy configuration for my WordPress environments.
- I've got multiple server pools for the "front-end" of each WordPress website.
- I force everyone to connect to one specific server for anything in the "back-end" of each WordPress website. For now... This just makes things simple for file syncing and database replication.
My clients are uploading massive amounts of photos to their websites in one big bulk upload. Most of the time, the files are uploading as expected. However, randomly and sporadically some of the photos are failing to upload with a fairly generic error code in WordPress of "HTTP Error".
The photos which fail to upload are never the same photo file. It's completely random in each case.
I've been trouble shooting this problem for about a week or so now, and can't seem to hunt down the problem.
- I've verified the DNS is correct and functioning as intended. The user is resolving to the HAProxy server, and the connection is forwarded to the appropriate server hosting the "back-end" of the WordPress website.
- I've thoroughly checked Apache, PHP, and MySQL configurations on the server hosting the "back-end" of each WordPress website.
- I've checked out everything related to WordPress from head to toe.
The only thing I can think of at this point is that there's a problem with my HAProxy configuration. I feel like there's too many requests being made too quickly, and the HAProxy Server isn't keeping up with the communication to and from the server hosting the "back-end" of the WordPress website (where the bulk uploads are taking place).
I'd like to solve this problem related to HAProxy. If you feel like my configuration could be simplified or improved upon elsewhere (unrelated to this particular problem), I'm open for improvements overall with my current HAProxy configuration, but the main focus is on this "HTTP Error" problem related to bulk uploading of files.
Here's my HAProxy 1.6.9 configuration:
defaults log global mode http option httplog option dontlognull errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http retries 3 option redispatch maxconn 2000 timeout connect 5000 timeout check 5000 timeout client 300000 timeout server 300000 frontend http-in bind *:80 option httplog option http-server-close acl has_domain hdr(host) -m found acl has_www hdr_beg(host) -i www. acl has_admin path_beg /wp-admin acl has_login path_beg /wp-login.php acl has_custom_login path_beg /manage acl has_server1 hdr_beg(host) -i server1. use_backend admin_servers if has_domain has_www has_admin or has_domain has_www has_login or has_domain has_www has_custom_login use_backend live_servers if has_domain !has_admin !has_login !has_custom_login or has_www !has_admin !has_login !has_custom_login use_backend default_servers if has_server1 default_backend default_servers backend default_servers mode http stats enable stats uri /haproxy?stats balance roundrobin option httpclose option forwardfor cookie SERVERID insert indirect nocache server server1 18.104.22.168:80 check cookie server1 backend admin_servers mode http stats enable stats uri /haproxy?stats balance roundrobin option httpclose option forwardfor cookie SERVERID insert indirect nocache server server1 22.214.171.124:80 check cookie server1 backend live_servers mode http stats enable stats uri /haproxy?stats balance roundrobin option httpclose option forwardfor cookie SERVERID insert indirect nocache server server1 126.96.36.199:80 check cookie server1 weight 200 server server2 188.8.131.52:80 check cookie server2 weight 25 maxconn 256