We are experiencing a high level of connections entering the TIME_WAIT state on our proxy server during a time window that produces bursty traffic - tens of thousands of connections every minute for about 15 minutes.
The proxy server sits behind a firewall that is providing a NAT. Upstream from the firewall is the destination service, which has tcp timestamps enabled, as well as tw_reuse and tw_recycle tcp settings.
The destination service is likely seeing a single IP due to the NAT'd firewall. Our firewall is dropping connections that it is receiving out of state and also, our proxy is producing a lot of 503 errors. The proxy has about 16,000 available ports.
Can anyone help provide a reasoning for this behavior? Based on the info given above, are there settings that we need to enable on the proxy to suppress this behavior?