0

I have an IIS website that serves content via Apache (ProxyPass and ProxyPassReverse - the reason for this weird configuration is historical). The Apache is behind other company proxy servers. The site has static files and one virtual directory for the WebAPI part that serves JSON.

Users are sometimes getting wrong content - for example, visiting the home page returns an image linked on the home page, or some other html page on the site, or some JSON (from the WebAPI). The home page has caching turned off.

This usually happens when the server is under stress (database freezes up), but happens otherwise as well (but rarely) but not with enough frequency or consistency that we can debug.

IIS: version 6.1

Apache: version 2.2.13

What could be causing this intermittent issue, or is there a good way to figure out what's going on?

  • 2
    That's a very old apache. In times gone by with apache there have been a variety of concurrency corner cases, in multi-threaded configuration and with persistent upstream connections, the results of which were that when user 1 and then user 2 issued requests, under certain circumstances user 2 would get some resource that had previously been delivered to user 1. If you are unable to update the apache, try ensuring it is running in prefork (should be an option on win), disable keep alives, and review the proxy configuration to not use connection pooling or any other form of resource sharing. – Jonah Benton Jul 9 '18 at 13:48
  • Are you sure that Apache is at fault and not IIS? Or that is also in doubt? If you could add your apache configuration it might help to debug the behavior. – Leo Jul 14 '18 at 3:22
0

It could be that the "error" page is redirecting to home page. Or just displaying the home page. This also explains why this problem show when you have high load on the server. I would check IIS config for this.

Here is how I would debug the problem in a generic way.

I would try to isolate the problem. For this, I would record the traffic on the following paths:

  • web client - apache
  • apache - IIS

Then I would check what happened. And try to match the request and responses. You can record the traffic with any network sniffer and analyze it with Wireshark. You can also record the traffic from a managed switch using SPAN or RSPAN.

Then I would try to reproduce the problem. You can clone the machines and test the clone with a load test software like JMeter. Check the error log of the web servers. Try to figure out which limits are you reaching. Could be connection pool limit, timeouts...

  • Thanks for taking the time to answer!. Pretty sure its not an error redirection. The (wrong) files / content being served up are from all over - indicative responses getting mixed up and not a consistent error redirection. While there are errors in the application logs (and related ones in the IIS and Apache logs), these indicate the root cause (eg. the database freezing up) but don't explain why the users are getting different responses. We eventually fix the root cause, but we don't want these mixed up responses in the meantime. – potatopeelings Jul 12 '18 at 1:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.