NOTE: This question has been resolved! Please see below for the solution in my case if you are having a similar issue!
This is a PHP/MySQL based, database driven application that makes heavy use of AJAX, and nearly all requests to the server occur this way. The requests are GET requests in this instance, and the server is responding via JSON. The user works inwards through the data set returning finer and finer results. The application is stable, and in production.
I have the ability to debug this on 3 platforms: My development laptop (OSX), an in-house Ubuntu server, and the production Ubuntu server.
Each Ubuntu server is running a MySQL server, and a copy of Apache.
My development laptop connects to the MySQL server running on our in-house Ubuntu server, and runs it's own Apache server.
Both Ubuntu servers are 9.10, x64, fully up-to-date via the official sources, Apache 2.2.12, PHP 5.2.10.
And now the fun part:
The application returns results correctly in all cases except one subsection of the data set. Requests for data within this subsection always return a 500 instead of a result set. The subsection in question is merely a query to return results based on known IDs. The records are known to exist, and contain no corrupt data.
I have debug logging turned on for Apache, and E_ALL set for PHP (logging to file). When the 500 occurs, I see the following in Apache's error.log (x's for masking):
[Sun Aug 01 15:43:54 2010] [debug] mod_deflate.c(615): [client 192.168.1.28] Zlib: Compressed 0 to 2 : URL /apps/xxxx/connectors/details_lookup.php, referer: http://xxxx.xxxx.local/apps/xxxx/
This implies a lack of data. However, there are no errors logged to PHP's error log whatsoever. At the same time, I can return results elsewhere in the data set with no issues.
So you might be thinking "The problem must be with the database."
However, when I use my development laptop (and it's own copy of Apache) to connect to the same MySQL server on our in-house network, I can run the request that generates the 500 with no errors at all.
So when I run the application on my Apache server on my laptop, everything's cool. When I perform the same request against the same data within the same database on the same database server but instead using that server's local copy of Apache, fail.
I'm utterly baffled. Any help would be greatly appreciated at this point.