I'm running Apache 2.2.17 with the peruser MPM on FreeBSD 8.2-RC1 on Amazon's EC2 (so it's XEN). It was installed from ports.
My problem is that, although Apache is running, listening for, and accepting connections, it doesn't actually respond to any or show them in the log at all.
If I telnet to the port it's listening on and type out an HTTP request:
GET / HTTP/1.1 Host: asdfasdf
And hit enter a couple of times, it just sits there... Nothing. No response requesting with a browser either. There doesn't appear to be anything helpful in the error log:
[Sun Jan 09 16:56:24 2011] [warn] Init: Session Cache is not configured [hint: SSLSessionCache] [Sun Jan 09 16:56:25 2011] [notice] Digest: generating secret for digest authentication ... [Sun Jan 09 16:56:25 2011] [notice] Digest: done [Sun Jan 09 16:56:25 2011] [notice] Apache/2.2.17 (FreeBSD) mod_ssl/2.2.17
The access log stays empty:
root:/var/log# wc httpd-access.log 0 0 0 httpd-access.log root:/var/log#
I've tried with accf_http and accf_data both enabled and disabled, and with both the stock configuration and my customized config. I also tried uninstalling apache22-peruser-mpm and just installing straight apache22... Still no luck. I tried removing all of the LoadModule lines from httpd.conf and just re-enabled the ones that were necessary to parse the config. Ended up with only the following loaded:
root:/usr/local/etc/apache22# /usr/local/sbin/apachectl -M Loaded Modules: core_module (static) mpm_peruser_module (static) http_module (static) so_module (static) authz_host_module (shared) log_config_module (shared) alias_module (shared) Syntax OK root:/usr/local/etc/apache22#
Apache is definitely what's listening on port 80:
root:/usr/local/etc/apache22# sockstat -4 | grep httpd root httpd 43789 3 tcp4 6 *:80 *:* root httpd 43789 4 tcp4 *:* *:* root:/usr/local/etc/apache22#
And I know it's not a firewall issue as there is nothing running locally, and connecting from the local box to 127.0.0.1:80 results in the same issue.
Does anyone have any idea what's going on? Why it would be doing this? I've exhausted all of my debugging expertise. :/
Thanks for any suggestions!
EDIT: As per Phil Hollenback's suggestion, I ran a trace on the system calls. This is what I found through both ktrace and truss.
48443 httpd CALL select(0,0,0,0,0xbf7feab4) 48443 httpd RET select 0 48443 httpd CALL gettimeofday(0xbf7feae4,0) 48443 httpd RET gettimeofday 0 48443 httpd CALL wait4(0xffffffff,0xbf7fea98,WNOHANG|WUNTRACED,0) 48443 httpd RET wait4 -1 errno 10 No child processes 48443 httpd CALL select(0,0,0,0,0xbf7feab4) 48443 httpd RET select 0 48443 httpd CALL gettimeofday(0xbf7feae4,0) 48443 httpd RET gettimeofday 0 48443 httpd CALL wait4(0xffffffff,0xbf7fea98,WNOHANG|WUNTRACED,0) 48443 httpd RET wait4 -1 errno 10 No child processes 48443 httpd CALL select(0,0,0,0,0xbf7feab4)
Same thing repeated over and over. (Those are the last few lines.) What appears to be happening is it's running through to the select, hangs a moment, returns 0, then tries again. I don't know if that's normal behaviour, but it seems okay to me as it sounds like it's just polling the socket?
What I am kind of concerned about is the wait returning an error that the process doesn't exist. The output of truss (which I started the process with, getting the initial setup) shows a call to fork() that seems to return fine (returns a PID, no error).
In case it helps, the output of truss: http://nucleardog.com/apache-truss.txt (Updated for
I did actually connect during that run - select never seems to return a new connection? As well, I turned LogLevel up to Debug in the Apache config and tried running again - it never logs any connection.
I think this is perhaps more of a FreeBSD issue than Apache, then, as it seems Apache never receives the request. :/
Any (more) ideas?
EDIT 2: This appears to be an issue with something in FreeBSD, given that Apache never seems to receive the connection request. I made a post at the FreeBSD forums (forums.freebsd.org/showthread.php?p=118612) to see if I can find any help there. Thanks folks for your time!