The setup details:
I have Apache 2.2.14 that runs under ubuntu 10.04. The document root directory for a virtualhost is set up as a symlink.
The requested script in a glance looks like:
<?php // other useful stuff here echo 'v1.2.42';
So apart from doing useful job the script outputs its version in the footer. The version is hardcoded in the script and is modified by a build script.
When the new version is deployed the document root is switched to a new version using pretty standard approach:
ln -s /path/to/app-1.2.43 app-new mv -Tf app-new app # where `app` is the document root
After that I don't do any
restart for apache.
Today right after another release that's what I've experienced:
- I upgraded
- Opened the application in the browser
- It was
42in the bottom
43appeared randomly. And after about 30 seconds the expected
43started showing consistently.
Another server that has the similar configuration behaves exactly the same. And seems like there is some constant timeout until it starts working fine :-S
More configuration details:
apache is configured to run via mpm-prefork. php's version is 5.3.2 and it's installed as
mod_php. There is NO any opcode cache installed.
Any ideas on where it could cache it? Would
apache2 reload guarantee this won't happen?
PS: I've proven (to myself) that
apache reload doesn't reset that state and the undefined behaviour continues.
I've found that every apache worker expands the symlink to a document root into a real path.
And when you perform a graceful
reload command - only spare workers are restarted, while the busy ones, including the ones who are serving Keep-Alive connections are kept until they are spare from work. I could reproduce it using 1 worker and it perfectly explains my observations.
Now the question is: how to interrupt keepalive connections gracefully?
Interesting article to read on topic: https://blogs.oracle.com/oswald/entry/apache_s_graceful_restart_reprise