3

When I run ps aux | grep /usr/local/apache/bin/httpd I get the following output.

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
www-data  9837  0.0  0.0  23112  1360 ?        S    Oct15   0:00 /usr/local/apache/bin/httpd -DSSL
www-data  9841  0.0  0.0  23112  1568 ?        S    Oct15   0:16 /usr/local/apache/bin/httpd -DSSL
www-data 29178  0.0  0.0  23112  1064 ?        S    Oct04   1:51 /usr/local/apache/bin/httpd -DSSL

What I find interesting though is that /usr/local/apache/bin/httpd doesn't exist. There is no /usr/local/apache/bin/ directory at all.

sudo cat /proc/9837/cmdline returns /usr/local/apache/bin/httpd -DSSL

When I run /proc/9837$ sudo /usr/local/apache/bin/httpd -DSSL I get:

/usr/local/apache/bin/httpd: command not found

When I run sudo ls -l /proc/9837/exe I get:

lrwxrwxrwx 1 www-data www-data 0 2012-10-17 02:06 /proc/9837/exe -> /usr/bin/perl

My question is, why is this happening? Shouldn't /proc/<pid>/cmdline and /proc/<pid>/exe be related? If this is not normal, what steps can I take to find out what caused it?

These processes remain even after running sudo /etc/init.d/apache2 stop

For reference, I'm running Ubuntu Ubuntu 10.04.4 LTS running Apache2 from the default apt repository.

sudo ls -l /proc/9837/fd outputs

lr-x------ 1 www-data www-data 64 2012-10-17 02:47 0 -> /dev/null
l-wx------ 1 www-data www-data 64 2012-10-17 02:47 1 -> pipe:[37796710]
l-wx------ 1 www-data www-data 64 2012-10-17 02:47 2 -> /var/log/apache2/error.log
lrwx------ 1 www-data www-data 64 2012-10-17 02:47 3 -> socket:[37796725]
lr-x------ 1 www-data www-data 64 2012-10-17 02:47 4 -> pipe:[40055427]

stat /proc/9837/root returns

  File: `/proc/9837/root' -> `/'
  Size: 0           Blocks: 0          IO Block: 1024   symbolic link
Device: 3h/3d   Inode: 49853155    Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2012-10-17 02:07:00.240782014 -0400
Modify: 2012-10-17 02:06:43.860777313 -0400
Change: 2012-10-17 02:06:43.860777313 -0400
Improve this question
2
  • Does /var/log/apache2/error.log actually exists? Is it possible that you have a perl program running as a wrapper to a chrooted Apache environment? You could check for a possible chroot() environment by inspecting /proc/[pid]/root. (by the way, this is a good reference)
    – Alexander Janssen
    Oct 17, 2012 at 6:57
  • /proc/$PID/root looks like a symlink to /. /var/log/apache2/error.log is valid as well.
    – Citricguy
    Oct 17, 2012 at 7:10

2 Answers 2

Reset to default
2

Smells fishy. Programs can override their $0. The program pretends to be Apache, but it's perl in fact.

Go to /proc/$PID/fd and have a look what files it's got opened, maybe this gives you a clue what's going on.

Improve this answer
2
  • I added the output to the end of the question. Is it possible to retrieve the original command elsewhere if /proc/$PID/cmdline has been overwritten?
    – Citricguy
    Oct 17, 2012 at 6:58
  • @Citricguy In fact, I'm already looking for that. I thought I've seen that before, but somehow I can't find it right know. Will keep looking.
    – Alexander Janssen
    Oct 17, 2012 at 7:00
0

you said that you had installed apache2 from default repository. can you run a consistency check based on that package to check what is missing ?

It seems that someone / something / somehow deleted some of the apache files. But apache had been started before that, so it still remains in memory

Improve this answer
3
  • The original/system apache2 is matches whats in Ubuntu's repositories and can restart without issue. This apache2 process loads from /usr/sbin/apache2 as it should.
    – Citricguy
    Oct 17, 2012 at 8:50
  • up to which depth you have the directories ? /usr/local/apache ? /usr/local/apache/bin/ ?
    – Nikolaidis Fotis
    Oct 17, 2012 at 9:04
  • Only as high as /usr/local/, but I don't think this is where Apache is located in Ubuntu 10.04 (Lucid). I believe the binaries live in /usr/sbin but I'll verify as this is a good point.
    – Citricguy
    Oct 17, 2012 at 9:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.