I was having the common issue of not being able to restart httpd because of a high number of locked semaphores. After clearing those and restarting httpd, I also doubled the semaphores.

However, I'm lead to believe some Apache process has a memory leak and it's not releasing them, so I'll still run into the issue but later.

Example ipcs -s:

------ Semaphore Arrays --------
key        semid      owner      perms      nsems     
0x00000000 13467648   apache     600        1         
0x00000000 13697025   apache     600        1         
0x00000000 13729794   apache     600        1         
0x00000000 13762563   apache     600        1         
0x00000000 13795332   apache     600        1         
0x00000000 14057477   apache     600        1         
0x00000000 14123014   apache     600        1         
0x00000000 14155783   apache     600        1         
0x00000000 14188552   apache     600        1         
0x00000000 14221321   apache     600        1         
0x00000000 14254090   apache     600        1         

So let's trace down what process was managing those ipcs -s -i 13697025:

Semaphore Array semid=13697025
uid=48   gid=48  cuid=0  cgid=0
mode=0600, access_perms=0600
nsems = 1
otime = Not set                   
ctime = Thu Jul 11 03:41:01 2019  
semnum     value      ncount     zcount     pid       
0          1          0          0          18395

And finally what corresponds to the pid ps --pid 18395:

  PID TTY          TIME CMD

So am I reading that right the second semaphore in the list belongs to a process that already died and didn't clean up?

For example, running through the process with the second to last in the list produced that it actually did belong to a running process:

  PID TTY          TIME CMD
22331 ?        00:03:42 httpd

It's obvious these are owned by Apache, but what would be the best way of debugging what's causing this. We have in the range of 50+ virtual hosts is there a way to log an actual server request and trace it back to the process spawned, etc?

I have reason to believe it's the php-sqlsrv.x86_64 extension or something related to it because of the time relation between when that was configured and when the issue started happening. I have the ability to roll back but just want to learn how to go deep into debugging, perhaps even to submit a patch for the bug.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.