Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am debugging a problem where MoinMoin on CentOS is throwing a permissions error, but I can't track down where the problematic file / directory is.

I ran strace -vp <pid> on the apache pid; when I have the problem I see this:

epoll_wait(10, {{EPOLLIN, {u32=3487534344, u64=140367313734920}}}, 2, 10000) = 1
accept4(6, {sa_family=AF_INET6, sin6_port=htons(52621), inet_pton
    (AF_INET6, "::ffff:", &sin6_addr), sin6_flowinfo=0, 
    sin6_scope_id=0}, [28], SOCK_CLOEXEC) = 11
## Later on...
read(7, 0x7fffa658ad7f, 1)              = -1 EAGAIN (Resource temporarily 

However, since apache is already running, I see no corresponding open() on the file referred to as 7; thus I see the permissions problem, but I still don't know which file is the problem.

I know I could try to catch all the file opens when I respawn apache, but I'm hoping there is a way to map file 7 to a real filename... is there a way to do this?


Using @lain's guidance, I ran lsof | grep 266474069, but the results are unclear...

[root@lnxlmf moin]# ls -la /proc/9707/fd/7
lr-x------. 1 root root 64 Aug 28 15:39 /proc/9707/fd/7 -> pipe:[266474069]
[root@lnxlmf moin]#

[root@lnxlmf moin]# lsof | grep 266474069
httpd      9703      root    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9703      root    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9705    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9705    apache    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9706    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9706    apache    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9707    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9707    apache    8w     FIFO                0,8          0t0  266474069 pipe
httpd      9733    apache    7r     FIFO                0,8          0t0  266474069 pipe
httpd      9733    apache    8w     FIFO                0,8          0t0  266474069 pipe
[root@lnxlmf moin]#

I see that this is a FIFO pipe, but what does this mean for my system configuration? How can I trace the root cause for the EAGAIN issue?


Thanks to @Alan Curry, running strace -fp <pid_of_wsgi_proc> seems to get me a bit farther...

[pid  9731] stat("/opt/moin/share/moin/", {st_mode=S_IFREG|0770, st_size=6463, ...}) = 0
[pid  9731] stat("/opt/moin/share/moin/data/pages", {st_mode=S_IFDIR|0770, st_size=4096, ...}) = 0
[pid  9731] access("/opt/moin/share/moin/data/pages", R_OK|W_OK|X_OK) = -1 EACCES (Permission denied)

However, both apache and wsgi run as the apache user...

[root@lnxlmf moin]# ps auxw | grep -E "apache|wsgi"
apache   10187  0.0  0.1 373488  5884 ?        Sl   17:18   0:00 moin_http_wsgi
apache   10188  0.0  0.1 373488  5884 ?        Sl   17:18   0:00 moin_https_wsgi
apache   10189  0.0  0.1 185096  5824 ?        S    17:18   0:00 /usr/sbin/httpd
root     10243  0.0  0.0 103240   848 pts/1    S+   17:21   0:00 grep -E apache|wsgi
[root@lnxlmf moin]#

Yet, when I run the following commands and restart apache, I still can't fix the problem...

[root@lnxlmf moin]# pwd
[root@lnxlmf moin]# chown -R apache:apache data/
[root@lnxlmf moin]# sudo chmod -R ug+rwx data/
[root@lnxlmf moin]# sudo chmod -R o-rwx data/

I am using this in my wiki http config:

<VirtualHost *:443>
  DocumentRoot /opt/moin/share/moin
  WSGIScriptAlias / /opt/moin/share/moin/server/moin.wsgi
  WSGIDaemonProcess moin_https display-name=moin_https_wsgi \
      user=apache group=apache \
      processes=1 threads=10 maximum-requests=1000 umask=0007
  WSGIProcessGroup moin_https
  WSGIApplicationGroup %{GLOBAL}

  # Generate with...
  # openssl req -new -x509 -days 365 -nodes -out netwiki.pem -keyout netwiki.key
  SSLEngine on
  SSLCertificateFile /etc/httpd/ssl/netwiki.pem
  SSLCertificateKeyFile /etc/httpd/ssl/netwiki.key
share|improve this question
It should be noted that EAGAIN is not a permissions issue. It's just a "buffer is empty right now" indicator. It appears quite often in the strace output of programs that are working perfectly. How sure are you that this particular EAGAIN is connected to your problem? – Alan Curry Aug 28 '12 at 22:08
Well, you're probably right... bad call on my part. I will run strace again and see if I can nail this down. – Mike Pennington Aug 28 '12 at 22:12
Since you're looking at an access() failure, I wonder if the real uid doesn't match the effective uid. Look at the Uid: line in the process's /proc/PID/status and if it has different numbers in the different fields, that'll be a clue. – Alan Curry Aug 28 '12 at 22:59
Well, everything is showing up as euid of 48... and grep 48 /etc/passwd yields apache:x:48:48:Apache:/var/www:/sbin/nologin... so I think I'm ok there... the apache user and group have rwx on the whole directory tree – Mike Pennington Aug 28 '12 at 23:08

Take a look in /proc/<PID>/fd which should list all of the open files that PID has open.

On my CentOS system fd 7 is

lrwx------. 1 root root 64 Aug 28 22:01 7 -> socket:[1872522]

I can then use netstat -ane | grep 1872522 to get

tcp    0  0 :::443              :::*               LISTEN      0          1872522

You can use

lsof | grep 266474069

to get information on the pipe.

share|improve this answer
Symlinked to a pipe??? /proc/9707/fd/7 -> pipe:[266474069] – Mike Pennington Aug 28 '12 at 20:40
Uh, lain was faster, sorry. Yes, a file descriptor can also point to a pipe. – Valentin Aug 28 '12 at 20:41
@MikePennington: on my system it's a socket lrwx------. 1 root root 64 Aug 28 21:34 7 -> socket:[1869276] – Iain Aug 28 '12 at 20:42
+1, good info... please see my update – Mike Pennington Aug 28 '12 at 22:05

Looking at my little VPS, I can determine the fd number the following way:

 ll /proc/17684/fd/ |colrm 1 46

0 -> /dev/null
1 -> /dev/null
10 -> /var/www/vhosts/
11 -> /var/www/vhosts/
12 -> /var/www/vhosts/ 
13 -> /var/www/vhosts/
14 -> /var/log/apache2/access.log

[and so on, where 17684 is the PID of the process I straced earlier]

share|improve this answer
+1, please see my update – Mike Pennington Aug 28 '12 at 22:06
up vote 0 down vote accepted

The problem was that I had SELINUX=enforcing in /etc/selinux/config.

After I set SELINUX=permissive, SELINUXTYPE=targeted, and rebooted wsgi can access all files correctly.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.