2

I have a process - a perl script - that does:

while true
    check a POP account on a server on the lan
    process any email found
    write logs - messages found, actions taken, errors
    sleep for 15 seconds

It's running on a redhat 7.3 server (I inherited it, I'm not happy about the age of that box). It's run out of /etc/inittab like:

spop:2345:respawn:/usr/local/gw/bin/popdmn 

If it dies, init restarts it.

In the last couple of days, the process will no longer work unless it's straced. When it's just running, it never logs into the pop server. As soon as it's straced (via "strace -Ff -p cat /usr/local/gw/var/popdmn.pid"), it works flawlessly.

As a workaround, I'm running screen on the server with an strace running. Obviously this is less than ideal.

Why would a process do this? I haven't seen this happen before.

5
  • What happens if you stop the strace command and send a CONT signal to the process?
    – kasperd
    May 9, 2014 at 8:31
  • When I do "kill -CONT cat popdmn.pid", it unfreezes. I've never seen that before, what's happening under the hood here? Also, thanks :) May 9, 2014 at 8:38
  • Last I checked using strace somehow made use of STOP and CONT signalling. I don't remember the details though. But somehow it was possible to leave the traced processed in stopped state, needing a CONT signal to continue. Now the question is, was the processed stopped with a STOP signal before you attached strace command the first time? Either the original problem was that a STOP signal had been sent, and using strace made it continue temporarily. Or the original problem has gone away, but in the meantime using strace has introduced the new problem.
    – kasperd
    May 9, 2014 at 12:17
  • Has anything been changed in the last few days ?
    – user9517
    May 9, 2014 at 12:48
  • Iain: Yes, the POP server has been replaced (the IP has been migrated to a different box). That new box has the same version of dovecot running on it. May 9, 2014 at 23:01

2 Answers 2

2

I think I've been bitten by an ancient strace bug:

https://bugzilla.redhat.com/show_bug.cgi?id=64303

https://bugzilla.redhat.com/show_bug.cgi?id=75709

This box has strace-4.4-4 on it, so it sounds possible that it's that bug. It sounds like this one is self-inflicted, as we were stracing while trying to debug - and made it worse.

kill -CONT works to resume the process.

Definitely time to upgrade this box.

0

Biggest point of difference is speed and signal handling I suppose.

Regarding speed, if the process is multithreaded, then strace will be altering the timing, which my change behaviour regarding race conditions etc. or timing information relating to protocol behaviour.

Example. Let's say the POP server has been upgraded and is now more careful in ensuring that a peer hasn't sent multiple POP commands at a time. This is more useful in a SMTP server as a means of spam prevention.

Does your process observe correct POP behaviour, in that it waits for a response from the server after each and every POP command? Or does it assume success or wait some period of time between commands.

If you capture the actual protocol traffic in a passing and failing case, is there any sign of a protocol violation?

1
  • POP server happens to have been upgraded, but it's the same dovecot version, so that hasn't changed fundamentally. The perl script is single threaded only. As far as I can see from watching the strace, it's a dead simple POP transaction - user, pass, list, retr, quit. It doesn't assume success as far as I can see; it looks to be waiting for server responses (+OK etc) after each command (it's a definite write, read cycle). May 9, 2014 at 12:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .