Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a custom daemon that is managed by upstart on my Ubuntu server. It works perfectly except that I need to capture (log) the daemon's output. The official stanzas page says that I can use console logged to do this, but what file does it log to?

I've also read that console logged is no longer a valid stanza. I'm currently using 0.3.9 (Hardy) but will upgrade to 0.6.x (Lucid) in a few months. If console logged in fact won't work with later versions, what do I use instead?

share|improve this question
1  
Can you simply update your custom daemon to send the output to syslog, or to a logfile specified in the daemon's configuration file? –  Zoredache Feb 18 '10 at 2:22

6 Answers 6

This snippet will pipe the output of your service into logger, while still allowing you to exec the service process (thus replacing the shell process) so that upstart doesn't get confused. It also makes sure the logger process is reparented to init, so it's not a child of your service, and it avoids leaving cruft sitting around in the filesystem, even though it needs to create a fifo temporarily.

script
  mkfifo /tmp/myservice-log-fifo
  ( logger -t myservice </tmp/myservice-log-fifo & )
  exec >/tmp/myservice-log-fifo
  rm /tmp/myservice-log-fifo
  exec myservice 2>/dev/null
end script

Here's how it works:

  1. mkfifo /tmp/myservice-log-fifo simply makes the fifo special file (aka named pipe). Type man 7 fifo for more info.
  2. ( logger ... </tmp/myservice-log-fifo & ) starts logger reading from the fifo, in the background. The parens cause the logger process to be reparented to init, rather than remaining a child of the current shell process.
  3. exec >/tmp/myservice-log-fifo redirects the current shell's stdout to the fifo. Now we have an open file descriptor for that fifo, and we don't actually need the filesystem entry any more...
  4. rm /tmp/myservice-log-fifo so we'll remove it.
  5. exec myservice 2>/dev/null simply runs the service in the usual way. Stdout is already going to the fifo, and that won't change when the new program executes.

UPDATE: set -e is not needed as Upstart runs scripts with this option by default (see http://upstart.ubuntu.com/cookbook/#develop-scripts-using-bin-sh)

share|improve this answer
    
great answer, and doesn't leave fifo files lying around anywhere. –  Ash Berlin Mar 29 '12 at 12:25
    
What is the set -e? –  Peter Mounce Sep 24 '12 at 22:23
1  
The set -e causes the script to exit immediately if any command fails. Without that line, the script would continue running subsequent commands uselessly (and possibly dangerously). –  Keith Rarick Oct 13 '12 at 23:53
1  
To log stderr as well as stdout, add the line exec 2>&1 above the rm line, and remove the 2>/dev/null from the last line. –  itsadok May 19 at 11:20

If you use the console output stanza, and then pipe your script's output to logger (the shell command interface to the syslog(3) system log module) then that will work.

For example

console output
exec /my/script | logger

will log to /var/log/messages

For example

console output
exec /my/script | logger -t my-script

will log to /var/log/messages and tag each message with my-script

logger --help for logger usage options.

(I'm on the Amazon Linux AMI, which is Centos 5.x based; YMMV)

share|improve this answer
    
It turns out this isn't a good solution. upstart latches onto the PID of logger, not of the process you actually want it to manage. –  Peter Mounce Jun 16 at 22:39

For recent Ubuntu versions (12.04+), simply use

console log

And the daemon output (STDOUT & STDERR) will be appended to /var/log/upstart/<service>.log

http://upstart.ubuntu.com/cookbook/#console-log

share|improve this answer

This is ugly but so far the best I have found

exec /path/to/server >> /tmp/upstart.log 2>&1

share|improve this answer
    
The solution is not a good when it comes to log rotation, because the application is logging directly to the file. Logging through syslog avoids related problems. –  Mark Stosberg Jun 29 at 2:05

I did not get the mkfifo trick to work satisfactorily; it did not seem to capture stderr, and attempts to redirect caused Upstart to bail with no errors.

It also has an unfortunate side effect of making the logger process hang around as a child of init, so the information about who "owns" the logger is lost, and anyone not already aware of the mkfifo might assume it's a dangling process that can be killed.

Instead I ended up with the following solution, which resolves all of these issues. It causes logger to become a child process, while preserving the service as the root process. Unfortunately, it requires exec'ing bash, but it just looks dirty.

script
  # ... setup commands here, e.g. environment, cd, ...
  exec bash <<EOT
    exec 1> >(logger -t myservice) 2>&1
    exec myservice
EOT
end

This uses a trick that redirects stdout and stderr to a command. Since we exec the service inside the bash command, this has the side effect of replacing the shell and magically making bash become a child process of the service, as shown by ps aufxw:

myservice 
 \_ bash -c exec 1> >(logger -t myservice) 2>&1 && exec myservice
    \_ logger -t myservice

For some reason the above command has to be wrapped in a bash -c. I assume this is because Upstart only pretends to run your script via Bash, but isn't actually. If anyone can suggest a way to avoid the extra bash shell, that would be awesome.

share|improve this answer
    
Yes! I'm so glad I found this. Looks better than the mkfifo variant and in my case I needed to exec bash -l << EOF so no loss there. –  thom_nic Mar 10 at 14:19

You can also redirect the output to syslog, e.g.

exec $SERVER 2>&1 | logger -t myservice -p local0.info

However, the pipeline may cause upstart to confuse the PID of the logging process with the PID of the daemon.

share|improve this answer
    
The daemon confusing the PID is actually a useless daemon, since it is supposed to watch the process, respawn it then. Any idea on how to make sure the right PID is being watched? –  Johann Philipp Strathausen Feb 10 '12 at 11:04
    
I think (but haven't tried) that you'll want the expect fork or expect daemon stanza. Or you could cat the pid-file into the log message otherwise, I guess. –  Peter Mounce Jun 20 '12 at 12:45

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.