Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I'm trying to set up Tomcat to start with upstart. I find the following works:

  description "Tomcat Server"

  start on runlevel [2345]
  stop on runlevel [!2345]
  respawn
  respawn limit 10 5

  setuid tomcat

  env JAVA_HOME=/usr/lib/jvm/default-java
  env CATALINA_HOME=/opt/apache-tomcat-7.0.34

  script
    chdir $CATALINA_HOME
    exec $CATALINA_HOME/bin/catalina.sh run
  end script

But if I remove the chdir, tomcat starts, but with plenty of FileNotFound exceptions when my .wars load. (That is: the .wars themselves do load but they throw exceptions on load.)

Note this behavior is different from what I see when I invoke catalina.sh from the command line. Invoking from the command line, I can run /opt/apache-tomcat-7.0.34/bin/catalina.sh run from any directory (without chdiring) and everything's fine.

So why is the chdir necessary in my upstart script? (How is the upstart environment different from my command-line environment?)


Example of the errors I'm seeing w/ upstart when I don't chdir:

Feb 22, 2013 3:00:11 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive my-war.war
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: my-war.log (Permission denied)
    at java.io.FileOutputStream.open(Native Method)
share|improve this question
1  
...It's probably changing the current working directory to /opt/apache-tomcat-7.0.34, and if you remove that it will quite obviously get "file not found" errors as it's trying to run the script in some other directory. –  bigbadonk420 Feb 17 '13 at 8:49
    
As I tried to ask in my question though: how come I can execute (on the command line) /opt/apache-tomcat-7.0.34/bin/catalina.sh run from anywhere, without chdiring, and things work fine? (Things only break when running from upstart without chdir). –  Bosh Feb 17 '13 at 18:29
    
Are you asking out of pure curiosity? Because the chdir is obviously there to change the current working directory, and while I can't answer why it's required it certainly makes sense. –  bigbadonk420 Feb 18 '13 at 19:40
1  
I would like to understand why upstart breaks without it, given that on the command-line I can run catalina.sh via absolute path from anywhere. This is for my own understanding (since I'm obviously missing something about the environment upstart presents). –  Bosh Feb 18 '13 at 22:37
    
This does not directly answer your question, but you should use ${CATALINA_HOME}/bin/startup.sh (without chdir). That script in turn execs catalina.sh. –  Nils Feb 19 '13 at 16:25

3 Answers 3

up vote 4 down vote accepted
+50

This is a write permission issue. User tomcat is not allowed to create a log file to a location where it tries to create them. The location it tries to create log files in differs depending on how you start tomcat.

This piece from from your log caught my attention:

java.io.FileNotFoundException: my-war.log (Permission denied) at java.io.FileOutputStream.open(Native Method)

It says that the user (tomcat) is not allowed to create the file my-war.log. Here are the different scenarios:

Upstart with chdir

Upstart first chdirs to $CATALINA_HOME. User tomcat is allowed to create files there. So everything works.

Upstart without chdir

Upstart runs as root, so the default directory is /. User tomcat is not allowed to create files there. So you get permission denied errors.

Running tomcat from your home dir

Now you run tomcat as yourself from your home dir. You have write permission to you own directory. So everything works again.

share|improve this answer
    
Bingo! I thought I was able to successfully run this from the command line, but it turns out I could only run it from the command line when starting in places where the tomcat user has write access. Thanks! –  Bosh Feb 24 '13 at 23:14

I don't have access to an upstart version with setuid support, but this is what I would do in your case.

It looks like an environment discrepancy issue. Most likely an environment variable that is not set when running from upstart. Could it be that setuid does not set $USER or $HOME?

Therefore, I would recommend that you compare the environments. A easy, simple way would be to modify your init script like this and restart the job.

  script
    env > /tmp/env-upstart.log
    chdir $CATALINA_HOME
    exec $CATALINA_HOME/bin/catalina.sh run
  end script

Then also run env > /tmp/env-console.log when you normally would start it from the console (if you are using sudo, then do it with sudo env).

Then compare the two /tmp/env-upstart.log and /tmp/env-console.log files (sort them and open them with vimdiff) and it should be easy to find which variable in env-upstart is missing from the other file (or is set to something that you would not expect).

Update 1

And if you are still getting errors, I would check the following:

  • User permissions: effective/real user ids might be differing. Compare the output of id, id -r -u, id -r -g, id -r -G, id -u, id -g, and id -G in both cases.
  • Process limits: It could be that your upstart scripts have stricter limits. Try putting an ulimit -a to compare.
  • Shell oddness: Not likely, but upstart uses sh while your user's shell is most likely bash or zsh. Try running the successful command from sh.
  • Debug the script: If you are still really, really stuck, run catalina.sh in debug mode. This is as easy as running it from upstart like this:

    exec /bin/sh -x $CATALINA_HOME/bin/catalina.sh run 2>/tmp/catalina-upstart.log
    

    Then you can compare the two debug logs and maybe spot where the two scripts did something different.

share|improve this answer
    
Comparing env output, the upstart environment was missing several variables (including $USER and $HOME). But even adding all of these environment variables to upstart does not reproduce the success of a command-line call. I added an example of the error (one of many) to the post above. (+1 for the vimdiff recommendation, thanks) –  Bosh Feb 22 '13 at 6:25
    
Well, I updated my answer with more ideas. I hope some of them point you in the right direction. –  chutz Feb 22 '13 at 6:37

To get a complete dump of all environmental elements between running your server on the command-line compared to running it under Upstart, try installing my procenv tool, running it in both environments and diffing the output files:

This is available in the ubuntu raring and debian sid archives (and FreeBSD).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.