6

I asked first this question on StackOverflow, but it seems that this site is a better fit.

I am setting up a web app environment to run Java apps. The environment is supposed to be:

  • AWS EC2 Ubuntu 18
  • Java 8
  • Tomcat 9

I launched an EC2 instance and installed java, then I followed this guide in order to set up Tomcat: https://www.digitalocean.com/community/tutorials/install-tomcat-9-ubuntu-1804

The problem now rises when systemctl tries to start/restart Tomcat service. Here is what I see when checking out systemctl logs:

Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: tomcat.service: Service hold-off time over, scheduling restart.
Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: tomcat.service: Scheduled restart job, restart counter is at 5.
Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: Stopped Apache Tomcat Web Application Container.
Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: Starting Apache Tomcat Web Application Container...
Dec 19 13:35:25 ip-10-165-10-74 startup.sh[1705]: Tomcat started.
Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: Started Apache Tomcat Web Application Container.
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: Dec 19, 2018 1:35:26 PM org.apache.catalina.startup.Catalina stopServer
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: SEVERE: Could not contact [localhost:8005] (base port [8005] and offset [0]). Tomcat may not be running.
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: Dec 19, 2018 1:35:26 PM org.apache.catalina.startup.Catalina stopServer
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: SEVERE: Error stopping Catalina
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: java.net.ConnectException: Connection refused (Connection refused)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.PlainSocketImpl.socketConnect(Native Method)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.Socket.connect(Socket.java:589)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.Socket.connect(Socket.java:538)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.Socket.(Socket.java:434)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.net.Socket.(Socket.java:211)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:513)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at java.lang.reflect.Method.invoke(Method.java:498)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:403)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]:         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:497)
Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: The stop command failed. Attempting to signal the process to stop through OS signal.
Dec 19 13:35:27 ip-10-165-10-74 shutdown.sh[1725]: Tomcat stopped.

The systemd config:

[Unit]
Description=Apache Tomcat Web Application Container
After=network.target

[Service]
Type=forking

Environment=JAVA_HOME=/usr/lib/java/jdk1.8.0_191
Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid
Environment=CATALINA_HOME=/opt/tomcat
Environment=CATALINA_BASE=/opt/tomcat
Environment='CATALINA_OPTS=-Xms256M -Xmx512M -server -XX:+UseParallelGC'
Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom'

ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh

User=tomcat
Group=tomcat
UMask=0007
RestartSec=10
Restart=always

[Install]
WantedBy=multi-user.target

As I understand the problem here is that Tomcat cannot connect to 8005 port in order to listen to shutdown commands.

However, I opened this port in EC2 Security Group console: 8005 127.0.0.1/32

The other strange thing is that Tomcat starts okay when rebooting ec2 instance, but fails to restart throwing the exception like above.

What could be a problem here? How do I check that port 8005 is actually open?

===========

Alright, I made some progress, I figured out that Tomcat seems to work fine. startup.sh works okay - you may see this in the log - it echoes "Tomcat started". This is the last line in catalina.sh start. However, I don't understand why systemctl tries to stop the service immediately after it starts it. It sends catalina.sh stop command.

4

3 Answers 3

4

Thanks @MichaelHampton for the good article: https://jdebp.eu/FGA/systemd-house-of-horror/tomcat.html

It explains the possible problems with a typical installation suggested in the tutorial I read (and tons of other copy-pasted ones). I could not figured out why the problem arises. Maybe it is the problem with all of the layers introduced by catalina.sh, startup.sh, etc.

So basically, I rewrote the systemd config script and it started to work.

0
4

I had the exact same problem. Only I was using Amazon Lightsail. Maybe there is a problem with the way that Amazon configures their Ubuntu machines, and the described way to create a Tomcat Service. I used the described way on a local Ubuntu machine that I set up myself and that worked fine.

I read the article by @michael-hampton and used the approach described there. I am a Linux noob, and I found that I had some problems getting it to work. I'm including the files I ended up with here so maybe other Linux noobs that have this problem can get an easier start. My Tomcat installation is at /opt/tomcat and these are the files I ended up with:

/etc/systemd/system/tomcat.service

[Unit]
Description=Apache Tomcat Web Application Container
After=network.target


[Service]
User=tomcat
Group=tomcat
UMask=0007

EnvironmentFile=/etc/default/tomcat
ExecStart=/usr/bin/env ${JAVA_HOME}/bin/java \
$JAVA_OPTS $CATALINA_OPTS \
-classpath ${CLASSPATH} \
-Dcatalina.base=${CATALINA_BASE} \
-Dcatalina.home=${CATALINA_HOME} \
-Djava.endorsed.dirs=${JAVA_ENDORSED_DIRS} \
-Djava.io.tmpdir=${CATALINA_TMPDIR} \
-Djava.util.logging.config.file=${CATALINA_BASE}/conf/logging.properties \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
org.apache.catalina.startup.Bootstrap \
start

ExecStop=/usr/bin/env ${JAVA_HOME}/bin/java \
$JAVA_OPTS $CATALINA_OPTS \
-classpath ${CLASSPATH} \
-Dcatalina.base=${CATALINA_BASE} \
-Dcatalina.home=${CATALINA_HOME} \
-Djava.endorsed.dirs=${JAVA_ENDORSED_DIRS} \
-Djava.io.tmpdir=${CATALINA_TMPDIR} \
-Djava.util.logging.config.file=${CATALINA_BASE}/conf/logging.properties \
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager \
org.apache.catalina.startup.Bootstrap \
stop

SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

/etc/default/tomcat

CATALINA_HOME=/opt/tomcat
CATALINA_BASE=/opt/tomcat
CATALINA_TMPDIR=/opt/tomcat/temp
CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC
JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom
JAVA_ENDORSED_DIRS=/opt/tomcat/endorsed
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
CLASSPATH=/opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar
1
  • Thank you. This helped me. I signed up to this stack only to upvote. Cheers. Oct 30, 2019 at 4:32
0

looks like this is due to a bug introduced in "recent" versions of Tomcat. Please check this answer: https://stackoverflow.com/a/54452870/4735682, as it points to the right bug reports.

The latest versions of Tomcat are free of this bug, so updating should fix your issue without having to fiddle with the systemd unit file.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .