I asked first this question on StackOverflow, but it seems that this site is a better fit.
I am setting up a web app environment to run Java apps. The environment is supposed to be:
- AWS EC2 Ubuntu 18
- Java 8
- Tomcat 9
I launched an EC2 instance and installed java, then I followed this guide in order to set up Tomcat: https://www.digitalocean.com/community/tutorials/install-tomcat-9-ubuntu-1804
The problem now rises when systemctl
tries to start/restart Tomcat service. Here is what I see when checking out systemctl logs:
Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: tomcat.service: Service hold-off time over, scheduling restart. Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: tomcat.service: Scheduled restart job, restart counter is at 5. Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: Stopped Apache Tomcat Web Application Container. Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: Starting Apache Tomcat Web Application Container... Dec 19 13:35:25 ip-10-165-10-74 startup.sh[1705]: Tomcat started. Dec 19 13:35:25 ip-10-165-10-74 systemd[1]: Started Apache Tomcat Web Application Container. Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: Dec 19, 2018 1:35:26 PM org.apache.catalina.startup.Catalina stopServer Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: SEVERE: Could not contact [localhost:8005] (base port [8005] and offset [0]). Tomcat may not be running. Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: Dec 19, 2018 1:35:26 PM org.apache.catalina.startup.Catalina stopServer Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: SEVERE: Error stopping Catalina Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: java.net.ConnectException: Connection refused (Connection refused) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.PlainSocketImpl.socketConnect(Native Method) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.Socket.connect(Socket.java:589) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.Socket.connect(Socket.java:538) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.Socket.(Socket.java:434) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.net.Socket.(Socket.java:211) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:513) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at java.lang.reflect.Method.invoke(Method.java:498) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:403) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:497) Dec 19 13:35:26 ip-10-165-10-74 shutdown.sh[1725]: The stop command failed. Attempting to signal the process to stop through OS signal. Dec 19 13:35:27 ip-10-165-10-74 shutdown.sh[1725]: Tomcat stopped.
The systemd config:
[Unit] Description=Apache Tomcat Web Application Container After=network.target [Service] Type=forking Environment=JAVA_HOME=/usr/lib/java/jdk1.8.0_191 Environment=CATALINA_PID=/opt/tomcat/temp/tomcat.pid Environment=CATALINA_HOME=/opt/tomcat Environment=CATALINA_BASE=/opt/tomcat Environment='CATALINA_OPTS=-Xms256M -Xmx512M -server -XX:+UseParallelGC' Environment='JAVA_OPTS=-Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom' ExecStart=/opt/tomcat/bin/startup.sh ExecStop=/opt/tomcat/bin/shutdown.sh User=tomcat Group=tomcat UMask=0007 RestartSec=10 Restart=always [Install] WantedBy=multi-user.target
As I understand the problem here is that Tomcat cannot connect to 8005 port in order to listen to shutdown commands.
However, I opened this port in EC2 Security Group console:
8005 127.0.0.1/32
The other strange thing is that Tomcat starts okay when rebooting ec2 instance, but fails to restart throwing the exception like above.
What could be a problem here? How do I check that port 8005 is actually open?
===========
Alright, I made some progress, I figured out that Tomcat seems to work fine. startup.sh
works okay - you may see this in the log - it echoes "Tomcat started". This is the last line in catalina.sh start
. However, I don't understand why systemctl
tries to stop the service immediately after it starts it. It sends catalina.sh stop
command.