0

I have a java project with embedded Tomcat 8.5. I have add a new service to sytemd to run it as a service in CentOS 7.2. When I use "systemctl start app.service" to start the server, the program will stop at

"Aug 16, 2018 5:54:13 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/8.5.32"  

I have used "jstack" command to check the process, then I got information below:


    2018-08-16 18:28:34
    Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.152-b16 mixed mode):

    "Attach Listener" #14 daemon prio=9 os_prio=0 tid=0x00007fe804001000 nid=0x19a5 waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "localhost-startStop-1" #13 daemon prio=5 os_prio=0 tid=0x00007fe7e0001800 nid=0x190c runnable [0x00007fe8204a8000]
       java.lang.Thread.State: RUNNABLE
        at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
        at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)
        at java.io.File.isDirectory(File.java:849)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2011)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsFile(ContextConfig.java:2020)
        at org.apache.catalina.startup.ContextConfig.processAnnotationsUrl(ContextConfig.java:1963)
        at org.apache.catalina.startup.ContextConfig.processAnnotations(ContextConfig.java:1913)
        at org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1153)
        at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:765)
        - locked  (a org.apache.catalina.startup.ContextConfig)
        at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:299)
        at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:94)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5154)
        - locked  (a org.apache.catalina.core.StandardContext)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked  (a org.apache.catalina.core.StandardContext)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1421)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1411)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    "Tomcat-startStop-1" #12 daemon prio=5 os_prio=0 tid=0x00007fe8486d7000 nid=0x190b waiting on condition [0x00007fe8205ad000]
       java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for   (a java.util.concurrent.FutureTask)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
        at java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941)
        - locked  (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:872)
        - locked  (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked  (a org.apache.catalina.core.StandardHost)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1421)
        at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1411)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

    "NioBlockingSelector.BlockPoller-1" #11 daemon prio=5 os_prio=0 tid=0x00007fe8486c3800 nid=0x190a runnable [0x00007fe8206ae000]
       java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked  (a sun.nio.ch.Util$3)
        - locked  (a java.util.Collections$UnmodifiableSet)
        - locked  (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:298)

    "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x00007fe848102800 nid=0x1902 runnable [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "C1 CompilerThread2" #8 daemon prio=9 os_prio=0 tid=0x00007fe8480ff800 nid=0x1901 waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "C2 CompilerThread1" #7 daemon prio=9 os_prio=0 tid=0x00007fe8480fd800 nid=0x1900 waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "C2 CompilerThread0" #6 daemon prio=9 os_prio=0 tid=0x00007fe8480fb000 nid=0x18ff waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "Signal Dispatcher" #5 daemon prio=9 os_prio=0 tid=0x00007fe8480f9000 nid=0x18fe runnable [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "Surrogate Locker Thread (Concurrent GC)" #4 daemon prio=9 os_prio=0 tid=0x00007fe8480f7800 nid=0x18fd waiting on condition [0x0000000000000000]
       java.lang.Thread.State: RUNNABLE

    "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fe8480c4000 nid=0x18fc in Object.wait() [0x00007fe821dfc000]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
        - locked  (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

    "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fe8480bf800 nid=0x18fb in Object.wait() [0x00007fe821efd000]
       java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
        - locked  (a java.lang.ref.Reference$Lock)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)

    "main" #1 prio=5 os_prio=0 tid=0x00007fe84800b800 nid=0x18f4 waiting on condition [0x00007fe8509c5000]
       java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for   (a java.util.concurrent.FutureTask)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
        at java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941)
        - locked  (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
        - locked  (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked  (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422)
        - locked  (a org.apache.catalina.core.StandardEngine)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked  (a org.apache.catalina.core.StandardService)
        at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:793)
        - locked  (a java.lang.Object)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        - locked  (a org.apache.catalina.core.StandardServer)
        at org.apache.catalina.startup.Tomcat.start(Tomcat.java:367)
        at app.test.server.HttpServer.startServer(HttpServer.java:107)
        at app.test.ServerBootstrap.main(ServerBootstrap.java:27)

    "VM Thread" os_prio=0 tid=0x00007fe8480b7800 nid=0x18fa runnable 

    "Gang worker#0 (Parallel GC Threads)" os_prio=0 tid=0x00007fe84801c800 nid=0x18f5 runnable 

    "Gang worker#1 (Parallel GC Threads)" os_prio=0 tid=0x00007fe84801e000 nid=0x18f6 runnable 

    "Gang worker#2 (Parallel GC Threads)" os_prio=0 tid=0x00007fe848020000 nid=0x18f7 runnable 

    "Gang worker#3 (Parallel GC Threads)" os_prio=0 tid=0x00007fe848022000 nid=0x18f8 runnable 

    "Concurrent Mark-Sweep GC Thread" os_prio=0 tid=0x00007fe848065000 nid=0x18f9 runnable 

    "VM Periodic Task Thread" os_prio=0 tid=0x00007fe848108000 nid=0x1903 waiting on condition 

    JNI global references: 64

I have found there is

  • locked <0x00000006cb0240e0> (a org.apache.catalina.startup.ContextConfig)

My service script in "/etc/systemd/system" is:


    [Unit]
    Description=testapp
    After=network.target
    [Service]
    Type=forking
    ExecStart=/test/app/startup.sh
    [Install]
    WantedBy=multi-user.target

My startup.sh script content is:


      #!/bin/sh
      LIB=/test/app/lib
      CONFIG_FILE_PATH=/test/app/config
      CP=.:$CLASSPATH
      for f in ${LIB}/*.*;
        do CP=${CP}:$f;
      done
      CP=${CP}:${CONFIG_FILE_PATH};

      export CLASSPATH=$CP
      nohup /usr/local/jdk1.7.0_80/bin/java -classpath $CLASSPATH  app.test.ServerBootstrap ${CONFIG_FILE_PATH} >> test.out 2>&1 &

When I run my "startup.sh" script, there is no problem. When I use "systemctl" command, the program will pause, but the process will keep running.

How can I deal with this problem. Is there any diference between running shell script and "systemctl" command?

  • What does this startup.sh script? Please post a copy of the script. – Michael Hampton Aug 29 '18 at 14:24
  • @MichaelHampton Thanks. I have added my "startup.sh' content. – lbear Aug 29 '18 at 14:30
  • Interesting. I guess that ought to work. But why don't you just start the app directly from the systemd unit, instead of calling the script? The only difference is that you'd have to either fix CLASSPATH or call a script with ExecStartPre= to re-generate it, if that is even important at all (probably not). – Michael Hampton Aug 29 '18 at 14:33
  • @MichaelHampton It works. And I would try to find out the reason why there is so many recursion of processAnnotationsFile mehtod later. Thank you so much. – lbear Aug 30 '18 at 3:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.