0

I have a java-based application to run on ubuntu, in order to start with the server and keep it alive, I wrote an upstart script.

The problem is the upstart service started, it consumes 100% of cpu, there are bunch of threads of this app, only one is high.

Note that start the java application from command-line will not have this problem.

description "adworker upstart service"
#umask 0007
respawn limit 15 5
oom never

start on (local-filesystems
    and net-device-up IFACE!=lo)
stop on shutdown

respawn

pre-start script
  . /etc/adworker.conf
  rm -rf $LOG_DIR
  mkdir -p -m0755 $LOG_DIR
  chown $USER:$GROUP $LOG_DIR
end script

script
  . /etc/adworker.conf
  OPTS="-Djava.ext.dirs=lib"
  chdir $APP_DIR
  JAVA_OPTS="-Xms${XMS} -Xmx${XMX} -Xss${XSS} $OPTS"
  exec su -s /bin/sh -c "/usr/bin/java ${JAVA_OPTS} -classpath ${CP} ${MAIN_CLASS}" $USER > ${LOG_DIR}/adworker.stdio 2>&1
end script

post-stop script
end script

Thank you.

Leonmax
  • 121
  • 1
  • 7

1 Answers1

1

OK, problem solved:

referenced the solution from this post

used top -H to identify

  • the process pid 7848
  • the high CPU thread pid 7849

then in python

>>> hex(7849)
'0x1ea9'

do threaddump:

$ sudo jstack 7849 >> threaddumps.log
$ cat threaddumps.log | less

search for 0x1ea9 and find it's on line 24 of the main class which has a stupid while loop read from stdin.

"main" prio=10 tid=0x00007f6714008800 nid=0x1ea9 runnable [0x00007f6719088000]
   java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.readBytes(Native Method)
    at java.io.FileInputStream.read(FileInputStream.java:242)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
    - locked <0x00000000bfb81980> (a java.io.BufferedInputStream)
    at com.adcade.SessionEventWorker.main(SessionEventWorker.java:24)

I believe it's ok in commandline because of the low process priority, which is not the case in upstart.

Community
  • 1
  • 1
Leonmax
  • 121
  • 1
  • 7