-2

Today we were struggling in a situation when execution blocked ("blocked" - this should be more technical word but at this point i can't figure out) in Java EE project in JBoss EAP alpha version and after sometime there were no more console messages.

Finally we figured that a colleague of mine by mistake placed System.out.println() for logging inside a for loop, instead of slf4j logger (loop iterates hundreds of times).

After replacing System.out.println() with log.xxxxx() of slf4j logger everything was fine.

I can't understand this strange issue with System.out.println(). also, Application server was running but unable to serve any request.

Can someone help me to understand this issue and why JBoss was unable to process other requests.

Milan
  • 1,903
  • 17
  • 16
AsSiDe
  • 1,826
  • 2
  • 15
  • 24
  • were you able to reproduce that error, i.e. put it back `System.out.println()` and see what happens. – Milan Dec 09 '15 at 18:22
  • yes, it was giving the same pain on replacing slf4j logger from sysout. we struggled for 2-3 hours to detect the issue. – AsSiDe Dec 09 '15 at 18:27
  • did you try to attach remote debugger to confirm that you get stuck on that line? – Milan Dec 09 '15 at 18:39
  • no, because situation doesn't allow me to debug from eclipse. i have to deploy war in jboss. – AsSiDe Dec 09 '15 at 18:55
  • 1
    you can still debug 'from' eclipse even if your web application is not running 'in eclipse'. You deploy it to JBOSS and attach **remote** debugger. Check Eclipse - JBOSS Remote debugging instructions – Milan Dec 09 '15 at 19:03
  • Does execution not 'block' if you cast your call to invoke `System.err.println((Object) x);`? – jmehrens Dec 09 '15 at 20:01

1 Answers1

0

JBoss remapps the System.err with a PrintStream that redirects its output to logging. I would imagine that you are running into a deadlock between the logger lock and the stream lock.

  • Thread 1 performs logger.log->(locks the logger)->Console Appender->System.err (locks the stream)
  • Thread 2 performs System.err.println->(locks the stream)->logger.log->(locks the logger).

As you can see each thread is grabbing locks in the opposite order.

Impossible you say! The source code for org.jboss.logging.util.LoggerStream doesn't have any synchronized keywords at all. How can it deadlock if the LoggerStream doesn't grab a lock?

Simple, they didn't override every method declared in java.io.PrintStream. So if you called System.err.println(int) or System.err.println(boolean) you are in trouble.

jmehrens
  • 10,580
  • 1
  • 38
  • 47