log4j redirect stdout to DailyRollingFileAppender

Question

I have a java app that uses log4j.

Config:

log4j.rootLogger=info, file

log4j.appender.file=org.apache.log4j.DailyRollingFileAppender
log4j.appender.file.File=${user.home}/logs/app.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d [%t] %c %p %m%n

So all the log statements are correctly appended to the file, but i am losing stdout and stderr. How do i redirect exception stack traces and sysouts to the daily rolled file ?

Answer 1

// I set up a ConsoleAppender in Log4J to format Stdout/Stderr
log4j.rootLogger=DEBUG, CONSOLE
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=[%t] %-5p %c - %m%n


// And I call this StdOutErrLog.tieSystemOutAndErrToLog() on startup

public class StdOutErrLog {

    private static final Logger logger = Logger.getLogger(StdOutErrLog.class);

    public static void tieSystemOutAndErrToLog() {
        System.setOut(createLoggingProxy(System.out));
        System.setErr(createLoggingProxy(System.err));
    }

    public static PrintStream createLoggingProxy(final PrintStream realPrintStream) {
        return new PrintStream(realPrintStream) {
            public void print(final String string) {
                realPrintStream.print(string);
                logger.info(string);
            }
        };
    }
}

Answer 2

I picked up the idea from Michael S., but like mentioned in one comment, it has some problems: it doesn't capture everything, and it prints some empty lines.

Also I wanted to separate System.out and System.err, so that System.out gets logged with log level 'INFO' and System.err gets logged with 'ERROR' (or 'WARN' if you like).

So this is my solution: First a class that extends OutputStream (it's easier to override all methods for OutputStream than for PrintStream). It logs with a specified log level and also copies everything to another OutputStream. And also it detects "empty" strings (containing whitespace only) and does not log them.

import java.io.IOException;
import java.io.OutputStream;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class LoggerStream extends OutputStream
{
private final Logger logger;
private final Level logLevel;
private final OutputStream outputStream;

public LoggerStream(Logger logger, Level logLevel, OutputStream outputStream)
{
    super();

    this.logger = logger;
    this.logLevel = logLevel;
    this.outputStream = outputStream;
}

@Override
public void write(byte[] b) throws IOException
{
    outputStream.write(b);
    String string = new String(b);
    if (!string.trim().isEmpty())
        logger.log(logLevel, string);
}

@Override
public void write(byte[] b, int off, int len) throws IOException
{
    outputStream.write(b, off, len);
    String string = new String(b, off, len);
    if (!string.trim().isEmpty())
        logger.log(logLevel, string);
}

@Override
public void write(int b) throws IOException
{
    outputStream.write(b);
    String string = String.valueOf((char) b);
    if (!string.trim().isEmpty())
        logger.log(logLevel, string);
}
}

And then a very simple utility class to set out and err:

import java.io.PrintStream;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class OutErrLogger
{
public static void setOutAndErrToLog()
{
    setOutToLog();
    setErrToLog();
}

public static void setOutToLog()
{
    System.setOut(new PrintStream(new LoggerStream(Logger.getLogger("out"), Level.INFO, System.out)));
}

public static void setErrToLog()
{
    System.setErr(new PrintStream(new LoggerStream(Logger.getLogger("err"), Level.ERROR, System.err)));
}

}

Answer 3

In Skaffman code : To remove empty lines in log4j logs, just add "println" method to PrintStream of createLoggingProxy

public static PrintStream createLoggingProxy(final PrintStream realPrintStream) {
    return new PrintStream(realPrintStream) {
        public void print(final String string) {
            logger.warn(string);
        }
        public void println(final String string) {
            logger.warn(string);
        }
    };
}

Answer 4

The answers above give an excellent idea to use proxy for the STDOUT/ERR logging. However provided implementation examples do not work nicely for all cases. For example, try

System.out.printf("Testing %s\n", "ABC");

The code from examples above will cut the output into separate pieces on a console and in multiple not readable Log4j entries.

The solution is to buffer the output until a trigger '\n' is found at buffer's end. Sometimes the buffer ends with '\r\n'. The class below addresses this issue. It is fully functional. Call the static method bind() to activate it.

import java.io.IOException;
import java.io.OutputStream;
import java.io.PrintStream;

import org.apache.log4j.Level;
import org.apache.log4j.Logger;

// Based on
// http://stackoverflow.com/questions/1200175/log4j-redirect-stdout-to-dailyrollingfileappender
public class Log4jStdOutErrProxy {

  public static void bind() {
    bind(Logger.getLogger("STDOUT"), Logger.getLogger("STDERR"));
  }

  @SuppressWarnings("resource")
  public static void bind(Logger loggerOut, Logger loggerErr) {
    System.setOut(new PrintStream(new LoggerStream(loggerOut, Level.INFO,  System.out), true));
    System.setErr(new PrintStream(new LoggerStream(loggerErr, Level.ERROR, System.err), true));
  }

  private static class LoggerStream extends OutputStream {
    private final Logger logger;
    private final Level logLevel;
    private final OutputStream outputStream;
    private StringBuilder sbBuffer;

    public LoggerStream(Logger logger, Level logLevel, OutputStream outputStream) {
      this.logger = logger;
      this.logLevel = logLevel;
      this.outputStream = outputStream;
      sbBuffer = new StringBuilder();
    }

    @Override
    public void write(byte[] b) throws IOException {
      doWrite(new String(b));
    }

    @Override
    public void write(byte[] b, int off, int len) throws IOException {
      doWrite(new String(b, off, len));
    }

    @Override
    public void write(int b) throws IOException {
      doWrite(String.valueOf((char)b));
    }

    private void doWrite(String str) throws IOException {
      sbBuffer.append(str);
      if (sbBuffer.charAt(sbBuffer.length() - 1) == '\n') {
        // The output is ready
        sbBuffer.setLength(sbBuffer.length() - 1); // remove '\n'
        if (sbBuffer.charAt(sbBuffer.length() - 1) == '\r') {
          sbBuffer.setLength(sbBuffer.length() - 1); // remove '\r'
        }
        String buf = sbBuffer.toString();
        sbBuffer.setLength(0);
        outputStream.write(buf.getBytes());
        outputStream.write('\n');
        logger.log(logLevel, buf);
      }
    }
  } // inner class LoggerStream  

}

Answer 5

For those looking for how to do this in log4j2. There is now a component to create these streams for you.

Requires including the log4j-iostreams jar
See: https://logging.apache.org/log4j/2.x/log4j-iostreams.html

Example:

PrintStream logger = IoBuilder.forLogger("System.out").setLevel(Level.DEBUG).buildPrintStream();
PrintStream errorLogger = IoBuilder.forLogger("System.err").setLevel(Level.ERROR).buildPrintStream();
System.setOut(logger);
System.setErr(errorLogger);

Answer 6

If you are using an application server, servlet container or something similar, see kgiannakakis's answer.

For standalone apps see this. You can reassing stdin, stdout and stderr using java.lang.System class. Basically you create a new subclass of PrintStream and set that instance to System.out.

Something along these lines in start of your app (untested).

// PrintStream object that prints stuff to log4j logger
public class Log4jStream extends PrintStream {
      public void write(byte buf[], int off, int len) {
        try {
           // write stuff to Log4J logger
        } catch (Exception e) {
       }
    }
}

// reassign standard output to go to log4j
System.setOut(new Log4jStream());

Answer 7

I presume you're logging stacktraces via e.printStackTrace() ? You can pass an exception object into the Log4j logging methods and those will appear in your log (see Logger.error(Object obj, Throwable t))

Note that you can change System.out and System.err to another PrintStream that redirects to Log4j. That would be a straightforward change and save you converting all your System.out.println() statements.

Answer 8

Standard output and error streams are managed from your container. For example Tomcat uses JULI to log output and error streams.

My recommendation is to leave these as it is. Avoid using System.out.print in your application. See here for stack traces.

Answer 9

The anser of @Michael is a good Point. But extending PrintStream is not very nice, because it uses a internal method void write(String) to write all things to an OutputStream.

I prefer to use the LoggingOutputStream Class from the Log4J Contrib package.

Then i redirect the system-streams like this:

public class SysStreamsLogger {
    private static Logger sysOutLogger = Logger.getLogger("SYSOUT");
    private static Logger sysErrLogger = Logger.getLogger("SYSERR");

    public static final PrintStream sysout = System.out;
    public static final PrintStream syserr = System.err;

    public static void bindSystemStreams() {
        // Enable autoflush
        System.setOut(new PrintStream(new LoggingOutputStream(sysOutLogger, LogLevel.INFO), true));
        System.setErr(new PrintStream(new LoggingOutputStream(sysErrLogger, LogLevel.ERROR), true));
    }

    public static void unbindSystemStreams() {
        System.setOut(sysout);
        System.setErr(syserr);
    }
}

Answer 10

Before using the System.setOut and System.setErr method we should reset the java.util.logging.LogManager object by using reset() method.

public static void tieSystemOutAndErrToLog() {

    try{

        // initialize logging to go to rolling log file
        LogManager logManager = LogManager.getLogManager();
        logManager.reset();

        // and output on the original stdout
        System.out.println("Hello on old stdout");
        System.setOut(createLoggingProxy(System.out));
        System.setErr(createLoggingProxy(System.err));

        //Following is to make sure whether system.out and system.err is redirecting to the stdlog.log file
        System.out.println("Hello world!");

        try {
            throw new RuntimeException("Test");
        } catch (Exception e) {
            e.printStackTrace();
        }

    }catch(Exception e){
        logger.error("Caught exception at StdOutErrLog ",e);
        e.printStackTrace();
    }
}