0

I would like to write logs into csv files for every minute, those csv files needs to be kept in different folders i.e., 60 minute log files in first hour folder and another 60 minutes which supposed to be the next hour folder.

When i use DailyRollingFileAppender which results in a exception after completing the particular hour, until the exception occurs files are getting created normally. If i start logging at 12:14 and runs fine up to 12:59 and exception occurs.

#include <unistd.h>
#include <log4cxx/logger.h>
#include <log4cxx/propertyconfigurator.h>

using namespace log4cxx;
using namespace log4cxx::helpers;

// Define static logger variable
LoggerPtr loggerMyMain(Logger::getLogger(""));
LoggerPtr loggerFunctionA(Logger::getLogger(""));

void functionA() {
    LOG4CXX_INFO(loggerFunctionA, "Executing functionA.");
    usleep(2000);
}

int main() {
    int value = 5;

    // Load properties style configuration file using PropertyConfigurator
    PropertyConfigurator::configure("log4j.properties");
    for(int j = 1; j < 10000; j++) {
        LOG4CXX_TRACE(
            loggerMyMain,
            "this is a debug message for detailed code discovery. Value=" << j);
        LOG4CXX_DEBUG(loggerMyMain, "this is a debug message.");
        LOG4CXX_INFO(loggerMyMain, "this is a info message, ignore. Value=" << j);
        LOG4CXX_WARN(loggerMyMain, "this is a warn message, not too bad.");
        LOG4CXX_ERROR(loggerMyMain,
                      "this is a error message, something serious is happening.");
        LOG4CXX_FATAL(loggerMyMain, "this is a fatal message!!!");

        functionA();
        if(j > 9998) j = 1;
    }
    return 0;
}

Configuration file log4j.properties:

# Define the root logger with appender file
log4j.rootLogger = DEBUG, FILE

# Define the file appender
log4j.appender.FILE=org.apache.log4j.DailyRollingFileAppender

# Set the name of the file
#log4j.appender.FILE.File=${'folder'/}

# Set the immediate flush to true (default)
log4j.appender.FILE.ImmediateFlush=false

# Set the threshold to debug mode
log4j.appender.FILE.Threshold=debug

# Set the append to false, should not overwrite
log4j.appender.FILE.Append=false

# Set the DatePattern
log4j.appender.FILE.DatePattern=yyyy-MM/yyyy-MM-dd/yyyy-MM-dd-HH/yyyy-MM-dd_HH:mm'.''csv'

# Define the layout for file appender
log4j.appender.FILE.layout=org.apache.log4j.PatternLayout
log4j.appender.FILE.layout.conversionPattern=%m%n

output is:

 g++ file.cpp  -lapr-1 -laprutil-1 -llog4cxx -o kk
 kk@deb:~/logs/logcfg$ ./kk
 log4cxx: Exception during rollover

expected output structure

   day01-|
         |----hour1 |----min1.csv
         |          |----min2.csv
         |          |----min60.csv
         |----hour-N
                |----min1.csv
                |----min60.csv
Ted Lyngmo
  • 93,841
  • 5
  • 60
  • 108
KaviKumar N
  • 35
  • 8
  • Please provide a [mcve]. You only provided the configuration and not your actual code. – Ted Lyngmo Jul 05 '19 at 08:53
  • Unrelated, but the whole point of splitting your path like that is that you eliminate files with common prefixes (as the worst case of filesystems is lots of files with a common prefix). A pattern like `yyyy-MM/dd/HH/yyyy-MM-dd_HH:mm'.''csv'` should be much friendlier on your file system – Botje Jul 05 '19 at 08:57
  • I have updated the minimal code. please have a look on it. – KaviKumar N Jul 05 '19 at 09:28
  • Logs written every minute? So when it crashes you have 1 minute old logs. – Marek R Jul 05 '19 at 09:42
  • If i start logging at 12.30 it runs fine until 12:59. when the clock hits 13 00 exception occurs. The exception occurs when it surpasses the hour to next hour. – KaviKumar N Jul 05 '19 at 09:44
  • 2
    `log4cxx` may throw exceptions so you should try to catch those by using the specific `log4cxx` exception class and see if that provides more info (doubtful, but still a good idea). – Ted Lyngmo Jul 05 '19 at 09:55
  • 1
    @KaviKumarN I've got two instances running with different configurations. One with your original and one with a simple `log4j.appender.FILE.DatePattern='file.'yyyy-MM-dd-HH-mm` pattern. Lets see what happens in 25 minutes :) – Ted Lyngmo Jul 05 '19 at 10:33
  • 1
    When catching the log4cxx exception (with `const Exception& ex` ) I didn't get more info than you. The instance using my simple `DatePattern` still runs though. My best guess is that this is a bug in `log4cxx`. It's probably not expecting to need to create the directories more than once. This should of course be checked on every rollover. – Ted Lyngmo Jul 05 '19 at 11:06
  • 1
    A note about the appender you use: "_DailyRollingFileAppender has been observed to exhibit synchronization issues and data loss. The log4j extras companion includes alternatives which should be considered for new deployments and which are discussed in the documentation for org.apache.log4j.rolling.RollingFileAppender._" – Ted Lyngmo Jul 05 '19 at 11:07
  • 1
    How did it work out? I got curious, but am sorry I couldn't provide an answer. – Ted Lyngmo Jul 06 '19 at 00:15

0 Answers0