1

I have a spring boot service which writes data from DB as a JSON(string) to text files.

Since, this goes on continuously 24X7, I want to do this in a rolling fashion as in when size limit of a file is reached, it should create new file, and also create new directory for each date.

I have used logback-spring.xml configuration to write the logs to files in a rolling fashion as described above, but I can't find any library or framework that can write JSON(string) to a text file in such fashion and not only logs.

Can I use logback configuration to write both logs and the JSON content to text files in a rolling out fashion or is there any other library that I can use or I have to manually code the rolling process in my java file?

Edit: So, I want to rotate the log files as defined in logback-spring.xml configuration but I also want to rotate the text files that I'm writing to some other location.

Edit 2: Okay, so I have data in 3 tables: table1, table2, table3. I'm reading the data from these tables and writing to a text file using below code:

private void processEntry(Map<String, Object> entry) {

    FileWriter writer;
    Gson gson = new Gson();
    File f = new File(CALL_DATA_FILE_PATH);

    try {
        if (f.exists())
            writer = new FileWriter(CALL_DATA_FILE_PATH, true);
        else
            writer = new FileWriter(CALL_DATA_FILE_PATH);

        
        if (f.exists()) {
            writer.append(gson.toJson(entry) + ",");
            log.info("File exists, appending data to it");
        } else {
            writer.write(gson.toJson(entry) + ", \n");
            log.info("File does not exist, writing data to new file");
        }
        writer.flush();

        
    } catch (Exception e) {
        e.printStackTrace();
    }

}

Now, I want to write the data and rotate the files based on date and time and don't want to manually handle that. Can this be achieved using logback configuration even if logback is currently used to write logs to specified location.

logback-spring.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="LOGS" value="./logs" />

<appender name="Console"
    class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.classic.PatternLayout">
        <Pattern>
            %d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} -
            %msg%n
        </Pattern>
    </layout>
</appender>

<appender name="RollingFile"
    class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>${LOGS}/logs.log</file>
    <encoder
        class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
        <Pattern>%d %p %C{1.} [%t] %m%n</Pattern>
    </encoder>

    <rollingPolicy
        class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
        <fileNamePattern>${LOGS}/archived/%d{yyyy-MM-dd}/logs-%d{yyyy-MM-dd}.%i.log
        </fileNamePattern>
        <maxFileSize>10KB</maxFileSize>
        <maxHistory>30</maxHistory>
        <totalSizeCap>100KB</totalSizeCap>
        <cleanHistoryOnStart>false</cleanHistoryOnStart>
                
    </rollingPolicy>
</appender>

<root level="trace">
    <appender-ref ref="Console" />
    <appender-ref ref="RollingFile" />
</root>
</configuration>
Akki
  • 754
  • 7
  • 22
  • I might be missing something, but have you checked the log rotation related configurations in the logback? For example here: https://stackoverflow.com/questions/50927845/how-to-enable-logback-to-rotate-the-log-files-every-x-minutes-5-minutes-30-minut To me it looks like a way to go – Mark Bramnik Jun 27 '22 at 04:18
  • @MarkBramnik updated the question. I want to rotate the text files where I'm writing JSON data from DB. – Akki Jun 27 '22 at 04:21
  • When you write logs they're written line by line anyway, so the json should be converted to the string, because, `log.info` (and other methods) accepts String as a parameter. So when you rotate the log it will be done automatically by logback library regardless the fact that its a json here. As for "some other location" - basically you should define different appenders and attach them to the same logger. – Mark Bramnik Jun 27 '22 at 04:28
  • @MarkBramnik Okay got it. In my case the rotation is required not only for log files, but also the text files(where I'm writing data from database) which will be in different location. – Akki Jun 27 '22 at 04:40
  • There is no difference between "log" file and "text" file - both are text files. For both you control the format. Appender defines where to write (to file or maybe DB, network over socket, etc). So you should associate 2 different appenders with the same logger. Each appender will manage writing to the different location, and might have a different pattern (format of the message itself) as well as a different rotation policy (create a new file every time the log reaches 10Mb and keep up to 5 files like this) So it all depends on your actual configurations. – Mark Bramnik Jun 27 '22 at 06:14

0 Answers0