3

I'm using imply to handle druid's cluster. But my logs files have increased to hundreds of gigabytes of storage. I'm talking about logs files present in imply/var/sv/ directory in which there are these 7 log files, broker.log, historical.log, middleManager.log zk.log, coordinator.log, imply-ui.log, and overlord.log.

Among them, this particular file called coordinator.log has increased to the really massive size of about 560 GBs in a matter of a few months. I have read all those logs, and they don't bother me much. What I'm concerned about is the size of the file which is eating my entire storage. I have tried finding ways to limit the size of those log files but believe me nothing worked for me.

I read in many places that druid uses log4j2 logger so we can limit the size using its configuration from log4j2.xml file. But again a big confusion there are four log4j2.xml files which one shall I modify?

I tried modifying all of them, but still, it didn't work. I'm kind of a fool while handling it seems like... Well so this is my request if anybody could point me in the right direction in limiting the size of these log files

Reza Mousavi
  • 4,420
  • 5
  • 31
  • 48
Point Networks
  • 1,071
  • 1
  • 13
  • 35

2 Answers2

1

You can setup a simple cron process to truncate these files periodically using truncate -s 0 imply/var/sv/*.log

mdeora
  • 4,152
  • 2
  • 19
  • 29
1

The default log level in imply distribution is set to info which generates a lots of logs. If they don't bother you much, you can set the log level to error so that logs will be generated only when there is any error during system runtime. To set that, you need to modify logger level inside conf/druid/_common/log4j2.xml file.

<?xml version="1.0" encoding="UTF-8" ?>
<Configuration status="WARN">
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{ISO8601} %p [%t] %c - %m%n"/>
        </Console>
    </Appenders>
    <Loggers>
        <Root level="error">
            <AppenderRef ref="Console"/>
        </Root>
    </Loggers>
</Configuration>

And even after do so, you should periodically truncate log files as @mdeora suggested.

Monzurul Shimul
  • 8,132
  • 2
  • 28
  • 42