How to configure a custom zap logger that writes to output only at error level?

Question

I’m trying to build a customized logger that keeps log messages below Error level in a buffer and flushes the buffer only after encountering an Error.

The problem is that I don’t know how to trigger the flushing of the logs to the output (Sync method) when encountering an Error.

The below code is an attempt to do so:

func CustomLogger() {
   // First, define our level-handling logic.
   lowPriority := zap.LevelEnablerFunc(func(lvl zapcore.Level) bool {
      return lvl < activationLevel
   })
   //define the output of the logs
   customWriteSyncer := Buffer(os.Stdout)

   consoleEncoder := zapcore.NewConsoleEncoder(zap.NewDevelopmentEncoderConfig())
   zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig())

   //creates a Core that writes logs to a WriteSyncer
   core := zapcore.NewCore(consoleEncoder, customWriteSyncer, lowPriority)

   logger := zap.New(core)

   defer logger.Sync()
  
   Logger = logger
}

type BufferWriterSync struct {
   buf *bufio.Writer
}

func Buffer(ws zapcore.WriteSyncer) zapcore.WriteSyncer {
   bw := &BufferWriterSync{
      buf: bufio.NewWriter(ws),
   }
   ws = zapcore.Lock(bw)
   return ws
}
// Sync syncs data to output
func (w BufferWriterSync) Sync() error {
   return w.buf.Flush()
}
// Write writes data to buffer
func (w BufferWriterSync) Write(p []byte) (int, error) {
   return w.buf.Write(p)
} 

Example, when performing:

• logger.Info("some Info message") this message ends up in the buffer of bufio.Writer and the Info message is not displayed

• logger.Info("some Info message2") this message ends up in the buffer of bufio.Writer and the Info message is not displayed

• logger.Error("some Error message") only when encountering logging of an error all the accumulated logs from the buffered must be flushed to the output, based on the above code example it should go to os.Stdout

Expected output:

some Info message
some Info message2
some Error message

NOTE: The functionality that I am trying to achieve is similar to fingers_crossed feature that is present in Php Symfony framework.

Answer 1

I don't think this is a good idea, in general. If your application never has errors, your buffer in theory could grow unbounded. This is the reason why Zap's own BufferedWriteSyncer has a size-based or timer-based flush: you must deterministically clear the buffer. However:

Print logs only at error level

To print logs only at a certain level, you can easily create a custom core with zap.NewAtomicLevelAt():

core := zapcore.NewCore(
        zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
        os.Stdout,
        zap.LevelEnablerFunc(func(level zapcore.Level) bool {
            return level == zapcore.ErrorLevel
        }),
    )
logger := zap.New(core)

logger.Info("bar")  // not logged
logger.Error("baz") // logged

Of course that will work similarly for other log levels.

Buffer and flush only at error level

As I said above, you should favor deterministic flush logic. Anyway if you can prove that your buffer will eventually clear, for completeness' sake, the following is a working solution. It's a bit awkward, because AFAIK there's no (sane) way to access the log entry details, as the log level, from the zapcore.WriteSyncer.

You have to create a custom core and implement buffering. The idea is to embed a zapcore.Encoder in your custom struct and implement Encoder.EncodeEntry with the buffering logic. The following is a demonstrative program (it is not meant to be thread-safe, memory-efficient, etc...):

// struct to "remember" buffered entries
type log struct {
    entry zapcore.Entry
    fields []zapcore.Field
}

// custom encoder
type bufferEncoder struct {
    // embeds a zapcore encoder
    zapcore.Encoder

    // the entry buffer itself
    buffer []*log

    // the buffer pool, to return encoded entries
    pool buffer.Pool
}

func (b *bufferEncoder) EncodeEntry(entry zapcore.Entry, fields []zapcore.Field) (*buffer.Buffer, error) {
    // buffer entries
    b.buffer = append(b.buffer, &log{entry, fields})

    // return an empty buffer if the level is not error
    if entry.Level != zap.ErrorLevel {
        return b.pool.Get(), nil
    }
    
    // new buffer
    buf := b.pool.Get()
    for _, log := range b.buffer {
        // encode buffered entries and append them to buf
        encoded, err := b.Encoder.EncodeEntry(log.entry, log.fields)
        if err != nil {
            return nil, err
        }
        buf.AppendString(encoded.String())
    }
    // reset the buffer before returning
    b.buffer = nil
    return buf, nil
}


func main() {
    enc := &bufferEncoder{
        Encoder: zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
        pool: buffer.NewPool(),
    }

    core := zapcore.NewCore(enc, os.Stdout, zap.NewAtomicLevelAt(zap.InfoLevel))
    logger := zap.New(core)

    logger.Info("info")
    fmt.Println("buffered info")
    time.Sleep(500 * time.Millisecond)

    logger.Warn("warn")
    fmt.Println("buffered warn")
    time.Sleep(500 * time.Millisecond)

    logger.Error("error")
    fmt.Println("flushed")
}