1

I'm using zapp to log error messages on a service hosted on google cloud, and am seeing that while errors are logged successfully, the text stored in the "message" field of the google cloud log is the stack trace, and not the error message I have logged.

Example code:

var log *zap.Logger
if err := doStuff(); err != nil {
    log.Error(<error message I want to log>, zap.Error(err))
}

This works well except google cloud logging and stackdriver will use the stack-trace caught by the call to zap.Error in the message field of the structured log. The message I've defined appears in the msg field, but the former appears to be the one displayed predominantly in the logging console and used by stackdriver for indexing errors.

This means that when navigating logs and errors via the console, I only see stacktraces, and no indication of the associated error string.

The tricky thing is I have no idea if this "issue" is cloud-side or zapp-side. I've spent some time digging around in Zapp to no avail, and am out of ideas.

blackgreen
  • 34,072
  • 23
  • 111
  • 129
DaveBensonPhillips
  • 3,134
  • 1
  • 20
  • 32

2 Answers2

1

zap by default puts the message under the msg key, the stacktrace under stacktrace, and prints log lines as json to stdout. You should be able to see this in action by just running your binary locally.

Your logging system presumably processes these log lines as they're printed. It will read them, parse them, and maybe do some restructuring or add some metadata, and then send them off somewhere else to be saved or processed more.

Since zap is in all likeliness working as intended, you need to look at the system that processes your logs. How does it expect them to look? Does it have special rules for any particular keys? Will it inject any keys of its own?

Note that you can configure zap to use different keys for all of its standard fields.

Alex Guerra
  • 2,556
  • 3
  • 18
  • 24
  • Thanks - yeah Zapp does seem to be working as intended and it sounds like I can try to reconfigure zap to put the error message under "message", and see if google cloud will work with that. I'll give it a shot and get back. – DaveBensonPhillips Sep 18 '17 at 16:08
  • My hunch is that the log processor expects a `message` field, and if it doesn't find one will try to make a best guess. `zap.Error` adds the field under the `error` key, and that seems like a sensible alternative if there's no `message` field. I think you're right that configuring `msg` to be `message` in zap will fix the issue. Good luck! – Alex Guerra Sep 18 '17 at 16:12
  • Thanks, I can set the flags with `config := zap.NewProductionConfig(); config.EncoderConfig.StacktraceKey = "stack"` – DaveBensonPhillips Sep 18 '17 at 16:28
  • @DaveBensonPhilips I am wondering have you found the root cause of this? I am having the same issue here with gcp logs, and the zap config should be alright with `EncoderConfig.MessageKey = "message"` – 3tbraden Aug 03 '22 at 04:09
0

To log correctly and effectively on GCP, first, you have to set appropriate Zap keys with the Logging's LogEntry

In case you are looking for a working example, I write a simple Zap config here: https://github.com/uber-go/zap/discussions/1110#discussioncomment-2955566

Cong
  • 131
  • 1
  • 5