1

we recently started using Google Chat in our organisation therefor we need to send alerts from Prometheus-Alertmanager to Google Chat room. below is my alertmanager config for Google chat

receiver:
    - name: 'gchat-receiver'
      webhook_configs:
        - url: "https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
          send_resolved: false

and when we implementing above config we are getting below error in our alertmanager logs:

level=error ts=2021-02-23T06:23:43.931Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
level=error ts=2021-02-23T06:23:43.971Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
level=error ts=2021-02-23T06:23:43.976Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=21 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"
level=error ts=2021-02-23T06:23:43.994Z caller=dispatch.go:309 component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="gchat-receiver/webhook[0]: notify retry canceled due to unrecoverable error after 1 attempts: unexpected status code 400: https://chat.googleapis.com/v1/spaces/AAAArcv5Snw/messages?key=KEY&token=TOKEN"

any help or guidance on this would be highly appreciated

rcarba
  • 667
  • 7
  • 22
chitender kumar
  • 394
  • 4
  • 21

1 Answers1

0

That is because Google Chat expects a specific JSON payload schema to be posted to that webhook address and the default Alertmanager payload doesn't match it, so you get HTTP 400 response: bad request.

You might need to put a custom application in between to transform the JSON to Google Chat acceptable format. There are already some repos in Github that match that general criteria as well. (e.g calert)

Ali Sattari
  • 314
  • 2
  • 6