0

I was just testing an event-trigger on my users table, and while the trigger works (it hits the webhook just fine) it fails attempting to log that the trigger works. I get the following error on 1.3.0 (via docker)

{
  "type": "event-trigger",
  "timestamp": "2020-08-12T01:03:20.880+0000",
  "level": "error",
  "detail": {
    "internal": {
      "statement": "\n          INSERT INTO hdb_catalog.event_invocation_logs (event_id, status, request, response)\n          VALUES ($1, $2, $3, $4)\n          ",
      "prepared": true,
      "error": {
        "exec_status": "FatalError",
        "hint": null,
        "message": "null value in column \"id\" violates not-null constraint",
        "status_code": "23502",
        "description": "Failing row contains (null, 2ee6ad0a-187b-46ba-9ae8-c62d21c908db, 200, {\"payload\":{\"event\":{\"session_variables\":{\"x-hasura-role\":\"admin..., {\"data\":{\"status\":200,\"body\":\"{\\\"job_id\\\": \\\"8b4ed8f6-79f8-44bd-..., 2020-08-12 01:03:21.860502)."
      },
      "arguments": [
        "(Oid 25,Just (\"2ee6ad0a-187b-46ba-9ae8-c62d21c908db\",Binary))",
        "(Oid 20,Just (\"\\NUL\\NUL\\NUL\\NUL\\NUL\\NUL\\NUL\\200\",Binary))",
        "(Oid 114,Just (\"{\\\"payload\\\":{\\\"event\\\":{\\\"session_variables\\\":{\\\"x-hasura-role\\\":\\\"admin\\\"},\\\"op\\\":\\\"MANUAL\\\",\\\"data\\\":{\\\"old\\\":null,\\\"new\\\":{\\\"email\\\":\\\"whatever@foo.com\\\",\\\"display_name\\\":\\\"Some Dude\\\",\\\"auth0_id\\\":\\\"auth0|00000000000\\\",\\\"username\\\":\\\"dude\\\",\\\"updated_at\\\":\\\"2020-08-12T00:45:43.004511+00:00\\\",\\\"created_at\\\":\\\"2020-08-06T21:23:52.923922+00:00\\\",\\\"id\\\":1,\\\"avatar_url\\\":null,\\\"description\\\":null}}},\\\"created_at\\\":\\\"2020-08-12T00:58:39.870033Z\\\",\\\"id\\\":\\\"2ee6ad0a-187b-46ba-9ae8-c62d21c908db\\\",\\\"delivery_info\\\":{\\\"max_retries\\\":3,\\\"current_retry\\\":0},\\\"trigger\\\":{\\\"name\\\":\\\"maintain_sengrid_lists\\\"},\\\"table\\\":{\\\"schema\\\":\\\"public\\\",\\\"name\\\":\\\"users\\\"}},\\\"headers\\\":[{\\\"value\\\":\\\"application/json\\\",\\\"name\\\":\\\"Content-Type\\\"},{\\\"value\\\":\\\"hasura-graphql-engine/v1.3.0\\\",\\\"name\\\":\\\"User-Agent\\\"}],\\\"version\\\":\\\"2\\\"}\",Binary))",
        "(Oid 114,Just (\"{\\\"data\\\":{\\\"status\\\":200,\\\"body\\\":\\\"{\\\\\\\"job_id\\\\\\\": \\\\\\\"8b4ed8f6-79f8-44bd-981e-fb59336a01e7\\\\\\\"}\\\",\\\"headers\\\":[{\\\"value\\\":\\\"50\\\",\\\"name\\\":\\\"Content-Length\\\"},{\\\"value\\\":\\\"application/json\\\",\\\"name\\\":\\\"Content-Type\\\"},{\\\"value\\\":\\\"Caddy\\\",\\\"name\\\":\\\"Server\\\"},{\\\"value\\\":\\\"5d5897\\\",\\\"name\\\":\\\"X-Request-Id\\\"},{\\\"value\\\":\\\"Wed, 12 Aug 2020 01:03:21 GMT\\\",\\\"name\\\":\\\"Date\\\"}]},\\\"version\\\":\\\"2\\\",\\\"type\\\":\\\"webhook_response\\\"}\",Binary))"
      ]
    },
    "path": "$",
    "error": "database query error",
    "code": "unexpected"
  }
}

The one thing I can think of that may be causing this is I created the event trigger, tried to trigger it, then noticed a console error regarding gen_random_uuid being a missing function. I did CREATE EXTENSION pgcrypto then attempted to recreate the trigger. As I said above, the webhook portion seems correct, it's hitting the correct hook and getting the correct results, this appears to be an event-trigger logging issue.

UPDATE: I also cannot create a cron-trigger. Getting this error when hitting create in the console:

{
    "statement": "INSERT INTO \"hdb_catalog\".\"hdb_cron_events\" ( \"trigger_name\", \"scheduled_time\" ) VALUES ('event_materializer', '2020-08-12T18:00:00Z'), ('event_materializer', '2020-08-12T18:10:00Z'), ('event_materializer', '2020-08-12T18:20:00Z'), ('event_materializer', '2020-08-12T18:30:00Z'), ('event_materializer', '2020-08-12T18:40:00Z'), ('event_materializer', '2020-08-12T18:50:00Z'), ('event_materializer', '2020-08-12T19:00:00Z'), ('event_materializer', '2020-08-12T19:10:00Z'), ('event_materializer', '2020-08-12T19:20:00Z'), ('event_materializer', '2020-08-12T19:30:00Z'), ('event_materializer', '2020-08-12T19:40:00Z'), ('event_materializer', '2020-08-12T19:50:00Z'), ('event_materializer', '2020-08-12T20:00:00Z'), ('event_materializer', '2020-08-12T20:10:00Z'), ('event_materializer', '2020-08-12T20:20:00Z'), ('event_materializer', '2020-08-12T20:30:00Z'), ('event_materializer', '2020-08-12T20:40:00Z'), ('event_materializer', '2020-08-12T20:50:00Z'), ('event_materializer', '2020-08-12T21:00:00Z'), ('event_materializer', '2020-08-12T21:10:00Z'), ('event_materializer', '2020-08-12T21:20:00Z'), ('event_materializer', '2020-08-12T21:30:00Z'), ('event_materializer', '2020-08-12T21:40:00Z'), ('event_materializer', '2020-08-12T21:50:00Z'), ('event_materializer', '2020-08-12T22:00:00Z'), ('event_materializer', '2020-08-12T22:10:00Z'), ('event_materializer', '2020-08-12T22:20:00Z'), ('event_materializer', '2020-08-12T22:30:00Z'), ('event_materializer', '2020-08-12T22:40:00Z'), ('event_materializer', '2020-08-12T22:50:00Z'), ('event_materializer', '2020-08-12T23:00:00Z'), ('event_materializer', '2020-08-12T23:10:00Z'), ('event_materializer', '2020-08-12T23:20:00Z'), ('event_materializer', '2020-08-12T23:30:00Z'), ('event_materializer', '2020-08-12T23:40:00Z'), ('event_materializer', '2020-08-12T23:50:00Z'), ('event_materializer', '2020-08-13T00:00:00Z'), ('event_materializer', '2020-08-13T00:10:00Z'), ('event_materializer', '2020-08-13T00:20:00Z'), ('event_materializer', '2020-08-13T00:30:00Z'), ('event_materializer', '2020-08-13T00:40:00Z'), ('event_materializer', '2020-08-13T00:50:00Z'), ('event_materializer', '2020-08-13T01:00:00Z'), ('event_materializer', '2020-08-13T01:10:00Z'), ('event_materializer', '2020-08-13T01:20:00Z'), ('event_materializer', '2020-08-13T01:30:00Z'), ('event_materializer', '2020-08-13T01:40:00Z'), ('event_materializer', '2020-08-13T01:50:00Z'), ('event_materializer', '2020-08-13T02:00:00Z'), ('event_materializer', '2020-08-13T02:10:00Z'), ('event_materializer', '2020-08-13T02:20:00Z'), ('event_materializer', '2020-08-13T02:30:00Z'), ('event_materializer', '2020-08-13T02:40:00Z'), ('event_materializer', '2020-08-13T02:50:00Z'), ('event_materializer', '2020-08-13T03:00:00Z'), ('event_materializer', '2020-08-13T03:10:00Z'), ('event_materializer', '2020-08-13T03:20:00Z'), ('event_materializer', '2020-08-13T03:30:00Z'), ('event_materializer', '2020-08-13T03:40:00Z'), ('event_materializer', '2020-08-13T03:50:00Z'), ('event_materializer', '2020-08-13T04:00:00Z'), ('event_materializer', '2020-08-13T04:10:00Z'), ('event_materializer', '2020-08-13T04:20:00Z'), ('event_materializer', '2020-08-13T04:30:00Z'), ('event_materializer', '2020-08-13T04:40:00Z'), ('event_materializer', '2020-08-13T04:50:00Z'), ('event_materializer', '2020-08-13T05:00:00Z'), ('event_materializer', '2020-08-13T05:10:00Z'), ('event_materializer', '2020-08-13T05:20:00Z'), ('event_materializer', '2020-08-13T05:30:00Z'), ('event_materializer', '2020-08-13T05:40:00Z'), ('event_materializer', '2020-08-13T05:50:00Z'), ('event_materializer', '2020-08-13T06:00:00Z'), ('event_materializer', '2020-08-13T06:10:00Z'), ('event_materializer', '2020-08-13T06:20:00Z'), ('event_materializer', '2020-08-13T06:30:00Z'), ('event_materializer', '2020-08-13T06:40:00Z'), ('event_materializer', '2020-08-13T06:50:00Z'), ('event_materializer', '2020-08-13T07:00:00Z'), ('event_materializer', '2020-08-13T07:10:00Z'), ('event_materializer', '2020-08-13T07:20:00Z'), ('event_materializer', '2020-08-13T07:30:00Z'), ('event_materializer', '2020-08-13T07:40:00Z'), ('event_materializer', '2020-08-13T07:50:00Z'), ('event_materializer', '2020-08-13T08:00:00Z'), ('event_materializer', '2020-08-13T08:10:00Z'), ('event_materializer', '2020-08-13T08:20:00Z'), ('event_materializer', '2020-08-13T08:30:00Z'), ('event_materializer', '2020-08-13T08:40:00Z'), ('event_materializer', '2020-08-13T08:50:00Z'), ('event_materializer', '2020-08-13T09:00:00Z'), ('event_materializer', '2020-08-13T09:10:00Z'), ('event_materializer', '2020-08-13T09:20:00Z'), ('event_materializer', '2020-08-13T09:30:00Z'), ('event_materializer', '2020-08-13T09:40:00Z'), ('event_materializer', '2020-08-13T09:50:00Z'), ('event_materializer', '2020-08-13T10:00:00Z'), ('event_materializer', '2020-08-13T10:10:00Z'), ('event_materializer', '2020-08-13T10:20:00Z'), ('event_materializer', '2020-08-13T10:30:00Z') ON CONFLICT DO NOTHING ",
    "prepared": false,
    "error": {
        "exec_status": "FatalError",
        "hint": null,
        "message": "null value in column \"id\" violates not-null constraint",
        "status_code": "23502",
        "description": "Failing row contains (null, event_materializer, 2020-08-12 18:00:00+00, scheduled, 0, 2020-08-12 17:52:16.068624, null)."
    },
    "arguments": []
}
Trey Stout
  • 6,231
  • 3
  • 24
  • 27
  • 1
    Seems like you may have been hitting this: https://github.com/hasura/graphql-engine/issues/4009 . – tselvan Aug 13 '20 at 03:56
  • Indeed 4009 came up from a dev on github. This was solved by going back and retroactively fixing a migration that dropped the public schema in postgres. Once that was cleared up everything started working again. Link for posterity https://github.com/hasura/graphql-engine/issues/5573 – Trey Stout Aug 13 '20 at 20:22

1 Answers1

2

root cause was a migration at one point on this database that did

DROP SCHEMA public CASCADE;

The cascade saw there were tables in a non-public schema that used functions from pycrypto. Since pycrypto was installed in the public schema it dropped the function, since hasura's schema uses those functions as column defaults, it actually modified the DDL of hasura's tables to remove the DEFAULT value of columns.

Truly insidious, many hours wasted chasing it down. Hi from the past!

https://github.com/hasura/graphql-engine/issues/5573

Trey Stout
  • 6,231
  • 3
  • 24
  • 27