0

I have models in dbt but I after dbt test, I want to trigger an API call which could kick-off my AWS glue job. Any suggestions for workaround on this?

Approaches tried:

  1. Dbt —> fal script (which runs python code) —> py script could call glue job Blocker: later found that they are not running Python on dbt Cloud, they are running it in Github actions

  2. Dbt —> Rest/POST API with on-run-end (hooker) —> glue job Blocker: can’t find any instance where API calls are done from dbt (we could call dbt jobs from API but not the other way around)

  3. Dbt new table —> SNS —> lambda fn —> glue job

Oj Sindher
  • 1
  • What are you using to orchestrate dbt? You could use that to add that as part of the workflow. Sounds like you are trying to use dbt for something it isnt meant to do. – Chris Albert Dec 07 '22 at 18:25
  • We need to know: 1. how are you deploying dbt? In dbt Cloud or are you running dbt Core on a server or as part of an orchestrator? 2. What RDBMS are you using? Redshift? – tconbeer Dec 07 '22 at 19:42
  • Yeah, I understand now that I would require an orchestration tool. Do you think Glue workflow can help? can't find any documentation for glue workflow with dbt – Oj Sindher Dec 07 '22 at 20:48
  • Its dbt cloud and redshift. – Oj Sindher Dec 07 '22 at 20:52

0 Answers0