2

We are using dbt to manage our data models in MSSQL, and we are considering moving to PostgreSQL Citus.

I'm not clear on how I could use Citus's features via dbt? I'm thinking specifically of data compression and columnar storage.

Any ideas?

MYK
  • 1,988
  • 7
  • 30

2 Answers2

1

TBH, I don't have any experiences with dbt.

However, given that dbt docs state that they support PostgreSQL, and Citus Columnar is a table-access-method level abstraction that doesn't require any changes to the queries(*), I wouldn't expect any compatibility issues when combining both.
My guess is that after creating your tables with USING columnar option, then you can just follow dbt docs for the rest.

(*): You might still want to have a look into that page to see what Citus Columnar does or doesn't support atm.
https://github.com/citusdata/citus/tree/master/src/backend/columnar#introduction

onurctirtir
  • 200
  • 5
1

was wondering if you had any updates on how this process has gone for you?

I'm looking to implement DBT at my company and was considering postgres/citus as well.

I believe you could use a combination of pre-hook/post-hook dbt functions and materializing incrementally instead of the table or view method.

Thinking is either pre-hook create columnar table and use the dbt incremental update, or materialize as view/table then post-hook SELECT alter_table_set_access_method('table_name', 'columnar');

MKlinck
  • 11
  • 1
  • Hi. I stuck with `SQL Server`; so no updates on this. I've been using https://github.com/dbt-msft/dbt-sqlserver and their pre/post hooks to manage indexes/columnar features in `MSSQL` – MYK Jan 19 '22 at 21:40
  • 1
    As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 20 '22 at 00:13