1

In DBT, whenever we deploy the models, the database name gets prefixed to each deployed model in the sql definition in database.

I need to configure the dbt project in a way that it doesn't prefix database name to the deployed models.

user961
  • 453
  • 6
  • 20

2 Answers2

0

Do you mean that:

  1. You don't want the schema name with a prefix added to it, like just be finance.modelname instead of dbname_finance.modelname, or
  2. you want the relation name to be rendered with a two-part name (schema.modelname) instead of the three-part name (database.schema.modelname)?

If #1, I recommend you read the entire custom schema names docs page, specifically the part about Advanced custom schema configuration

If it's #2, this is a change required at the adapter level. Since you've tagged synapse, I'd wager a guess that you're using Synapse SQL Serverless Pools because I have also encountered the fact that you can't use three-part names in Serverless pools. Last week, I actually made dbt-synapse-serverless a separate adapter from dbt-synapse which in fact disables the three-part name.

Anders Swanson
  • 3,637
  • 1
  • 18
  • 43
  • Thank you so much for your response! It is the #2 scenario that I am talking about in my question. Yes I am using Azure synapse, I am not sure if it is serverless or not , will have to check for that. Will keep you posted on my findings. – user961 Oct 09 '21 at 17:45
  • 1
    This is not working as we are using dedicated SQL pool. Is there any other solution I can follow to solve this problem for dedicated SQL pool in Azure Synapse. – user961 Oct 11 '21 at 07:08
  • @user961 can you please provide an example of the error you get? This is news to me that three-part names not being supported in dedicated pool. Are you trying to work on the master db? – Anders Swanson Oct 11 '21 at 20:30
  • I think there is some confusion. So I am already getting the three-part names in the dedicated pool but I want the two part names convention in the dedicated pools. Can you please guide me on how to achieve that i.e. two part naming convention in dedicated pool when deploying using DBT using dbt run command. We want the models to be deployed as schemaname.modelname – user961 Oct 12 '21 at 07:19
  • Can you suggest me some solution/workaround for the scenario I mentioned in the above comment. Thanks – user961 Oct 14 '21 at 04:33
  • Headed to bed now. But I think I can make a new branch of dbt-synapse that might help you tmw. Stay tuned – Anders Swanson Oct 14 '21 at 04:35
  • sure..looking forward to it.. – user961 Oct 14 '21 at 07:53
  • ok can you try installing [this branch](https://github.com/dbt-msft/dbt-synapse/pull/66) using `pip install git+https://github.com/dbt-msft/dbt-synapse.git@no_dbname` and let me know if that fixes things? – Anders Swanson Oct 14 '21 at 16:29
  • This is not helping either. This one is also rendering the model with three part naming convention. can we try something else? – user961 Oct 18 '21 at 09:27
  • Also, I wanted to understand that there is not much documentation around DBT, and I mostly have to clarify my doubts or put my questions on stackoverflow. Is it not a widely used tool? This process makes me highly dependent on stackoverflow to be able to proceed on my work. – user961 Oct 20 '21 at 05:28
  • 1. let's move this discussion over to the relevant [GitHub Issue](https://github.com/dbt-msft/dbt-synapse/issues/65) 2. the vast majority of product support for dbt happens in the [dbt slack](https://www.getdbt.com/community/join-the-community). you should join! I already have a thread there w/ your coworker Avi. 3. dbt is widely used (outside of MSFT) but less so at w/ MSFT dbs 4. dbt-synapse in particular is an open-source plugin largely maintained by me and my team. 5. your ask is new enhancement that isn't supported today. I'm on vacation right now, but looking into it for you anyway. – Anders Swanson Oct 20 '21 at 22:55
  • Thank you very much for your efforts on taking up this enhancement just for us. This really means a lot and I am looking forward to it. Have a great vacation till then!! – user961 Oct 21 '21 at 03:31
0

You can overwrite the built-in ref macro. This macro returns a Relation object, so we can manipulate its output like this:

{% macro ref(model_name) %}

    {% do return(builtins.ref(model_name).include(database=false)) %}

{% endmacro %}

So, from there, all models that use the ref function will return the Relation object without the database specification.

dbt code:
select * from {{ ref('model') }}

compiled code:
select * from schema_name.model

EDIT:

As you requested, here's an example to remove the database name from the sources:

{% macro source(source_name, table_name) %}

    {% do return(builtins.source(source_name, table_name).include(database=false)) %}

{% endmacro %}

I've worked with sources from different databases, so if you ever get to that case, you might want to edit the macro to offer an option to include the database name, for example:

{% macro source(source_name, table_name, include_database = False) %}

    {% do return(builtins.source(source_name, table_name).include(database = include_database)) %}

{% endmacro %}
dbt code:
select * from {{ source('kaggle_aps', 'coaches') }}
select * from {{ source('kaggle_aps', 'coaches', include_database = True) }}

compiled code:
select * from schema_name.object_name
select * from database_name.schema_name.object_name

More details can be found in the official documentation

gasscoelho
  • 562
  • 5
  • 8
  • while this is helpful in removing the database name from the generated models, but I am still seeing this three part naming convention with the sources. Is it possible to remove the database name from the sources as well. – user961 Nov 10 '21 at 09:21
  • yeap, you can do the same using the `builtins.source`. I'll edit my answer with the code snippet. – gasscoelho Nov 11 '21 at 01:45
  • [A comment was just posted as answer](https://stackoverflow.com/a/76970129/5211833): "While this works for removing database_name from the models and the sources as well, there is still a problem left. While running the tests using the dbt test command, the default database_name ("" in my case) still creates a problem leading to PARSE_SYNTAX_ERROR. The table name appears like this - .schema_name.table_name Error: [PARSE_SYNTAX_ERROR] Syntax error at or near '.': extra input '.'" – Adriaan Aug 24 '23 at 14:14