[CT-1393] Macro in BigQuery takes no keyword argument 'relation'

[Larchenko-Ksenia]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Have created 2 python models and 2 sql simple models, dbt version == 1.3.0.
When trying to launch "dbt compile --profiles-dir C:\Users\path_to_profile_file" or "dbt run" with profiles dir got the following:

Runtime Error
  Compilation Error in operation project_name_-on-run-start-0 (.\dbt_project.yml)
    macro 'dbt_macro__create_schema' takes no keyword argument 'relation'

    > in macro create_udfs (macros\bigquery_udfs\create_udfs.sql)
    > called by macro create_schema (macros\bigquery_udfs\create_schema.sql)
    > called by operation project_name_-on-run-start-0 (.\dbt_project.yml)

Expected Behavior

Successful models running and compilation within dbt=1.3.0.
The SQL models were launched within dbt version == 1.0.0, and there vere no issues, as usual.

dbt run -m stg_sql_model
   Running with dbt=1.0.0

Steps To Reproduce

1. Upgrade DBT version, 1.3.0
2. Create python and sql models in 'models' folder
3. Have local "profiles.yml" in .dbt folder
4. Compile or run models setting --profiles-dir

Relevant log output

(dbt-env) C:\Users\User_name\project_name>dbt compile --profiles-dir C:\Users\User_name\.dbt              
12:11:53  Running with dbt=1.3.0
12:11:53  Found _ models, _ tests, _ snapshots, _ analyses, _ macros, _ operation, _ seed files, _ sources, _ exposures, _ metrics
12:11:53  
12:11:54  Concurrency: 4 threads (target='dev')
12:11:54  
12:11:56  Encountered an error:
Runtime Error
  Compilation Error in operation project_name-on-run-start-0 (.\dbt_project.yml)
    macro 'dbt_macro__create_schema' takes no keyword argument 'relation'

    > in macro create_udfs (macros\bigquery_udfs\create_udfs.sql)
    > called by macro create_schema (macros\bigquery_udfs\create_schema.sql)
    > called by operation project_name--on-run-start-0 (.\dbt_project.yml)

Environment

- OS: Windows 10
- Python: 3.9.13
- dbt: 1.3.0

Which database adapter are you using with dbt?

bigquery

Additional Context

Macro create_udfs

{% macro create_udfs() %}

{% do adapter.create_schema(api.Relation.create(database=target.database, schema="user_defined_functions")) %}

{{f_beauty_string_source()}}
{{f_process_json_source_proxy()}}
{{f_decode_uri_component()}}

{% endmacro %}

[lostmygithubaccount]

hi @Larchenko-Ksenia, I wasn't able to reproduce this -- are you thinking that this is related to Python models, the --profiles-dir option, or something else? can you provide more details why?

I copied over the macro to macros/create_udfs.sql, but had to comment out the custom macros:

{% macro create_udfs() %}

{% do adapter.create_schema(api.Relation.create(database=target.database, schema="user_defined_functions")) %}

{#
{{f_beauty_string_source()}}
{{f_process_json_source_proxy()}}
{{f_decode_uri_component()}}
#}

{% endmacro %}

I created a couple .sql and .py models with effectively select 1 as id statements. I added in dbt_projects.yml:

on-run-start:
  - "{{ create_udfs() }}"

for good measure, I also called the macro from one of the SQL models. in all cases, with or without --profiles-dir profiles/, where I'd copied over my ~/.dbt/profiles.yml file, everything ran successfully on 1.3 using BigQuery and Dataproc Serverless for the Python models

are there any more details you can provide to help us investigate and ideally reproduce?

[dataders]

What's the output of dbt --version

[jeremyyeo]

Unable to reproduce myself.

[Larchenko-Ksenia]

Hello, @lostmygithubaccount, @jtcohen6, @jeremyyeo, @dataders,
Thanks for your reply!
I see some additional details are needed to be provided to reproduce the issue above. Currently I'm trying to figure them out. Will answer as soon as possible.

[lostmygithubaccount]

thanks! will throw the awaiting_response label back on

[lostmygithubaccount]

@Larchenko-Ksenia from @jtcohen6 internally -- summarizing, the create_schema macro was added in v1.3 for BigQuery so this might be a naming collision:

Ah - could be related to the fact that, in v1.3 for the first time, dbt-bigquery will actually use/respect a custom create_schema macro (previously dbt created schemas via bq client API, and skipped Jinja-SQL entirely): dbt-labs/dbt-bigquery#183

So the fact is, their custom version of the create_schema macro takes a keyword argument named schema_name, whereas the built-in version takes a keyword argument named relation. And there must be places in the codebase where we pass in the keyword argument by its expected name (relation)

let us know if that's the issue

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[alfredgunnar]

I'm experiencing the same issue with the exact same use case (creating UDFs in BigQuery). Although, I'm running version 1.4.4 of dbt-core. Did you solve this, @Larchenko-Ksenia?

I found my issue.

I think both me and @Larchenko-Ksenia visited this thread and copy pasted their solution with a macro called create_schema defined as

-- macros/create_schema.sql

{% macro create_schema (schema_name) %}

{% do adapter.create_schema(target.project, schema_name) %}

{% endmacro %}

It seems like this is where the collision happened. Removing the create_schema.sql file and instead calling do adapter.create_schema(target.project, schema_name) directly from the create_udfs macro solved it for me.