Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in base_source_columns #465

Closed
1 of 6 tasks
nam-97 opened this issue Jun 24, 2024 · 4 comments · Fixed by #468
Closed
1 of 6 tasks

Error in base_source_columns #465

nam-97 opened this issue Jun 24, 2024 · 4 comments · Fixed by #468
Labels
bug Something isn't working

Comments

@nam-97
Copy link

nam-97 commented Jun 24, 2024

Describe the bug

When running the model base_source_columns by dbt run, I encounter below error
image

Steps to reproduce

Expected results

Actual results

Screenshots and log output

System information

The contents of your packages.yml file:

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • trino/starburst
  • other (specify: ____________)

The output of dbt --version:

1.8.1

Additional context

The problem locates on the post-hook parameter in the model, since the query to create table still can be run in Snowflake. Probably, the post-hook query is trying to insert more columns than the ones existing in the output object
image

Are you interested in contributing the fix?

@nam-97 nam-97 added the bug Something isn't working label Jun 24, 2024
@b-per
Copy link
Collaborator

b-per commented Jun 24, 2024

Thanks for raising this.

Would you be able to check the dbt log file and/or Snowflake logs to find the exact statement that failed?

@kverburg
Copy link

kverburg commented Jul 1, 2024

Also got this error in BQ. Error is there: Inserted row has wrong column count; Has 8, expected 5 at [3:1]

Hereby the exact query the db got. Did trim it a bit, but the gest is still there.

/* {"app": "dbt", "dbt_version": "1.8.1", "profile_name": "dbt_project", "target_name": "test", "node_id": "model.dbt_project_evaluator.base_source_columns"} */
INSERT INTO
    `project`.`dataset`.`base_source_columns`
VALUES
    (
        '''source.dbt_project.dbt_source.source_table''',
        '''_airbyte_raw_id''',
        '''''',
        '''string''',
        '''[]''',
        FALSE,
        0,
        '''None'''
    ),
    ...
    (
        '''source.dbt_project.dbt_source.source_table''',
        '''_airbyte_extracted_at''',
        '''''',
        '''timestamp''',
        '''[]''',
        FALSE,
        0,
        '''None'''
    );

@b-per
Copy link
Collaborator

b-per commented Jul 1, 2024

Thanks a lot!

OK, so I can definitely see the error now. This is a bug in the code but we are actually not testing it in CI. I will add a new test and release a new patch version soon.

@b-per
Copy link
Collaborator

b-per commented Jul 1, 2024

0.12.1 should be available in a bit in the dbt Hub

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
3 participants