Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

star() macro now includes as alias by default, breaking expected behavior #469

Closed
1 of 5 tasks
foundinblank opened this issue Dec 21, 2021 · 3 comments · Fixed by #468
Closed
1 of 5 tasks

star() macro now includes as alias by default, breaking expected behavior #469

foundinblank opened this issue Dec 21, 2021 · 3 comments · Fixed by #468
Labels
bug Something isn't working good first issue

Comments

@foundinblank
Copy link
Contributor

foundinblank commented Dec 21, 2021

Describe the bug

In previous versions, the star() macro produced a comma-separated list of fields, such as

id,
date,
field

With #436 as part of 0.8.0 it now will produce a comma-separated list of fields and aliases, such that the output is

id as id,
date as date,
field as field

Steps to reproduce, expected results, actual results

We have a specific usage of a star() macro where

{% set upstream_model = 'model1' %}
{% set partition_columns = star(from=ref(model1)) %}

select
    *,
    first_value(category) over (partition by {{ partition_columns }}
        order by start_date desc) as past_category
from {{ ref(model1) }} 

Under dbt-utils 0.7.5 this is the generated output:

select
    *,
    first_value(category) over (partition by 
"FIELD1", 
"FIELD2", 
"FIELD3"
        order by start_date desc) as past_category
from staging.model1

Under dbt-utils 0.8.0 this is the generated output, which fails because we can't alias fields within an over() clause

select
    *,
    first_value(category) over (partition by 
"FIELD1" as "FIELD1", 
"FIELD2" as "FIELD2", 
"FIELD3" as "FIELD3"
        order by start_date desc) as past_category
from staging.model1

System information

The contents of your packages.yml file:

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 1.0.0
   latest version: 1.0.0

Up to date!

Plugins:
  - snowflake: 1.0.0

Are you interested in contributing the fix?

I would be, I think what it needs is flags to see if prefix and suffix exist and if neither exist, don't append the as clause. This is the specific line:

{%- if relation_alias %}{{ relation_alias }}.{% else %}{%- endif -%}{{ adapter.quote(col)|trim }} as {{ adapter.quote(prefix ~ col ~ suffix)|trim }}

So in psuedocode it'd be something like:

{%- if relation_alias %}
{{ relation_alias }}.{% else %}{%- endif -%}{{ adapter.quote(col)|trim }} 

--if prefix is not null or suffix is not null 
as {{ adapter.quote(prefix ~ col ~ suffix)|trim }} 

{%- if not loop.last %},{{ '\n  ' }}{% endif %}
@foundinblank foundinblank added bug Something isn't working triage labels Dec 21, 2021
@joellabes
Copy link
Contributor

joellabes commented Jan 4, 2022

Yes good call @foundinblank! Even if it wasn't breaking the partition, it's ugly to alias something to itself 😅 I should have thought about this in code review.

If/when you do have a crack at this, could I prevail upon you to add an integration test to the star macro that incorporates a window function or something similar, so that we can catch this sort of thing automatically in the future?

@joellabes
Copy link
Contributor

Just found #468 which looks like it solves this!

@foundinblank
Copy link
Contributor Author

🙌 great!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue
Projects
None yet
2 participants