Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[env_vars] [false positives] environment-aware logic for sources should not cause sources to always be marked as modified #9573

Closed
Tracked by #9562
graciegoheen opened this issue Feb 14, 2024 · 2 comments · Fixed by #10675
Assignees
Labels
enhancement New feature or request state: modified state Stateful selection (state:modified, defer)

Comments

@graciegoheen
Copy link
Contributor

graciegoheen commented Feb 14, 2024

To support this pattern with sources (and probably other "yaml-only" resource types), I believe we'd need to also do the work of treating database + schema + identifier/alias as configs, such that we can save their unrendered values when configs are calculated. In theory, that's happening here, but it doesn't work as I'd expect for the one actual source config we do support today, enabled.

Original issue for this:

Originally posted by @jtcohen6 in #9563 (comment)

I want to point to a different locations for my sources depending on the environment I am in (different database for prod vs. dev). To do so, I configure my sources as follows:

sources:
  - name: jaffle_shop
    database: "{{ env_var('DBT_DATABASE'}}"
    tables:
      - name: customers

But, this counts as a modified change everytime I do --select state:modified comparing two different environments.

Instead, I tried putting this logic in dbt_project.yml:

sources:
  project:
    +database: ...

and that did not work, as it seems the only configuration that is available to sources in the dbt_project.yml is enabled - docs here.

Acceptance Criteria

  • I want to use env_vars in my configuration, and the resource should not always be marked as modified

We should open a separate ticket to handle env_vars for other resources

Relevant to #4304

@graciegoheen graciegoheen changed the title environment-aware logic for sources should not cause sources to always be marked as modified [vars] environment-aware logic for sources should not cause sources to always be marked as modified Feb 14, 2024
@graciegoheen graciegoheen changed the title [vars] environment-aware logic for sources should not cause sources to always be marked as modified environment-aware logic for sources should not cause sources to always be marked as modified Feb 14, 2024
@graciegoheen graciegoheen changed the title environment-aware logic for sources should not cause sources to always be marked as modified [env_vars] environment-aware logic for sources should not cause sources to always be marked as modified Feb 14, 2024
@graciegoheen graciegoheen added enhancement New feature or request state Stateful selection (state:modified, defer) state: modified labels Feb 14, 2024
@graciegoheen graciegoheen changed the title [env_vars] environment-aware logic for sources should not cause sources to always be marked as modified [env_vars] [false positives] environment-aware logic for sources should not cause sources to always be marked as modified Feb 14, 2024
@ashleemtib
Copy link

Hi there, just curious if this is slotted for work any time soon?

@graciegoheen
Copy link
Contributor Author

Hi @ashleemtib, yes! Currently, we're planning to tackle this work for 1.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request state: modified state Stateful selection (state:modified, defer)
Projects
None yet
3 participants