Add --target and --profile to global config

[barton996]

resolves #7798
Related to pull/7920

Problem

--profile and --target are not part of the global config and cannot be set once at env var level without using env_var() in profiles.yml. This feature should be provided out of the box as is the case for --profiles-dir.

Solution

--profile and --target have been elevated to global config and can now be set by the DBT_PROFILE and DBT_TARGET env vars.

Checklist

• I have read the contributing guide and understand what's expected of me
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
• This PR includes type annotations for new and modified functions

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[barton996]

@dbeatty10

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.02%. Comparing base (2c1926c) to head (873af90).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9081      +/-   ##
==========================================
- Coverage   88.11%   88.02%   -0.09%     
==========================================
  Files         178      178              
  Lines       22487    22461      -26     
==========================================
- Hits        19815    19772      -43     
- Misses       2672     2689      +17     

Flag	Coverage Δ
integration	85.41% <100.00%> (-0.16%)	⬇️
unit	61.73% <100.00%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dbeatty10]

Question

source

@jtcohen6 in the vein of #7036, does the following need to be added to UserConfig?

    profile: Optional[str] = None
    target: Optional[str] = None

Answer

Outside of GitHub, @jtcohen6 and I discussed this, and he replied with the following:

I think it would be confusing to set those in UserConfig, given that:

• profile is already set in dbt_project.yml
• default target is already set within each profile

Decision

In light of those considerations, we're going to add profile and target the list of exceptions of global flags not included within UserConfig.

[jaklan]

Any updates on that?

In docs I can see:

but it doesn't seem to work, especially the type path is pretty confusing here.

[dbeatty10]

Any updates on that?

it doesn't seem to work, especially the type path is pretty confusing here.

Thanks for calling this out @jaklan -- you are right, these docs will need to be updated until this PR is merged+released.

I just opened this issue to update the docs in the meantime: dbt-labs/docs.getdbt.com#5111

[dbeatty10]

Approved from a product and user experience perspective

[FishtownBuildBot]

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#5134

[yu-iskw]

@barton996 @dbeatty10 I have a question about the behavior of the environment variables, though the case might be an edge case. If we set DBT_TAGET and --target at the same time, DBT_TAGET is used isn't it? Personally, it would sound natural that --target can have higher priority.

$ DBT_TARGET=no_such_target dbt build --target dev 

02:46:21  Running with dbt=1.8.0
02:46:21  Encountered an error:
Runtime Error
  The profile 'jaffle_shop' does not have a target named 'no_such_target'. The valid target names for this profile are:
   - dev
   - prod

[dbeatty10]

@yu-iskw It is supposed to use the following precedence order:

Global config precedence

dbt will pick the config in the following order (lower takes priority):

1. user config
2. environment variable
3. CLI flag

i.e., if all three are provided, then the CLI flag takes precedence.

If this is not the case for target, would you be willing to open a bug report ?