[CT-2626] [Regression] dbt snapshot does not fetch the target path from environment variables

[ADraginda]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

dbt 1.5.1

• we don't have target_path in our profiles.yml
• we have the env var: 'DBT_TARGET_PATH': '/tmp/198e1313-1c7f-44d7-92a5-9556f5d203bd'
• but when we run dbt --debug snapshot --model <model_name> --target production we see:

running dbt with arguments {... 'target_path': 'None', ... }

Expected Behavior

I would expect the target_path set in the environment variable respected (Note that LOG_FILE_PATH is respected from the environment variables)

Steps To Reproduce

export DBT_TARGET_PATH="/tmp/198e1313-1c7f-44d7-92a5-9556f5d203bd"
dbt --debug snapshot --model <model_name>

Relevant log output

No response

Environment

- OS: Centos (AWS MWAA 2.4.3)
- Python: 3.10.10
- dbt: 1.5.1

Which database adapter are you using with dbt?

snowflake

Additional Context

No response

[dbeatty10]

Thanks for reaching out @ADraginda !

I tried out each of the configuration options here, and I observed something similar as you reported, but slightly different.

For the third option (target-path in dbt_project.yml), it didn't log out the target directory and printed None instead. As it seems you already know, this option is is deprecated in 1.5. Backward compatibility is still supported in 1.5 but will be removed in a future update.

But in all three cases, it appeared to create and use the target directory correctly (despite the --debug output for the deprecated option).

export DBT_TARGET_PATH=target_environment_variable                                                                                         
echo $DBT_TARGET_PATH
dbt --debug run -s my_model

Output:

target_environment_variable

21:31:53  Running with dbt=1.5.0
21:31:53  running dbt with arguments {... 'target_path': 'target_environment_variable', ...}

Is it not writing to the target directory specified in DBT_TARGET_PATH for you?

Even when /tmp/198e1313-1c7f-44d7-92a5-9556f5d203bd didn't already exist, it successfully wrote there on my machine.

[ADraginda]

Apologies - I have updated my ticket with the actual command I sent: It should have been dbt snapshot not dbt run. I just tested again and indeed, the snapshot was not using my environment variable DBT_TARGET_PATH (it reports as None, and does not create the folder, instead just creating /target in my current path)

[dbeatty10]

Ah, I see @ADraginda.

It looks like #7419 was accidentally not yet included in 1.5.0 or 1.5.1, so we'll need to backport it within an upcoming release. Once we backport and release it, I believe it will resolve the issue you are reporting.

[ADraginda]

Wonderful, thanks so much @dbeatty10 !

[dbeatty10]

The backport PR in #7826 will resolve this issue. But since it targets the 1.5.latest, branch rather than the default branch (main), it won't auto-close it afterwards (bummer!).

So I'm going to preemptively close this to avoid accidentally forgetting to do so later.