Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment collection replace API strips equals sign at the end #197603

Closed
Tracked by #22879
xEverth opened this issue Oct 29, 2023 · 16 comments · Fixed by #207889 or #226980
Closed
Tracked by #22879

Environment collection replace API strips equals sign at the end #197603

xEverth opened this issue Oct 29, 2023 · 16 comments · Fixed by #207889 or #226980
Assignees
Labels
bug Issue identified by VS Code Team member as probable bug terminal Integrated terminal issues terminal-shell-bash An issue in the terminal specific to bash terminal-shell-fish An issue in the terminal specific to fish terminal-shell-integration Shell integration, command decorations, etc. terminal-shell-pwsh An issue in the terminal specific to PowerShell verified Verification succeeded

Comments

@xEverth
Copy link

xEverth commented Oct 29, 2023

VS Code version: Code - Insiders 1.84.0-insider (35419fc, 2023-10-27T10:41:08.734Z)
OS version: Windows_NT x64 10.0.19045

Try applying these variables using the environment collection API:

TEST_KEY="SOMETHING"
TEST_KEY_2="SOMETHING="
TEST_KEY_3=SOMETHING=
TEST_KEY_4='SOMETHING='

Even though screenshot says they're applied as expected:

image

equal sign is stripped from the end when printing the value:

image

Original bug report


Type: <b>Bug</b>

<!-- Please fill in all XXX markers -->
# Behaviour
## Expected vs. Actual

Expected:
Environment keys are read correctly, both in powershell and in python

Actual
When the .env file is set in the Python extension configuration, if the keys have equals sign they are stripped out, whether they are quoted or not. This happens both if the .env file is the default ${workspaceFolder}/.env or if this is changed. When the environments variables are loaded by the extension, this may change the behaviour of programs which use os.getenv("KEY") to retrieve the values.

## Steps to reproduce:

1. Have a .env file with the following content:

TEST_KEY="SOMETHING"
TEST_KEY_2="SOMETHING="
TEST_KEY_3=SOMETHING=
TEST_KEY_4='SOMETHING='

2) Set the path to this .env file in the Python extension settings (Python:Env File)

3) Open a new terminal (Powershell/Windows)

4) Check the value of the environment variables with $env:PATH

5) See that the '=' character at the end is stripped

6) Open a python interpreter and attempt loading the environment file again (python-dotenv)

7) The equal signs are still stripped

<!--
**After** creating the issue on GitHub, you can add screenshots and GIFs of what is happening. Consider tools like https://www.cockos.com/licecap/, https://github.com/phw/peek or https://www.screentogif.com/ for GIF creation.
-->

<!-- **NOTE**: Everything below except Python output panel is auto-generated; no editing required. Please do provide Python output panel. -->
# Diagnostic data

-   Python version (& distribution if applicable, e.g. Anaconda): 3.11.0
-   Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
-   Value of the `python.languageServer` setting: Pylance

<details>

<summary>Output for <code>Python</code> in the <code>Output</code> panel (<code>View</code>→<code>Output</code>, change the drop-down the upper-right of the <code>Output</code> panel to <code>Python</code>)
</summary>

<p>

From a python interpreter

from dotenv import load_dotenv; import os
with open(".env", "r") as f:
f.readlines()
os.getenv("TEST_KEY")
os.getenv("TEST_KEY_2")
os.getenv("TEST_KEY_3")
os.getenv("TEST_KEY_4")

This may print out SOMETHING in all 4 cases, without the = sign

Repeat after deactivating the environment variable loading from the Python extension and see the = sign for TEST_KEY_2, TEST_KEY_3 and TEST_KEY_4


</p>
</details>

<details>

<summary>User Settings</summary>

<p>

envFile: ""

venvPath: ""

venvFolders: ""

languageServer: "Pylance"

testing
• unittestEnabled: true

experiments
• optInto: ["pythonTerminalEnvVarActivation"]


</p>
</details>


Extension version: 2023.18.0
VS Code version: Code - Insiders 1.84.0-insider (35419fc016b8d3d5dbc59aa11a0be957b0897309, 2023-10-27T10:41:08.734Z)
OS version: Windows_NT x64 10.0.19045
Modes:

<details>
<summary>System Info</summary>

|Item|Value|
|---|---|
|CPUs|Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz (16 x 2304)|
|GPU Status|2d_canvas: enabled<br>canvas_oop_rasterization: enabled_on<br>direct_rendering_display_compositor: disabled_off_ok<br>gpu_compositing: enabled<br>multiple_raster_threads: enabled_on<br>opengl: enabled_on<br>rasterization: enabled<br>raw_draw: disabled_off_ok<br>video_decode: enabled<br>video_encode: enabled<br>vulkan: disabled_off<br>webgl: enabled<br>webgl2: enabled<br>webgpu: enabled|
|Load (avg)|undefined|
|Memory (System)|31.84GB (16.35GB free)|
|Process Argv|--log trace|
|Screen Reader|no|
|VM|0%|
</details>
<!-- generated by issue reporter -->

@xEverth

This comment was marked as resolved.

@karrtikr
Copy link
Contributor

karrtikr commented Oct 30, 2023

Thanks for the bug report! We investigate issues in order based on priority and severity, which includes the impact it has on your ability to use the extension to do productive work, and the number of people affected. If other users come forward and leave a comment demonstrating they are seeing/reproducing the problem then we will raise this issue's priority. Thanks for understanding and the patience!

@xEverth

This comment was marked as resolved.

@karrtikr
Copy link
Contributor

karrtikr commented Nov 7, 2023

Thanks, I'm able to reproduce it. Looks like an issue with environment collection API in VS Code itself, hence transferring.

@karrtikr karrtikr removed their assignment Nov 7, 2023
@karrtikr karrtikr transferred this issue from microsoft/vscode-python Nov 7, 2023
@karrtikr karrtikr changed the title Reading .env file through Python extension strips equals sign at the end Environment collection replace API strips equals sign at the end Nov 7, 2023
@DonJayamanne DonJayamanne assigned Tyriar and unassigned DonJayamanne Nov 7, 2023
@joyceerhl joyceerhl added bug Issue identified by VS Code Team member as probable bug terminal Integrated terminal issues labels Dec 14, 2023
@Tyriar
Copy link
Member

Tyriar commented Dec 15, 2023

This makes sense as we currently split on = and : so those characters won't work currently when applied via shell integration:

For pwsh:

if ($env:VSCODE_ENV_REPLACE) {
$Split = $env:VSCODE_ENV_REPLACE.Split(":")
foreach ($Item in $Split) {
$Inner = $Item.Split('=')
[Environment]::SetEnvironmentVariable($Inner[0], $Inner[1].Replace('\x3a', ':'))
}
$env:VSCODE_ENV_REPLACE = $null
}
if ($env:VSCODE_ENV_PREPEND) {
$Split = $env:VSCODE_ENV_PREPEND.Split(":")
foreach ($Item in $Split) {
$Inner = $Item.Split('=')
[Environment]::SetEnvironmentVariable($Inner[0], $Inner[1].Replace('\x3a', ':') + [Environment]::GetEnvironmentVariable($Inner[0]))
}
$env:VSCODE_ENV_PREPEND = $null
}
if ($env:VSCODE_ENV_APPEND) {
$Split = $env:VSCODE_ENV_APPEND.Split(":")
foreach ($Item in $Split) {
$Inner = $Item.Split('=')
[Environment]::SetEnvironmentVariable($Inner[0], [Environment]::GetEnvironmentVariable($Inner[0]) + $Inner[1].Replace('\x3a', ':'))
}
$env:VSCODE_ENV_APPEND = $null
}

@Tyriar Tyriar added the help wanted Issues identified as good community contribution opportunities label Dec 15, 2023
@Tyriar Tyriar added this to the Backlog milestone Dec 15, 2023
@upascal
Copy link

upascal commented Jan 4, 2024

I have this issue too. I noticed if I run my backend server (which takes a secret key from the .env file) in my system terminal it behaves as expected, but if I run it in the vscode terminal it truncates the key after the equals sign in the key. This broke my backend! Please fix this. Thanks.

@babiyvadim
Copy link

I face this issue too. Waiting for fixing. Thanks
As a potential workaround, you can run the command source .env directly from the terminal after every .env file changes for each terminal window

@maximiliank
Copy link

I am having the same issue when the project is setup for a C++ python extension with leak sanitizers. This requires to LD_PRELOAD the sanitizer library libasan.so and disable leak detection via ASAN_OPTIONS=detect_leaks=0. Unfortunately, in the terminal only ASAN_OPTIONS=detect_leaks remains which yields to a unusable terminal due to

bash: export: `=': not a valid identifier
AddressSanitizer: ERROR: expected '=' in ASAN_OPTIONS

See microsoft/vscode-python#22792 (comment)

@whusterj
Copy link

I am also experiencing this issue with a Django Secret key containing an '=' sign and would appreciate a fix. I can work around it, but it's not ideal since this runs counter to the expected behavior and how the source command behaves. Thank you.

@N3xus8
Copy link

N3xus8 commented Feb 28, 2024

This so frustrating. I had my code working perfectly I was testing the whole day and then suddenly it broke. I spent hours figuring it out.
I managed to debug it and found that the == signs in the .env variables were stripped out.
I was so confused why did it work earlier and then I realised that a new version of VS Code was installed today.
Did it break my code?Not 100% sure .
Anyway VS Code is awesome. But the lesson of the day : stable environment . Don't update it constantly.

@jaywfoster
Copy link

I'm facing the same issue. Here are some potential clues based on my project: it only occurs in the init constructor for a class. The class is located in a .py that is in a different folder than the .env. The error occurs when the code is invoked from an api endpoint but works fine otherwise. I tried making adjustments by setting the .env path explicitly but no luck. Please help.

@Dnny44
Copy link

Dnny44 commented Mar 15, 2024

I am also experiencing this issue with a Django Secret key containing an '=' sign and would appreciate a fix. I can work around it, but it's not ideal since this runs counter to the expected behavior and how the source command behaves. Thank you.

Can confirm that I am also seeing this behavior in the exact same situation.

VSCode Terminal - drops "="
image

Plain Terminal - returns correct
image

@Tyriar Tyriar modified the milestones: Backlog, March 2024 Mar 15, 2024
@Tyriar Tyriar added terminal-shell-integration Shell integration, command decorations, etc. terminal-shell-bash An issue in the terminal specific to bash terminal-shell-pwsh An issue in the terminal specific to PowerShell labels Mar 15, 2024
@VSCodeTriageBot VSCodeTriageBot added the unreleased Patch has not yet been released in VS Code Insiders label Mar 15, 2024
@VSCodeTriageBot VSCodeTriageBot added insiders-released Patch has been released in VS Code Insiders and removed unreleased Patch has not yet been released in VS Code Insiders labels Mar 18, 2024
@TylerLeonhardt TylerLeonhardt added the verification-steps-needed Steps to verify are needed for verification label Mar 27, 2024
@Tyriar
Copy link
Member

Tyriar commented Mar 27, 2024

Verification steps:

  • Create an extension that does: context.environmentVariableCollection.replace('TEST', 'FOO=')
  • Debug the extension
  • Activate it
  • Open a powershell terminal
  • Run $env:TEST, ensure that it prints FOO=
  • Repeat the same in a bash terminal but run echo $TEST

@Tyriar Tyriar removed the verification-steps-needed Steps to verify are needed for verification label Mar 27, 2024
@amunger amunger added the verified Verification succeeded label Mar 27, 2024
@microsoft microsoft locked and limited conversation to collaborators Jun 11, 2024
@Tyriar Tyriar reopened this Sep 1, 2024
@vs-code-engineering vs-code-engineering bot removed the insiders-released Patch has been released in VS Code Insiders label Sep 1, 2024
@Tyriar
Copy link
Member

Tyriar commented Sep 1, 2024

This still had an issue for fish shell which should be fixed in #226980

To verify, repeat steps above but for fish.

@Tyriar Tyriar modified the milestones: March 2024, September 2024 Sep 1, 2024
@Tyriar Tyriar added terminal-shell-fish An issue in the terminal specific to fish and removed help wanted Issues identified as good community contribution opportunities verified Verification succeeded labels Sep 1, 2024
@connor4312 connor4312 added the verified Verification succeeded label Sep 25, 2024
@connor4312
Copy link
Member

verified bash/fish/zsh

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Issue identified by VS Code Team member as probable bug terminal Integrated terminal issues terminal-shell-bash An issue in the terminal specific to bash terminal-shell-fish An issue in the terminal specific to fish terminal-shell-integration Shell integration, command decorations, etc. terminal-shell-pwsh An issue in the terminal specific to PowerShell verified Verification succeeded
Projects
None yet