pixi shell-hook without checking

[gshiba]

Using direnv with eval "$(pixi shell-hook)" in .envrc is great because "things just work" when working inside the repo, except it takes ~7secs for our large project just to cd into the directory. (feels even worse when you're in the habit of opening many bash sessions in tmux)

Requests:

• run pixi shell-hook without the "check" (--skip-check flag?, probably applies to other pixi commands too)
• a command to explicitly check the env

I'm assuming the "check current env against pixi.toml" step is the time consuming part.
My pixi.lock is ~800KB or ~24k lines.

In order to ensure up-to-date env (thinking out loud):

# .envrc
watch_file .pixi.lock

# if hash of .pixi.lock does not match .pixi.lock.hash
#   - echo "your env needs updating; this may take a few seconds."
#   - run pixi install
#   - update .pixi.lock.hash

eval "$(pixi shell-hook --skip-check)"

Alternatives considered:

• Instead of using pixi shell-hook, I considered just doing path_add .pixi/envs/default/bin in .envrc, however I found some tools needed the whole env exported to work.

[pavelzw]

Isn't pixi shell-hook --frozen exactly what you are looking for?
The part that's taking so long should be the re-solving, the deserialization of pixi.lock should be fairly quick.

❯ du -h pixi.lock
908K	pixi.lock
❯ time pixi shell-hook --frozen
export ...

________________________________________________________
Executed in   82.32 millis    fish           external
   usr time   58.75 millis   60.00 micros   58.69 millis
   sys time   26.65 millis  464.00 micros   26.19 millis

--frozen takes the lockfile as it is and doesn't try to update it from pixi.toml.

Instead of using pixi shell-hook, I considered just doing path_add .pixi/envs/default/bin in .envrc, however I found some tools needed the whole env exported to work.

i would recommend against this for exact these reasons

[gshiba]

No, only conda packages.

https://gist.github.com/gshiba/8c8621e8e0b8b556d4916a593f871572

shortened version:

[project]
name = "try_pixi"
channels = ["conda-forge", "bioconda"]
platforms = ["linux-64", "osx-64"]

[tasks]
bootstrap_init = "bash tools/bin/bootstrap_init"

[dependencies]
# About 70 packages

[pypi-dependencies]
# Nothing here (but table name exists as a placeholder)

[target.linux-64.dependencies]
# About 10 packagse

[environments]
# Nothing here (but table name exists as a placeholder)

[system-requirements]
linux = "4.15.0"

[pavelzw]

that's a huge environment 🤯

i'm wondering how pixi checks whether the environment is up to date with the lockfile. @ruben-arts could you shed some light on this?

❯ pixi list | wc -l
     748

i would expect checking if 748 files in .pixi/envs/default/conda-meta exist shouldn't take 3s, right?
If it actually takes that long, it might make sense to add an additional file in conda-meta that contains all currently installed packages as a list for faster checks?

[gshiba]

(is it huge? it doesn't feel huge because pixi/rattler is so quick 🎉 😁 )

[ruben-arts]

It does some checksum checking I believe. Possibly a big file somewhere in the list of dependencies. I can't try it out right now. Maybe anyone else can debug the files for you?

[gshiba]

Are the jsons being checksum'ed?

$ ls -lSh ./.pixi/envs/default/conda-meta/ | head
total 41M
-rw-r--r-- 1 ... Users 5.1M Jun 21 16:30 r-bh-1.84.0_0-r43hc72bb7e_0.json
-rw-r--r-- 1 ... Users 3.5M Jun 21 16:30 awscli-2.16.6-py310h2ec42d9_0.json
-rw-r--r-- 1 ... Users 2.2M Jun 21 16:30 libdb-6.2.32-he49afe7_0.json
-rw-r--r-- 1 ... Users 1.3M Jun 21 16:30 pandas-2.2.2-py310hbf2a7f0_1.json
-rw-r--r-- 1 ... Users 1.1M Jun 21 16:30 statsmodels-0.14.2-py310hde789be_0.json
-rw-r--r-- 1 ... Users 1.1M Jun 21 16:30 ncurses-6.5-h5846eda_0.json
-rw-r--r-- 1 ... Users 884K Jun 21 16:30 perl-5.32.1-7_h10d778d_perl5.json
-rw-r--r-- 1 ... Users 880K Jun 21 16:30 nodejs-20.12.2-hfc0f20e_0.json
-rw-r--r-- 1 ... Users 867K Jun 21 16:30 jedi-0.19.1-pyhd8ed1ab_0.json