(De)Duplication of nested environments #583
liquidcarbon
started this conversation in
Ideas
Replies: 1 comment 2 replies
-
Pixi already hardlinks files from a central cache. This means that unless you delete the cache in between runs the disk space is most likely already deduplicated between all environments on your machine. |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
There's a pattern I use with exploratory work in notebooks, where I'd have a root
pixi.toml
and then folders with their ownpixi.toml
.It's similar to how examples are set up here, or to how one would have a base conda and more custom stuff on top.
Is there a way / plans for Pixi to become smart around not installing the same dependencies into nested environments? Especially python itself. How about in the event there is the same python version in the parent folder, the inner
.pixi
folder creates symlinks to the outer python binaries, but then installs its own packages?In my workflows, I've got the first cell of Jupyter notebooks (which runs from the base pixi layer) doing the opposite - adding the inner layer to
sys.path
:Both pixi folders contain ~20000 files that are likely 99% identical.
What can we do?
Beta Was this translation helpful? Give feedback.
All reactions