Adding documentation for persistent conda environments

[facusapienza21]

Hi ❄️!

I put a draft about how to create persistent environments in the CryoCloud Hub so people can 1) install new conda environments that are located in a specific environment folder and 2) create a new iPython kernel that can be used to run code in that environment directly from a Jupyter Notebook.

I did this in my fork of the CryoCloud website. I thought the right place for this was the Python Installation tab inside How Tos. I can expand this also a little bit more to include some of the descriptions we use in the Stat 159 course with @fperez, it may be useful to include some of the materials for git, GitHub, and Jupyter.

@tsnow03 I can give a last check to my changes (eg, grammar check and maybe some extra explanations) and then create a PR to the central repository, but I wanted first to check out with you if you find this useful and you want me to include/exclude some contents. Happy to help :)

Also: is there any way to test that the JupyterBook is build correctly at the moment of doing a PR? I just want to be sure that the format in the markdown file is compatible with the current JupyterBook version. I remember trying to make JB visible upon PR in the past but I couldn't make it work.

Hope it helps!

[wsauthoff]

Hey @facusapienza21, thanks so much for contributing to CryoCloud! This tutorial will be super useful for new and future users. I like the addition of mamba installation (perhaps add a comment on why mamba may be preferred over conda? e.g., faster Python package installs, enhance dependency conflict resolution?). I'm curious why you removed the section about installing older version of Python? I notice that content is integrated into your example of creating a persistent environment, but perhaps it'd useful to comment that you are setting up that environment to use a specific Python older version to make it clear that is a possibility? I also think it might be useful to add a brief markdown cell explaining why you would want to create a persistent environment towards the beginning of that section (e.g., so you don't need to re-install packages that are not in the base CryoCloud hub image each time you launch CryoCloud hub or to recreate an environment from a collaborator or from a data repo)

To answer your second question: yes, there is a way to preview the JupyterBook build detailed here. I need to edit this document to make it bit clearer, so here are the necessary steps to preview your local changes:

• in CryoCloud hub terminal:

cd PATH_TO/CryoCloudWebsite
jupyter-book build book

The terminal output will tell you the path to the JupyterBook preview build. From there you can navigate to the page you've edited (e.g., PATH_TO/CryoCloudWebsite/book/_build/html/how_tos/background/python.html) and double click to preview or you double click any preview html page (e.g., PATH_TO/CryoCloudWebsite/book/_build/html/index.html) and navigate within the html preview to the page you've edited.

[tsnow03]

This looks awesome @facusapienza21! I think the document looks pretty complete. Because this will be so important on it's own, I think that all of the text from the Working in an Environment section should be made into it's own md file. You can call it environments.md. You can add a note stating mamba is preferred and that instructions for install are in the python.md. There are a couple typo changes we can make to the text itself aside from @wsauthoff's ideas, but I think that will cover most of it. This will be extremely useful for a lot of different people who are working on tutorials right now for the hackweeks!

[facusapienza21]

Hi @wsauthoff @tsnow03 ! Thank you for your answer!

I can definitely include these changes in a new version and also make some typo check (and then we can revisit it in the PR in case there is still something that is not clear). I like the idea @wsauthoff of adding a short description of why we want to work in a persistent environment and the comments about mamba, definitely addthem.

Regarding why I removed how to install past versions of Python. My thought was that that part can be included directly later in the tutorial and that it doesn't make much sense to create a new environment from the terminal given the fact that is not going to persist. From a notebook cell, it still makes sense to make a conda install because that will last during the Hub session. I can instead make a note about the fact the user can do these two, but I still believe it's better to set the persistent environment first.

Regarding a new tab for the persistent environment. I also had this first thought @tsnow03 , but then I decided to put it all in the Python installation because most of the things already discussed in this tab regard Python environments, and creating a persistent conda environment is the best way of making a Python installation, both in terms of having fewer conflicts and more reproducibility support. What do you think? If you still believe is more useful to have it in a different tab, I can just move it, but also I like the idea of having the workflow explained all together in the same tab. I also noticed that in the JupyterBook Preview there is already some description of how to create an iPython kernel for a given environment. Maybe it makes more sense to add some of this text in the current Environment setup page? Happy to put this together too.

Regarding my second question. Thank you @wsauthoff for the answer. I can build the JupyterBook locally, but I also knew there was a way (that I never was able to figure out by my own) in which you can automatically build the JupyterBook using GitHub at the moment of doing a PR, in such a way that you can test if the JB works well without you manually building the book. Just a curiosity.

One extra point. It took me quite some time to figure out what How Tos/HOW TOS meant. Probably this is a combination that i) English is not my first language and ii) I am a little bit slow with grammar, but it may be a reason of confusion for some other people too. Just by writting HOW TOs I think this will read more easely, or maybe just intruction or tutorials.

Thank you for all the comments! Let me know what do you think about this points and then I can just move forward and implement the required changes.

[tsnow03]

Thank you for all of the answers! I think you make a good point about keeping the environment info in the Python.md. Let's change the header on the file to Python Install and Environments instead so it is easy to find this info. I think a lot of people will need to find it and Python Installation won't be super intuitive.

The comment about How Tos we can definitely improve by switching everything to "How To..." and HOW_TO

[wsauthoff]

@facusapienza21, I have seen some GitHub repo's with a Github action enabled that auto builds a preview of the JupyterBook if you apply a label (e.g., preview) to your PR if that's what you're thinking of. The CryoCloud website repo doesn't have that GitHub action enabled at the moment, so the method I mentioned in my last comment is your best bet to preview before submitting your PR.