Dev

Overview

This documentation is for individuals either wanting to contribute to Heatmapper, or deploy it to a server.

Deployment

For reference, you should look at Running Client-Side PyShiny, as those instructions are also applicable to hosting Heatmapper on a server.

The setup.sh script located in the root of the repository can setup a complete environment for running Heatmapper, including setting up a virtual environment, installing dependencies, cloning Heatmapper, and resolving LFS files. It’s a bash script, so deployment on a Windows server will need to be done manually. From the repo, find setup.sh from the file list. Upon clicking on it, GitHub should take you to a viewer, with a download button in the top-right corner. Or, if you only have access to a terminal, you can curl the script via:

curl -O https://raw.githubusercontent.com/WishartLab/heatmapper2/main/setup.sh

From there, make it executable:

chmod +x setup.sh

Then, place it into the directory you want Heatmapper to live in. setup.sh will create two directories:

1. The python virtual environment in venv
2. The Heatmapper source code in heatmapper2

Once the script is finished, or you’ve manually handled dependencies and installation, you’ll next need to activate the Virtual Environment (Assuming you’re using a venv and not just installing dependencies to the system). From the folder containing the venv folder, run:

source venv/bin/activate

You can deactivate the virtual environment at any time by typing:

deactivate

Now, enter the heatmapper2 directory. For batch deployment, there are two scripts which automate the process:

1. deploy.sh will deploy each application on the host, starting at port 8000 for Expression, and ending with 8006 for Spatial. Each process will run in a separate process, so the script (and user session) can be closed without tearing down the applications themselves.
2. teardown.sh will send a KILL signal to all applications listening on the ports 8000-8006. If you’re only selectively hosting Heatmapper’s applications, this might kill non-related applications if they’re listening to that port.

However, if you want to be more selective about which applications are run, you have two primary options:

1. Running it as a PyShiny application. To do this, navigate into the project to run, such as expression, and enter its src directory. From there, execute: shiny run --host 0.0.0.0. The host argument is important to be listening on all network interfaces. If you want to enable reloading, so that changes to the src folder will be transparently noted and changed within the application—without needing to stop it—add --reload. To specify a port, use the --port argument
2. Running it as a Static, WebAssembly application. This mode will instead server a connecting client with the WebAssembly files, which are then run on their computer. From the project folder expression, there are two sub-folders, src and site. Simply run python3 -m http.server --directory site --bind localhost 8008, where the value 8008 specifies the port.

Contributing

This section outlines some general guidance on working within the Heatmapper repository.

Coding Convention

For sake of consistency, Python Code should:

1. Always use from imports, rather than importing the entire module: Do from shiny import App, not import shiny
2. Use double quotes rather than single quotes for strings
3. Use tabs, rather than spaces
4. For naming convention:
   1. Local variables should use snake_case
   2. Global variables, functions, classes, and Shiny IDs should use PascalCase
5. Prefer code that is more concise. If a function only has a single line, put in the function definition, such as async def Reset(): await DataCache.Purge(input)
6. Strive to consistently document the code-base. All non-trivial functions should have doc strings, which should follow Doxygen format.
7. Use shared.py definitions over creating something custom. If functionality is missing, add it to the shared.py implementation.
8. Always use the Cache object for handling input
9. Always use the Filter function to determine column names.
10. Always use the NavBar function to create a navigation bar shared across all applications.
11. shared.py should always be a symlink within the src folder. Do not copy it.

When creating a new Application, there’s a few things to note:

1. You should create a DataCache variable from the shared.Cache class, which will handle all your user-input. This should be in the server function.
2. If you need to extend the Cache, such as adding more file-types, create a function that you can pass to the Cache call.
   1. Treat it like a switch statement. You will be passed a single argument, path. Compare against the suffix to see if it matches your custom file type. If it doesn’t, return DataCache.DefaultHandler(path). Do not modify the Default Handler, it bogs down all the applications.
3. FileSelection should be used to generate the UI for uploading/selecting input. Importantly:
   1. It will create Shiny input IDs SourceFile for whether the user is selecting Upload/Example. File for the user-uploaded file, and Example for the selected example. Additionally, it will create the ExampleInfoButton and ExampleInfo IDs. ID conflicts cause Shiny to fail.
   2. You will need to manually set ExampleInfo. The easiest way to is to make a reactive function that looks at a dictionary defined in the server: def ExampleInfo(): return Info[input.Example()]
   3. The multiple argument should be used with caution. It requires you do manually handle parsing input. See Spatial for an implementation
4. The MainTab function supports adding additional tabs via the *args argument. See Spatial or Expression for implementations. It will create IDs: Interactive, which should be your main page Heatmap, Table, which you shouldn’t need to touch, as it handles creating all the associated values, and itself has an ID of MainTab. You may need to add ID’s Update and Reset so that your reactive functions update when the user updates the table.
5. You will need to manually Filter columns. This involves calling Filter in a reactive function with the following arguments:
   1. The input, usually (await DataCache.Load(input).columns
   2. The type of column to look for, see shared.py for values.
   3. A UI element to update, such as NameColumn

Rebasing

When changes are made within the code-base, they are not reflected in the WebAssembly site, which can cause incongruity when pushed to GitHub. From a particular application, run shinylive export src site to update it. Alternatively, run the rebase.sh script at the root of the repository to perform this action across all applications.

Configuration

Heatmapper is designed to be easily deployed for different purposes, and to this effect most of the interface can be modified without having to modify the code itself (Technically you modify code, but that’s just so that configuration is bundled in web assembly).

Each project contains a config.py file, a Python file which provides defaults and overrides to every configurable option in that program. However, the base config.py is within Heatmapper’s version control system, which means that modification of it can cause clashes when attempting to update. For that reason, you should copy config.py, creating a file named user.py. Heatmapper will first check if user.py exists, and use that for configuration, only falling back to config.py if the former doesn’t exist. Do not modify config.py Consider the configuration provided in Pairwise:

# Distance/Correlation
"MatrixType": Config(selected="Distance", visible=True),

This variable is attached to the associated input.MatrixType which defines whether the user wants to select a Distance Matrix, or Correlation Matrix. Let’s break it down:

• MatrixType, the input name, and cannot be modified as it’s explicitly used within the main program. You cannot add new configurations (Every user input that can be modified is already present in the file)
• Config is from shared.py, and is simply a class that wraps configuration. Every configuration is an Config object.
• selected is the only required argument of any configuration. This specifies what Heatmapper should assign as the default value when loading the application. A comment above each Config outlines what your values can be. Some configurations uses value instead, which is simply because some inputs “select” a value, such as the titular ui.input_select, whereas others simply have a value, such as ui.input_checkbox. The configuration already provides the correct keyword, so this has no impact on actually configuring the application so long as the original configuration keyword isn’t deleted.
• visible is an optional argument that defaults to True. When visible is True, the associated user input in the sidebar will be visible when loading the application, and the user can make modifications to the value. When visible is False, the input will be hidden from the sidebar, and the user will be unable to change the selected value. This is useful where an application has no need for the option to be available (Such as only needing to display Distance Matrices) and helps declutter the sidebar and prevent user confusion.
• Finally, something that is not shown in any of the default configurations, is that the Config class takes any key-word argument and stores it, applying them directly to the Shiny input object. Therefore, if we wanted to make sure the MatrixType’s radio buttons are not inline, we could modify the configuration to MatrixType = Config(default="Distance", inline=False). You may notice that Heatmapper already defines inline=True within Pairwise’s code, but Config objects will check for these conflicts, and will default to the Configuration. You can therefore override all of the parameters of the input, save the input type itself. Refer to Shiny’s excellent documentation if you want to make any such changes; note that you cannot change the input type itself, and some modifications may cause issues with the application (IE specifying multiple=True where Heatmapper does not expect multiple inputs)