Interface

Overview

The Expression Heatmap

Heatmapper is a Shiny Application that borrows much of its design from the previous, R-Based iteration. All the applications are structured similarly, with three important parts:

1. The Navigation Bar. At the top of the window, the Navigation Bar holds a collection of links to the other projects, alongside a link to this About Wiki. Application links open in the same tab to speed up loading, with the about page being opened in a separate tab.
2. The Main Tab. The center and focus of the application, where the Heatmap is displayed, the Table to the underlying data can be inspected and modified, and any other functionalities can be displayed (Such as Expression’s Dendrogram tabs).
3. The Sidebar. On the left of the window, the Sidebar holds all the functionality for selecting examples, uploading files, and all the settings that can be configured for whatever is currently selected in the Main Tab.

Column Filtering

For most of the applications, Heatmapper will try to intelligently choose columns based on a simple white/black listing scheme. When input is provided, Heatmapper will look within the data for column names associated with values such as Names, Values, or Dates, and will remove columns that don’t match what is requested (For example, if looking for a Names Column, Heatmapper will exclude column names such as “Longitude”). These values are then presented in the Sidebar, so that the user can make manual changes, or change which column should be used if there are multiple candidates. For more information about this filtering, check the Source Code

Filtering also ignores the order of columns, which means the internal ordering of data does not matter. Heatmapper will look for a Name column, but it does not need to be the first, second, X column unless explicitly specified in Format

Tables

The Table for Example 1 of the Expression Heatmap

Almost all of the applications within Heatmapper use tables are their inputs. These are internally stored as pandas.DataFrames, which means that they can be sourced from either comma-spaced values, tab-separated values, excel spreadsheets, or any format that can be parsed by pandas.read_table. All Tables within Heatmapper can be directly modified; to do this, simply double-click on the cell in the table to modify, and you will be able to input a new value for that cell. When on the table tab, the sidebar is populated with relevant settings:

• The Datatype radio buttons define what the newly inserted information should be inserted as. For non-name columns, these should be either integers (Whole numbers) or floating (decimal numbers). Trying to place a string in a location where Heatmapper expects data can cause errors.
• The Reset Values button erases any session modifications made to the table, returning it to the original state. This is useful if modification have caused errors, but values cannot be restored after being reset!

Global Settings

Many of the applications share settings in the Sidebar, which will be discussed in this section. The most important is the Upload/Example inputs, found at the top of the Sidebar. It consists of several components, some of which are conditionally:

• The Help link will take you to the Interface page for the current heatmap class.
• The Format link will take you to this About Wiki, where expected data formats are outlined in Format
• The Source File input defines whether you want to use an Example Input, or you want to upload your own data to Heatmapper.
  • When Example is chosen, there is a input to select a specific example, alongside an Info hyperlink to provide information about that specific example. In a WebAssembly environment, these files are fetched from online; in a Python environment, they are fetched locally.
  • When Upload is chosen, a file-upload input is provided. These dialogues filter allowed files based on supported file extensions (IE .csv, .txt, etc). If your file is not showing up, it has an incompatible extension. It is usually possible to override this, but there may be unexpected behavior.

Most applications also support an Explicit Update toggle. In these applications, there are two ways of updating the rendered heatmap: dynamic or Auto Update, and manual or Update. The update button sits underneath the file selection inputs mentioned above; by default, it is set as Auto Update, which means that any changes to the options in the sidebar will automatically update the heatmap visualization. This can be convenient, and thus is the default, but for computationally expensive updates, it can be undesirable for the heatmap to update on each change, especially when multiple modifications are needed. Toggling the input slider within the button will toggle to regular Update. In this mode, the heatmap will not be updated until you explicitly hit the update function; clicking the button while in Auto Update mode will have no effect.

Additionally, there is also a Download Table button on the Table tab, that allows you to download the currently loaded input table as download.csv You may need to change the extension of this download to match the corresponding type.

Image Settings

For MatPlotLib based applications (Pairwise, Expression, Image, and Spatial), you have additional heatmap settings:

• Size defines the size of the heatmap on your screen. Setting higher values may require you to horizontally scroll to see the entire thing.
• DPI defines the resolution of the heatmap. Higher values are sharper, but take more time to compute (At least for the initial computation before caching). If you increase the size of the plot, you may want to increase the DPI as well in order to maintain visual sharpness. Additionally, DPI affects the downloaded plot
• Download will download the plot as a .png. You can additionally right click the plot itself. You’re DPI selection will apply to the resultant image.

3D Rendering

For all aforementioned Heatmap types, except for Spatial, there is additional support for a 3D renderer, which seamlessly toggles based on the Elevation parameter exposed in the sidebar. While Elevation is at the default value of 90 (IE top-down) the 2D renderer is used, and all 3D settings described below are hidden:

• Elevation controls whether the plot is done as a 2D, or 3D render. When at 90, the plots are displayed in the standard 2D render. When at any other value, it controls the elevation angle that the camera in the 3D render is placed at in respect to the model.
• Rotation controls the angle of rotation around the model.
• Zoom controls the zoom level of the camera for Pairwise and Expression
• Inter controls interpolation for Pairwise and Expression. It specifies a multiplier, a value of 2 will interpolate the data from an NxM to a 2Nx2M, effectively quadrupling the resolution of each data point by interpolating it into 4.
• Scaling controls whether Pairwise and Expression will scale all values by its minimum value—ensuring that values are above 0 and thus do not extend below the X-Y plane.
• Opacity controls the opacity of the bars.
• Slices, used in Image, determine whether 2D projections should be drawn on the XY, XZ, and YZ planes.
• Height, used in Pairwise, can specify whether to use another metric for height values, independent of the coloring. The Cube option toggles a unique method of rendering, requiring an X, Y, and Z column to compute a pairwise matrix placed on the XY, XZ, and YZ Planes. Additionally, a Z Axis feature checkbox is provided in the Features section of the application, which toggles the visibility of the Z Axis.

Expression

Heatmap

The Expression Heatmap

Expression hosts numerous different settings that modifies the output visualization:

Heatmap

• Names specifies the column used for naming. It is often either NAME or UNIQID (In many cases, you can change between these if you want a more verbose name on the heatmap axis).
• Clustering defines the Clustering Method used to group the entries in the input. For more information, see Here
• Distance defines the Distance Metric to use. For more information, see Here
• Text Size defines the size of the text on the axis of the heatmap.
• Scale specifies whether values in a respective cell should be normalized to either its Row, Column, or None.
• Inter defines the interpolation algorithm to apply to the figure. This can cause values to bleed together, which may not be wanted. For more information, see Here

Colors

• If the Custom checkbox is enabled, the drop-down input will be replaced with a variable text field. You can then select colors from a drop-down, which will populate the field. Heatmapper will use the selected colors to create a color-map, with the first color representing low values, and the last color representing high values. There is no maximum to the amount of colors used, and they will create a linear segmented color map. A minimum of two colors are needed.
• Number controls the amount of distinctive bins used to create the color map. Fewer bins means fewer shades derived from the defined colors.

Features

• Row Dendrogram will toggle the dendrogram along the X axis.
• Column Dendrogram will toggle the dendrogram along the Y axis.
• X Labels will toggle the data labels along the X axis.
• Y Labels will toggle the data labels along the Y axis.
• Z Labels will toggle the data labels along the Y axis.
• Legend will toggle the color-bar legend.

Dendrogram

Row Dendrogram for Example 1 of the Expression Heatmap

The Expression Heatmap has two additional entries in the Main Tab: Row Dendrogram and Column Dendrogram. These display the respective dendrogram’s that are affixed to the main figure’s axis.

• Names, Clustering, Distance, and Text Size are all the same as the ones described in the Heatmap Tab.
• Dendrogram defines the orientation of figure.

Pairwise

The Pairwise Heatmap

Pairwise has numerous toggle settings, some of which only apply to certain input types and modes:

Heatmap

• Matrix defines what kind of visualization to output. Choices include either Distance or Correlation.
  • If Distance is selected, there is an input to provide the Distance Method used to calculate the distances between values. See Here
  • If Correlation is selected, there is an input to provide the Correlation Method used to calculate the correlation between values. See Here
• Text Size defines the size of the text on the axis of the heatmap.
• Inter defines the interpolation algorithm to apply to the figure. This can cause values to bleed together, which may not be wanted. For more information, see Here
• Chain is used only when the input file is a PDB file, and defines which Chain within the file should be displayed. See Here
• K-Mer is used only when then input file is a FASTA file, and defines K. See Here
• Distance if only visible when Matrix is set to Distance, and defines the method used to calculate distance between data points.
• Correlation if only visible when Matrix is set to Correlation, and defines the method to determine correlation between data points.

Colors

• If the Custom checkbox is enabled, the drop-down input will be replaced with a variable text field. You can then select colors from a drop-down, which will populate the field. Heatmapper will use the selected colors to create a color-map, with the first color representing low values, and the last color representing high values. There is no maximum to the amount of colors used, and they will create a linear segmented color map. A minimum of two colors are needed.
• Number of Colors controls the amount of distinctive bins used to create the color map. Fewer bins means fewer shades derived from the defined colors.

Features

• X Labels will toggle the data labels along the X axis.
• Y Labels will toggle the data labels along the Y axis.
• Z Labels will toggle the data labels along the Z axis if rendering as a 3D model.
• The Data Labels feature displays the associated value for each point on the Heatmap. For large data files, this can be a taxing operation, and likely ineligible to read.
• Legend will toggle the color-bar legend.

Image

The Image Heatmap

Image only has a single mode of visualization, so the follow settings apply universally:

Heatmap

• Text Size defines the size of the text on the axis of the heatmap.
• Map defines the color scheme used to visualize the data.
• Contour defines the algorithm used to generate the contours of the heatmap. See Here
• Levels defines the number of contour levels generated. Higher numbers create more refined contours.
• Opacity defines the opacity of the heatmap overlain on the underlying image. Lower values make the background image more visible.

Image Settings

• Quality defines the multiplier to downscale the image, improving rendering speed

Features

• X Labels will toggle the data labels along the X axis.
• Y Labels will toggle the data labels along the Y axis.
• Z Labels will toggle the data labels along the Z axis if rendering as a 3D model.
• Legend will toggle the color-bar legend.

Geomap

The Geomap Heatmap

Geomap has two modes of visualization, static or temporal, with some changes between each:

• Specify a GeoJSON File defines the source for the .geojson that is used to lookup locations and their coordinates. If Provided, a drop-down input is shown to select a .geojson provided by Heatmapper. If data has a corresponding .geojson, Upload provided a file-upload to use instead.

Columns/Properties

• Key specifies the column name Heatmapper will use to determine names, associated with values with the .geojson. These values must match; the GeoJSON tab on the Main Tab provides a immutable table of the names within the latter, which can be used to modify the values in input via the Table tab.
• Value specifies which column has the data that should be plotted. If Temporal Choropleth is specified, and the data does not have an associated Temporal column, this value is disregarded.
• GeoJSON defines the property within the GeoJSON file that should be mapped to the names in the data. If the GeoJSON property and Key Column don’t match, then Heatmapper will be unable to assign values to territory.

Heatmap

• Temporal defines whether the input should be interpreted as data over time, which can be navigated with a time slider embedded into the map. The input semantics of Temporal Choropleths can differ from regular choropleths, consult Format for details
• Map determines the underlying map type. OpenStreetMap is map used in the original Heatmapper, where CartoDB is a more modern, grayscale map type.
• Heatmap Opacity defines the opacity of the heatmap overlain on the underlying image. Lower values make the background image more visible.

Colors

• Map defines the color scheme used to visualize the data.
• Number defines how many colors should be used in the Heatmap. In essence, this controls the number of bins that data is put into.

Range of Interest

• Range of Interest defines a minimum and maximum bound to which data in the input will be disregarded if it falls below or above respectively. Note that for Temporal Choropleth’s, these values are applied universally, even if there is a unique column for time stamp.
• The Enable (Lower/Upper) Checkbox will toggle whether the Range is actually applied, allowing you to make modifications before updating the map
• The Remove/Round radio buttons define the mode of value culling. In Remove mode, values that fall outside the defined range are removed from the data, whereas in Round mode values that fall outside the defined range are brought to the maximum or minimum value.
• The twin numerical input define the lower and upper bound respectively.

Other

• Download will generate an HTML version of the map that can be locally downloaded.

Geocoordinate

The Geocoordinate Heatmap

Geocoordinate, like Geomap, has both a static and temporal visualization, which changes the functionality and what some settings do:

Columns

• • Time defines a Temporal Column within the data to use to plot data over time. The map will have a player that allows for time to be visualized like a video, where controls for rewinding, stopping, playing, and setting FPS can be defined. There are some difference in file format between normal Geocoordinate input, and temporal geocoordinate input outlined in Format. If None is selected, the data will be treated as static.
• Value specifies which column in the input data has the associated values for each longitude, latitude pair. If Uniform is selected, each data point will be given a uniform value, rather than using values from a column.

Heatmap

• Render defines the rendering mode. In Raster mode, the data is drawn as an image, and then plotted to the map, whereas in Vector mode, each data point is drawn as an individual circle directly to the map. Raster is faster, but less accurate, whereas Vector is more precise, but can be slower and there can be overlapping artifacts at high zoom. Render does not apply to temporal heatmaps.
• Shape defines the shape applied to the vector renderer. Circles are good for non-contiguous data, whereas Rectangles are better for contiguous data.
• Map determines the underlying map type. OpenStreetMap is map used in the original Heatmapper, where CartoDB is a more modern, grayscale map type.
• Size defines how large each point on the map are visualized. This value fluctuates between Temporal/Static and Scaled/Non-Scaled heatmaps.
• Opacity defines the opacity of the heatmap overlain on the underlying image. Lower values make the background image more visible.
• Blurring defines how much points within a similar location bleed into one another. This can make the heatmap appear more homogeneous, rather than being made up of individual points.
• Inter defines an Interpolation value to apply to the data, improving blending between data; this option may provide artifacts if the data is not contiguous.

Range of Interest

• Range of Interest defines a minimum and maximum bound to which data in the input will be disregarded if it falls below or above respectively. Note that for Temporal Choropleth’s, these values are applied universally, even if there is a unique column for time stamp.
• The Enable (Lower/Upper) Checkbox will toggle whether the Range is actually applied, allowing you to make modifications before updating the map
• The Remove/Round radio buttons define the mode of value culling. In Remove mode, values that fall outside the defined range are removed from the data, whereas in Round mode values that fall outside the defined range are brought to the maximum or minimum value.
• The twin numerical input define the lower and upper bound respectively.

Features

• KDE toggles Kernel Density Estimation when Scaled is not enabled, where the value of points will be modified so that denser areas are hotter than sparser ones.

Other

• Download Heatmap will generate an HTML version of the map that can be locally downloaded.

3D

The 3D Heatmapper

3D has three modes of operations based on the provided input, with different configuration for each, which can be classified into two main categories: Protein and Objects. Because these modes are so different, the sidebar will not be populated until a input file or example is provided.

Protein Models

The 3D Protein Model Viewer is accessible by providing a PDB File to the application. There are many options to customize the behavior and visualization, but some settings are only applicable to certain Styles; unless otherwise specified, a setting will apply to all styles.

Heatmap

• Scheme defines the coloring of the model. The default option, spectrum, applies a reversed gradient based on residue number. See here for a detailed explanation of the other color schemes.
• Opacity defines the opacity of the Heatmap

Surface

• Scheme defines the coloring of the surface. The default option, spectrum, applies a reversed gradient based on residue number. See here for a detailed explanation of the other color schemes.
• Opacity defines the opacity of the surface

Customization

• Model # defines what Model to use within the PDB, if the PDB has multiple models.
• Styles defines the rendering style of the model.
• Surface Type defines the type of surface that should be drawn over the model.
• Thickness only applies to the Cartoon Style, and specifies the cartoon strand thickness.
• Width only applies to the Cartoon Style, and specifies the cartoon strand width.
• Radius applies to Stick, Sphere, and Cross Styles, and specifies the fixed radius of spheres.
• Scale applies to Stick, Sphere, and Cross Styles, and specifies a scalar to modify the VDW radius.
• Size specifies the width and height of the viewer frame within the application.

Features

• Dashed Bonds applies to the Stick Style, and specifies whether to draw bonds as dashed bonds.
• Show Non-Bonded applies to the Stick Style, and specifies whether to display non-bonded atoms as spheres.
• Single Bonds applied to the Stick Style, and specifies whether to draw bonds as single bonds.
• Tubes applies to the Cartoon Style, and specifies whether to display alpha helices as simple cylinders.
• Trace applies to the Cartoon Style, and specifies whether to draw the model as a simple trace.

Object Models

The Object Model Viewer is accessible by providing an Object file, or .obj file. This displays a 3D interactive model of the provided model, to which either a texture file can be placed on the object, or a table of values. The following settings only apply to the latter case, as there is no customization if a texture is provided.

Heatmap

• Opacity defines the opacity of the Heatmap, only applicable to table input.
• Style dictates how to render the model.

Colors

• Number defines how many colors are used in the Heatmap, essentially defining the number of bins, only applicable to table input.
• Map defines the color scheme used to visualize the data, only applicable to table input

Features

• Edges will toggle rendering visible edges to each triangle face, making them more distinct
• Lighting toggles whether lighting is applied to the model; this can darken/lighten values for a smooth map, but will hurt accuracy of colors
• Interpolation applies interpolation of triangles.
• Smoothing Shading applies shading to the model, but Lightning needs to be toggled first.

Spatial

One important deviation across all of Spatial’s visualization is that the Download Table button will save user-uploaded files into a single h5ad file, that can be re-read by Heatmapper at a later date. Importantly, calculations, such as centrality scores and co-occurrence, are saved into this file, which means subsequent loads of the application with this file will not need to re-compute these values.

As mentioned in the Table section, Spatial’s data cannot be directly modified, only viewed in its respective tab. However, due to the nature of Spatial’s input, the Table Tab has a Table radio input, as there are two distinct tables that can be output from the input:

• Observations are values from the counts file.
• Variable are values that are calculated by Spatial for the various statistics it can output.

There are three kinds of input that Spatial can take, classified into two categories:

1. Raw files, such as the output of Space Ranger or NanoString. Multiple files will be needed.
2. Processed files, specifically the .h5ad files that are provided when downloading from Spatial. Only a single file will be needed.

The Upload Type input directly underneath the file selection dialog specifies which format has been uploaded; all of the example files are in Visium format.

Heatmap

The Spatial Heatmapper

Spatial has numerous different visualizations, with a wealth of functionality for each type. The Heatmap Tab contains the most settings:

Annotation Keys

• Annotation Keys defines which keys are used for annotation within the variables/genes. See Here More than one can be specified, which are displayed next to one another.
  • The Annotation Key dialog often has hundreds, if not thousands of keys depending on the dataset. You can type in the input box to filter the keys to what you’re looking for, as they typically aren’t alphabetically sorted.

Heatmap

• Static is the statistic to be calculated. Options include:
  • Moran’s I auto-correlation: See Here
  • Geary’s C auto-correlation: “A measure of spatial auto-correlation that attempts to determine if observations of the same variable are spatially autocorrelated globally". See Here
  • Sepal Score: “Simulates a diffusion process to quantify spatial structure in tissue.” See Here
• Map defines the color scheme used to visualize the data.
• Shape defines whether the shape of the individual points on the heatmap.
• Columns define how many keys are plotted horizontally before a new row is plotted below.
• Spacing defines the amount of space between plots, in case screen size / column size puts them too close together.

Opacity

• Image defines the opacity of the underlying image behind the data.
• Data defines the opacity of the data rendered on top of the image.

Features

• Image toggles the visibility of the underlying image
• Legend toggles the visibility of the color-bar legend. Due to quirk between Shiny and Squidpy, the Legend cannot be rendered unless more than one annotation key is specified.
• Frame toggles a frame around each plot.

Centrality Scores

Closeness Centrality computed for Example 1

Centrality Score has one major setting to determine which score is visualized:

• Score defines which score is displayed. Options include:
  • Closeness Centrality: measure of how close the group is to other nodes.
  • Average Clustering: measure of the degree to which nodes cluster together.
  • Degree Centrality: fraction of non-group members connected to group member See Here

Ripley’s Function

Ripley’s L-Function computed for Example 1

Ripley’s have two settings to determine which function is used, and the metric to compute it:

• Function defines which function should be used. Options include:
  • L: “[A] descriptive statistic for detecting deviations from spatial homogeneity.” See Here
  • F: "Works by analyzing the distance to points in the pattern from locations in empty space." See Here
  • G: “Summarizes the distances between each point in the pattern and its nearest neighbor.” See Here
• Distance Method defines the Distance Metric to use. For more information, see Here

Co-Occurrence

Co-Occurrence computed for Example 1

Co-Occurrence has two different forms of visualization:

• Graph Type defines whether to visualize co-occurrence in a scatter plot, or in a line plot.
  • Scatter will render all clusters in a scatter plot
  • Line will render a single cluster in comparison to the others.
• Cluster, only available if Graph Type is set to Line specifies which Cluster should be used for the line plot.
• Distance Interval defines the interval at which co-occurrence is computed. See Here
• Splits defines the number of splits in which to divide spatial coordinates. Leaving it at $0$ will tell Heatmapper to automatically choose a sensible value. See Here

Spectral

The Spectral Heatmapper

The Spectral Heatmap has two modes of visualization, both of which share the same set of features and customization.

Heatmap

• Text defines the text size of the heatmaps.
• Map defines the Color Map used.
• Peak Type defines what method to use to generate peaks in the data.
• Contour Size is used only on the main heatmap, and defines the interpolation size of the heatmap. Higher values are more accurate, but are more computationally expensive to generate.

Features

• X Labels will toggle the data labels along the X axis.
• Y Labels will toggle the data labels along the Y axis.
• Z Labels will toggle the data labels along the Z axis.
• Legend will toggle the color-bar legend