initial graph

[yankovs]

Karton graph visualization

Introduction

This pull request adds a visualization of the kartons and the connections between them. A graph tab is added to the dashboard, and in it a directed graph. The graph is interactive, allowing zooming, moving the graph and moving the nodes. A click on a node shows info about it: its name, version (if available) and description (if available).

In more details

Stack used

• echarts.js - for the visualization
• marked.js - for description parsing
• DOMPurify - to sanitize the description
• networkx - for graph building in the backend, also used to generate a gexf of the graph

Implementation details

A /graph route was added. When accessing it, the graph is built out of the binds and the outputs in redis. For each karton we create a KartonNode object representing it, which then allows to see if one karton should have an edge to another (i.e., if the former karton's outputs are contained in the latter's binds).

After creating a graph object, we style it and provide size and color for each node. We then output it in gexf format. This gexf format graph is passed to the frontend. An echarts instance is created from the graph and displayed.

Possible uses

A few possible uses for the graph might be:

• Discovery of faulty kartons; If a karton doesn't have any incoming or outgoing edges it might indicate of a problem
• Karton cycle detection; Is it okay that there's a loop between kartons?
• General karton system health checks

Future work

• Currently the graph is calculated each time /graph is accessed, maybe some caching mechanism can be used
• Currently the backend it blocking when requesting /graph so a user needs to wait until they get the graph with a reasonably big karton system changed so now a loading screen shows, but loading time should still be addressed
• Organize the graph in a better way, maybe as something that resembles a directed acyclic graph (although it is not acyclic)
• Implement highlighting of the cycles in the graph
• Implement highlighting all of the paths between two nodes in the graph

[nazywam]

Looks great, thanks for the contribution! <3

I've written down a few notes while playing around with it, we don't have to include them in this PR but if you agree with some of them I'll create issues to keep track & sort them out in the future.

Good to have:

• host the js dependencies locally - always nice to have your application self-contained
• visualize dead/inactive services - using a partial alpha or some icon maybe?
• add a slight duration when focusing a given edge/node - right now if you wave the cursor around the screen ends up flashing quite a bit and this can get annoying when using the graph regularly. I've tried to find the option responsible in echart but failed :(

Nice to have/loose ideas:

• the navbar is skewed a bit to the right (when compared to other subpages)
• show information about possible task headers in edges
• visualize number of tasks as edge weight - not even sure we store enough data right now to calculate that

Again, thanks for your contribution!
I'll give @psrok1 chance to take a look as well and then we'll get this merged!

[psrok1]

Wow, that's great ❤️ As @nazywam said before: there is some room for improvements, but as an initial version I think it's ready to merge. Huge thanks for that feature.

Sidenote: I applied small fixes on my own (mainly related with typings): 650e510

[nazywam]

:lgtm: