Error when generating a node depth accelerator using exine depth

[tonyjie]

Hi, I'm trying to run some Pollen before our meeting. I basically followed the installation guide for Pollen, Calyx, ODGI, and got an error when generating a node accelerator here. I think the error is still something related to ODGI-python binding.

Do you have any idea on how to fix this?

Error message

(py39) ➜  ~/Other_Repos/pollen/test/depth git:(main) exine depth -a -r t.og
warning [libhandlegraph]: Serialized object does not appear to match deserialzation type.
warning [libhandlegraph]: It is either an old version or in the wrong format.
warning [libhandlegraph]: Attempting to load it anyway. Future releases will reject it!
Traceback (most recent call last):
  File "/home/jl4257/.local/bin/exine", line 8, in <module>
    sys.exit(main())
  File "/home/jl4257/.local/lib/python3.9/site-packages/pollen/main.py", line 26, in main
    depth.run(args)
  File "/home/jl4257/.local/lib/python3.9/site-packages/pollen/depth/main.py", line 180, in run
    run_accel(args, tmp_dir_name)
  File "/home/jl4257/.local/lib/python3.9/site-packages/pollen/depth/main.py", line 104, in run_accel
    parse_data.run(args)
  File "/home/jl4257/.local/lib/python3.9/site-packages/pollen/depth/parse_data.py", line 281, in run
    max_nodes, max_steps, max_paths = get_dimensions(args)
  File "/home/jl4257/.local/lib/python3.9/site-packages/pollen/depth/parse_data.py", line 190, in get_dimensions
    max_nodes, max_steps, max_paths = get_maxes(filename)
  File "/home/jl4257/.local/lib/python3.9/site-packages/pollen/depth/parse_data.py", line 167, in get_maxes
    graph.load(filename)
RuntimeError: Error rewinding to load non-magic-prefixed SerializableHandleGraph

My current environment

• python 3.9.16
• import odgi is successfull after I followed the detailed installation here from bioconda.
• Calyx installed. fud check shows the following messages.

global:
 ✔ /home/jl4257/Other_Repos/calyx exists.

stages.calyx.exec:
 ✔ ./target/debug/calyx installed.
   ./target/debug/calyx is a relative path and will not work from every directory.

stages.interpreter.exec:
 ✖ /home/jl4257/Other_Repos/calyx/target/debug/interp not installed.

stages.dahlia.exec:
 ✖ dahlia not installed.

stages.verilog.exec:
 ✖ verilator not installed.

stages.vcd.exec:
 ✖ vcdump not installed.

stages.jq.exec:
 ✖ jq not installed.

stages.synth-verilog.exec:
 ✖ vivado not installed.

stages.vivado-hls.exec:
 ✖ vivado_hls not installed.

stages.futil.exec:
 ✔ /home/jl4257/Other_Repos/calyx/target/debug/futil installed.

interpreter, dahlia, verilog, vcd, jq, synth-verilog, vivado-hls were not installed correctly.
Configuration instructions: https://docs.calyxir.org/fud/#configuration

[tonyjie]

I found the problem is actually due to those .og file is symlink when I clone the project.
exine depth -a -r t.gfa doesn't help. It leads to MemoryError: std::bad_alloc error.
But I build the t.og on my own using odgi build. Now odgi-py is able to load the .og file.
Then I update the key of stages.calyx.exec to be the absolute path.

Now I am able to go to the new error:

(py39) ➜  ~/Other_Repos/pollen/test git:(main) exine depth -a -r k.og


⠋ calyx.run_futil

⠙ calyx.run_futil

⠹ calyx.run_futil

⠸ calyx.run_futil

⠼ interpreter.interpret
[fud] ERROR: `/home/jl4257/Other_Repos/calyx/target/debug/interp  --raw  -l /home/jl4257/Other_Repos/calyx --data /tmp/tmpfznhcvmq/data.json /tmp/tmp8nqn8xuq' failed:
=====STDERR=====
/bin/sh: 1: /home/jl4257/Other_Repos/calyx/target/debug/interp: not found

=====STDOUT=====

Seems like this is due to my Calyx installation:

stages.interpreter.exec:
 ✖ /home/jl4257/Other_Repos/calyx/target/debug/interp not installed.

Is there any step that I missed? I installed Calyx from source following this.

[anshumanmohan]

Thank you for checking us out, Jiajie! It's very helpful as we try to take flight.

Sorry about the incorrect tip re: using .gfa files. I realized it would not work and then deleted my message; didn't realize you had already seen it 😅

I think you've diagnosed this issue as well as I could have, so now I'd like to wait for @susan-garry to chime in and give us a solution. This stuff is her domain!

Susan, I think this may be a little additional nudge re: getting rid of those symlinked .og files. Earlier we thought it was just a stylistic issue, but the issue that Jiajie ran into may be more compelling than just style?

[sampsyo]

Hi, @tonyjie! Can you try doing cargo build -p interp in the Calyx directory?

[tonyjie]

Hi, @tonyjie! Can you try doing cargo build -p interp in the Calyx directory?

It seems to solve my issue! Now exine depth -a -r k.og can work!

[anshumanmohan]

Reopening: we should do some tidying in our text, and possibly some reorganization of our symlinked .og files, before closing this issue. Thanks @tonyjie!

[sampsyo]

Perhaps this is the place where we want to mention the need to build the Calyx interpreter:

pollen/README.md

Line 42 in 35c690d

Follow these [instructions](https://docs.calyxir.org/) to install calyx. You must complete the [first](https://docs.calyxir.org/#compiler-installation) and [third](https://docs.calyxir.org/#installing-the-command-line-driver) sections, but feel free to skip the second. The last step should be running `fud check`, which will report that some tools are unavailable. This is okay for our purposes.