Consider organization of metadata files to avoid name clashes with salmon

[rob-p]

As pointed out by @jashapiro in issue 688 in the salmon repo, there is a name collision between the cmd_info.json file written by alevin-fry and the one written by salmon alevin if the quantifications are placed in the same directory as the input RAD file. Further, both tools produce their own meta_info.json file, though a clash is avoided because one writes this to an aux subdirectory while the other doesn't.

We should design a better scheme for naming and placement of metadata files, so that names don't clash even if the quantification output is written to the same directory as the initial RAD file.

[rob-p]

Here is my current proposal. I would appreciate any feedback @DongzeHE, @k3yavi, @hiraksarkar, @mohsenzakeri and @jashapiro.

Get rid of the cmd_info.json generated by the quant command of alevin-fry. At this point in the process, we don't have access to the information that was available to salmon when generating it's cmd_info.json file, and the spoofed version we created was only intended to aid in alevinQC compatibility. But since we've already started discussing with @csoneson adding alevin-fry support to alevinQC directly, we can assume this spoofing will no longer be needed.

Rename meta_info.json as output by alevin-fry quant to quant.json. This matches the naming convention used by other stages in the pipeline, like collate. This also avoids potential name clashes with the meta_info.json written by salmon alevin. However, this rename will require us to change the relevant load_fry functions in the usefulaf repo as well as in fishpond. For that, I propose that we first look for quant.json, and if we don't find it, look for meta_info.json for backward compatibility.

Any thoughts on this proposal?

[jashapiro]

This sounds quite reasonable to me.

I would also request that as this is implemented, you also add version info to each of the .json outputs. This is already present in the generate_permit_list.json file, but not in collate.json or the current meta-info.json (to be quant.json)

[rob-p]

Hi @jashapiro,

This has been implemented in 0.4.1 which has been tagged and merged into bioconda (bioconda/bioconda-recipes#29716). Hopefully the image should show up on biocontainers soon. Once you confirm on your side, we can close this issue.

--Rob

[jashapiro]

Thanks @rob-p, I will look at it soon!

[DongzeHE]

I like the neat idea of changing the name of alevin-fry quant from cmd_info.json to something like quant.json. If we decide to go this way, I will take care of the implementation of read_fry() everywhere.

[rob-p]

@DongzeHE --- I implemented this and made the changes in the python and R functions in usefulaf. I also updated tutorials. Could you take care of updating in the same way in fishpond? You can see how I handled it in usefulaf (basically, I look for quant.json first, and if I don't find it, I fall back to meta_info.json assuming it's from an older version).

[jashapiro]

Just confirming that I have tested this version (using the docker image), and the updates look great. Thanks for the additional info in each of the output files! Thanks again @rob-p!

-Josh

[rob-p]

Awesome --- I think we'll consider this issue closed then.