Biothings transformer support for arbitrary complex queries

[tokebe]

Currently, the Biothings transformer gets input IDs from POST queries by passing the query associated with a hit into generateCurie(). This only supports queries formatted with the input ID at the very end of the string, as generateCurie() splits a string by : and takes the last item.

Complex query strings can easily cause problems if they don't leave the ID at the end of the string. After discussion with @colleenXu, the most doable fix at the moment is to run through queryInputs (accessible in that code section via this.edge) and check for inclusion in the query string, taking the first item that matches, and then passing that to generateCurie().

[tokebe]

Note that there could be similar issues in other transformers, and further review is required to determine how this might interact with the JQ PRs. @rjawesome , do you have any insight?

[rjawesome]

The JQ has access to this.edge, so a fix for this could be implemented by modifying the JQ transformers for biothings.

[rjawesome]

I just pushed a fix for this on jq branch with a generateCurieWithInputs jq function that takes in the curie input so this can easily be used for any other APIs as well.

@tokebe Just wanted to know, is the field "query" returned by biothings ever an array as currently my code doesn't handle that.

[tokebe]

I don't believe so, @colleenXu or @erikyao might be able to answer with more certainty.

[colleenXu]

@rjawesome not sure if you mean "queryInputs" for "the curie input" mentioned in your post.

I don't pay a lot of attention to what's in the "query" field...so I don't know. Maybe @erikyao or @newgene would know..

[colleenXu]

Also @tokebe I wonder if you could check @rjawesome's commit? biothings/api-respone-transform.js@ea9db14

I'm not sure that I'm understanding it correctly, but it seems to be doing something with prefixes and upper-case.

---

This may not be relevant, but I'll note some things about prefixes, IDs, and letter-capitalization here

• we're still figuring out the ID prefix stuff handling ID-prefixes in BTE in a consistent, straightforward way #591 , but most elements in queryInputs probably don't have prefixes?
• not all biolink-model/Translator prefixes are all-caps and some have special characters like the "." (ncats.bioplanet, UniProtKB are two common examples)
• some IDs (not the prefixes!) have letters in them (DBSNP has rs, UMLS and NCIT has C, MESH has D, CHEMBL has CHEMBL)
  • I think the BioThings APIs have the "correct" letter capitalizations for these IDs, and that they're biolink-model/Translator-compliant. However, I'm not sure because I haven't asked. I remember asking for particular IDs before and not getting a straightforward answer...
• FYI: Other APIs (TRAPI and external = non-TRAPI/non-BioThings) sometimes have different capitalizations for prefixes and IDs. It'd be nice if we were able to handle these gracefully (convert to our desired format?). I notice that SRI Node Norm seems case-agnostic for inputs but provides outputs in the desired format for capitalization (example with MESH ID)

[rjawesome]

I just converted everything to uppercase when checking for inclusion so that casing wouldn't be an issue. Currently my code assumes queryInputs doesn't have a prefix but the case of a prefix could be handled by just splitting by ":".

[colleenXu]

This is tied to the JQ work #489 and biothings/api-respone-transform.js#38

[colleenXu]

@tokebe Is it okay to close this issue? Isn't this only deployed on Dev/CI instances? (so we can't adjust the registered SmartAPI yamls if we want)

[tokebe]

This was auto-closed by commit-linking, whoops!

[tokebe]

Relevant changes deployed to Prod.