Update uom for measurements #20
Conversation
First option is from the XML housed in PHS Ontology Second is directly from the observationfact.units_cd If both fail the value is set to 'unknown' concept 0.
Addded logic to OMOPMeasurement to limit the join to i2o_onotology_lab only based on the most recent XML entry. Cleaned up the preparePHSOntology.sql
…ta at the moment. May bring them back in. Removed the inner join to the visit_occurrence which does not change the number of records as every measurement does have a visit_occurrence. Added more restrictions to the i2o_ontology_lab records that are used. Fixed the second join to the i2o_unitsmap
Added soem additional comments
MSSQL/preparePHSOntology.sql
Outdated
| @@ -0,0 +1,75 @@ | |||
| --------------------------------------------------------------------------------------------- | |||
There was a problem hiding this comment.
This script almost could be used generally, not just at PHS. You don't need to make those changes, but please describe what it's doing in the documentation and it is adaptable to any site using low/high/units in metadata XML.
There was a problem hiding this comment.
Also I would rename it something like extractUnitsFromOntology
There was a problem hiding this comment.
Renamed the script to be not PHS specific.
MSSQL/OMOPLoader.sql
Outdated
| @@ -1805,15 +1805,17 @@ cast(m.start_Date as datetime) measurement_datetime, | |||
| -- NOTE: Are these concept_ids the ones analysts want? | |||
| CASE isnull(m.VALUEFLAG_CD,'0') WHEN 'H' THEN '45876384' WHEN 'L' THEN '45881666' WHEN 'A' THEN '45878745' WHEN 'N' THEN '45884153' ELSE '0' END value_as_concept_id, | |||
| CASE WHEN m.ValType_Cd='N' THEN m.NVAL_NUM ELSE null END value_as_number, | |||
| isnull(m.Units_CD,'') unit_source_value, | |||
| -- NOTE: Units are pulled from PHS Ontology first (extracted from XML) , if not present then it falls back to units_cd on the observation fact | |||
There was a problem hiding this comment.
Please elaborate to include what it is actually doing (pulling from i_unit in ontology vs units_cd in fact table)
There was a problem hiding this comment.
Added more elaborated comments.
MSSQL/OMOPLoader.sql
Outdated
| left outer join i2o_unitsmap u on u.units_name=m.units_cd | ||
| left outer join i2o_unitsmap u2 on u.units_name=m.i_unit |
There was a problem hiding this comment.
I think it'd be cleaner to use one left join with and OR in the ON clause?
There was a problem hiding this comment.
Per our discussion on Friday leaving this as 2 left outer joins with the isnull() function in the select provide more control over precedence and avoids accidental duplicate records.
MSSQL/preparePHSOntology.sql
Outdated
| GO | ||
|
|
||
| --Manually UPSERT unit mappings for most commonly used unit values in the PHS XML values |
There was a problem hiding this comment.
These seem useful even to people not using the XML-extraction above. Maybe they should go in a different script?
There was a problem hiding this comment.
This has already been separated out into it's own file.
Added additional comments
This pull request covers the following...