Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: standarize refs across all docs #199

Merged
merged 10 commits into from
Jul 31, 2024
Merged

Conversation

tomasciccola
Copy link
Contributor

@tomasciccola tomasciccola commented Jul 25, 2024

should close #195
So, I tried to standarize references in docs by creating a ref protobuf and importing it in every document that needs it.
For protobufs is useful that we can import, which I'm not doing in JSONSchema (I'm basically copy-pasting ref on documents that use it, changing only the description). This can became kinda hard to track (but not a huge issue). I know JSONSchema allows to reference other schemas but I know that we're not using that feature for i.e. Common so there's probably a shortcoming I'm forgetting about.
Besides docId and versionId I added type so that we can track which type of documents are referenced. This was already used in tracks and translations (through schemaRef, now removed). Regarding this, I think it may make sense to have runtime checks (i.e. tracks should only reference observations) instead of having different ref types which only differ on the type of doc they reference

(missing is iconId in preset)
@EvanHahn
Copy link
Contributor

I'll review this soon. Could you fix the merge conflicts before I do?

@tomasciccola
Copy link
Contributor Author

Regarding asserting correct reference types:

  1. Tracks should only reference observations
  2. Presets should only reference fields or icons
  3. Translations? (On the top of my head we translate fields and presets, but maybe we want to keep it lean?)

Maybe this is smth we don't need to do and we can be lean about it...

Copy link
Member

@gmaclennan gmaclennan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial feedback/thoughts on a quick read-through:

  • A versionId is not a hex encoded string. We already have a proto def for this called Link. We encode it and decode it by hex-encoding the discovery key and joining with / and the index.
  • Also change Link to use the same definition.
  • I think we should have a ref definition that does not include a type, with the type implicit. Otherwise we are allowing for invalid refs. E.g. a preset can have a fieldRefs field, which can only be refs to fields, and don't need a type. Right now the only place I think where we need a type for refs is for translations. Since it's not an array, maybe we can just keep it as a separate refType field? I'm not sure about this one and would welcome Evan's thoughts.
  • Where possible, we should avoid calling this field ref or refs, and include a more descriptive name like fieldRefs, to make it easier to understand.
  • For tracks, I think it's maybe ok to just say the refs are observationRefs. I think if we want to link other things to a track, we can do it as a separate field.

Copy link
Contributor

@EvanHahn EvanHahn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have a ref definition that does not include a type, with the type implicit. Otherwise we are allowing for invalid refs. E.g. a preset can have a fieldRefs field, which can only be refs to fields, and don't need a type. Right now the only place I think where we need a type for refs is for translations. Since it's not an array, maybe we can just keep it as a separate refType field? I'm not sure about this one and would welcome Evan's thoughts.

I agree with this. I think it makes more sense to have two fields: fieldRefs and iconRefs, or observationRefs.

That probably means removing RefType from proto/ref/v1.proto.

proto/translation/v1.proto Outdated Show resolved Hide resolved
Tomás Ciccola added 3 commits July 30, 2024 14:54
* update ref fields and use a more specific name
* use ref.type only on translation
* remove unnecessary checks
* fix types and encode/decode
* add checks for repeated fields
@tomasciccola
Copy link
Contributor Author

tomasciccola commented Jul 30, 2024

Following the feedback I:

  • Add a VersionId proto to be used in Common.links and other refs
  • Delete Ref_1 proto
  • Add specific ref messages to the corresponding protos (and schemas):
    1. fieldRefs and iconRef on presets
    2. observationRefs on tracks
    3. docRef on translation
  • parse versionIds using getVersionId and parseVersionId on encode/decode
  • Add checks for undefined fields. This mostly happens when we have a list of refs (repeated) where, for some reason that I don't know, versionId can be undefined. Types are generated like that and from what I can see non-primitives in protobufs generate types with | undefined
  • Fix types and tests
  • I took de liberty of renaming fieldRef to propertyRef in translation since fieldRef can be misleading (its not related to Field whatsoever). propertyRef seemed more apt since its actually a 'dotprop' string. I can rollback that change if y'all feel like it

@EvanHahn EvanHahn changed the title feat: standarize refs accross all docs feat: standarize refs across all docs Jul 31, 2024
Copy link
Contributor

@EvanHahn EvanHahn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than a few questions on tests.

test/fixtures/bad-docs.js Outdated Show resolved Hide resolved
test/fixtures/bad-docs.js Outdated Show resolved Hide resolved
test/fixtures/bad-docs.js Outdated Show resolved Hide resolved
test/fixtures/bad-docs.js Outdated Show resolved Hide resolved
@tomasciccola
Copy link
Contributor Author

LGTM other than a few questions on tests.

Thanks for catching this. It was left from when Ref was the same across every docType and with a type field.
Now it doesn't make sense to keep it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: Standardize all refs to include both docId and versionId
3 participants