Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: versioned protobufs #32

Merged
merged 18 commits into from
Mar 7, 2023
Merged

feat: versioned protobufs #32

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
daa2118
feat: minor refactoring. move formatSchemaType to utils.js
Feb 14, 2023
d2ae537
chore: update schemasPrefix.js
Feb 16, 2023
cae214d
feat: version protobufs. contemplate invalid schemaVersion on tests
Feb 21, 2023
3e33597
feat: fromPartial on encoding to allow optional lists on input
Feb 21, 2023
5668f73
feat: throw error if invalid type_schemaVersion on decoding
Feb 21, 2023
0527860
feat: add formatSchemaType on utils to get key to schemas
Feb 21, 2023
4bab3a0
feat: add preset and filter proto and make semi-working example for both
Feb 22, 2023
549197e
feat: build versioning of schemas from filesystem instead of on schemas
Feb 22, 2023
89965ac
feat:
Feb 22, 2023
4348219
chore: solved minor type error
Feb 22, 2023
17750cf
feat: handle type casing with schemaPrefix
Feb 23, 2023
88653d8
chore: logic error on `inheritsFromCommon`
Feb 23, 2023
2193743
feat: add tests for many doc types. Test present fields and values
Feb 28, 2023
ffd7b25
feat: add coreOwnership and device schema
Mar 1, 2023
760a9fe
feat: add schema for role
Mar 1, 2023
604161c
feat: add testing for validation of JSONSchema docs
Mar 1, 2023
670cc9e
feat: created_at is saved as Timestamp on protobuf
Mar 1, 2023
4cdcb4a
feat: timestamp is saved as Timestamp on protobuf
Mar 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions examples/schema_test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,9 @@ import { randomBytes } from 'node:crypto'

const obj = {
id: randomBytes(32).toString('hex'),
type: 'Observation',
schemaVersion: 5,
type: 'observation',
schemaVersion: 4,
created_at: new Date().toJSON(),
links: [],
refs: [],
attachments: [],
tags: {},
metadata: {
manual_location: true,
},
Expand Down
7 changes: 2 additions & 5 deletions index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,11 +157,8 @@ export const encode = (obj) => {
schemaVersion: obj.schemaVersion,
})
const record = jsonSchemaToProto(obj)
const protobuf = ProtobufSchemas[
`${formatSchemaType(obj.type)}_${obj.schemaVersion}`
]
.encode(record)
.finish()
const partial = ProtobufSchemas[key].fromPartial(record)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Using fromPartial to initialized lists to an empty list (since optional repeated doesn't exists), means initializing every non passed field to the initial default value, which means required fields that are missing don't get catched...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could catch this before it gets to the encode step. Is this only applicable to required arrays?

const protobuf = ProtobufSchemas[key].encode(partial).finish()
return Buffer.concat([blockPrefix, protobuf])
}

Expand Down
2 changes: 1 addition & 1 deletion proto/observation/v4.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ message Observation_4 {
optional float lon = 10;
repeated google.protobuf.Struct refs = 11;
repeated google.protobuf.Struct attachments = 12;
google.protobuf.Struct tags = 13;
optional google.protobuf.Struct tags = 13;
message Metadata {
optional bool manual_location = 1;

Expand Down