Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: invite.encryptionKeys should be required #223

Closed
wants to merge 8 commits into from

Conversation

gmaclennan
Copy link
Member

This unfortunately requires more than just an update to the proto definition because ts-proto does not support making messages required. This means we need to check that ourselves and throw / ignore invites if missing.

There also seem to be a few type errors in src/rpc.js that need fixing. Going to keep this PR open to address those issues.

@gmaclennan gmaclennan linked an issue Aug 24, 2023 that may be closed by this pull request
@gmaclennan gmaclennan self-assigned this Aug 24, 2023
@achou11
Copy link
Member

achou11 commented Sep 6, 2023

the type error that stands out to me is the one regarding conflicting type definitions of the transformer and the protobuf interface that ts-proto outputs, basically what's described in stephenh/ts-proto#881. The reasoning in that issue makes sense if you're strictly working with TS files, but in our case we're not so attempting to create a JSDoc typedef out of convenience doesn't work unless:

  1. you rename the import of the transformer e.g. import { Invite as InviteTransformer } from '../generated/rpc.js'
  2. avoid creating a typedef and inlining the import(...) bit each time you want to reference it in jsdoc
  3. name the created typedef something different

a quick glance at the ts-proto options didn't surface anything immediately helpful, but maybe with fresher eyes there will be something that works. otherwise, gotta pick our poison with one of the above options I guess 😄

Another thing I was thinking is that maybe we should create another directory e.g. src/transformers where we can use that as the source of imports for anything related to the proto stuff. for example, the rpc one would be similar to the below and any code that needs to work with the proto stuff would import from here instead of src/generated:

import {
  Invite as InviteTransformer,
  InviteResponse as InviteResponseTransformer,
  InviteResponse_Decision as InviteResponse_DecisionTransformer,
} from '../generated/rpc.js'

// 👇 TS automatically exports typedefs
/** @typedef {import('../generated/rpc.js').Invite} Invite */
/** @typedef {import('../generated/rpc.js').InviteResponse} InviteResponse */
/** @typedef {import('../generated/rpc.js').InviteResponse_Decision} InviteResponse_Decision */

export {
  InviteTransformer,
  InviteResponseTransformer,
  InviteResponse_DecisionTransformer,
}

yes, it's boilerplate-y and somewhat verbose, but this approach would also provide a location for us to override/customize the encode/decode functions, which we may need to do based on Gregor's assessment of the original issue

EDIT: even with the above, it's still kind of iffy since the TS will show that the typedef could potentially reflect either the interface or the transformer (although it usually does what we want and uses the interface). think the ideal solution would be for ts-proto to generate a different name for the transformer compared to the interface

EDIT: Maybe the simplest solution is to just prefix all of the names of imported interface types with I (letter i capitalized) e.g. /** @typedef {import('../generated/rpc.js').Invite} IInvite */...

@achou11 achou11 self-requested a review September 7, 2023 21:14
Copy link
Member

@achou11 achou11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we wanted also add changes to handle/ignore validation errors from the encode/decode calls so I haven't updated relevant callsites yet.

@@ -103,7 +126,7 @@ test('Send invite and already on project', async (t) => {
replicate(r1, r2)
})

test('Send invite with encryption key', async (t) => {
test('Send invite with encryption keys', async (t) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps this test should be renamed, since it used to assume that the providing some encryption keys was optional

Comment on lines +22 to +32
// TODO: Ideally this asserts that `message.encryptionKeys` is non-nullable but TS isn't cooperating,
// and we can't add the assert annotation directly here because this needs to return a value
/**
* @param {import('../generated/rpc.js').IInvite} message
* @param {Parameters<typeof Invite['encode']>[1]} [writer]
* @returns {ReturnType<typeof Invite['encode']>}
*/
function inviteEncode(message, writer) {
inviteValidate(message)
return Invite.encode(message, writer)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guess the assert is more useful if this function were called in contexts where the input is not controlled by us, which isn't the case at the moment. still, an inconvenient behavior of TS

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the editor experience with this isn't as ideal as I'd want it to be (the redefined interfaces aren't always expanded to the full definition), so just recognizing that with this comment 😄

// 1. Renaming the exported interfaces so that they're prefixed with `I`
// 2. Updating generated TS source files to re-import relevant transformers (since it gets lost in the renaming step)
// https://github.com/digidem/mapeo-core-next/pull/223#issuecomment-1709125859
async function fixTsProtoOutput(directoryPath) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

happy to explain this in more detail if it's not clear enough from the comments

@achou11 achou11 marked this pull request as ready for review September 7, 2023 21:24
@achou11
Copy link
Member

achou11 commented Sep 11, 2023

@gmaclennan would you mind taking a look at the recent changes I added? To summarize:

  • generally fixed the types issues that you alluded to. involved using https://github.com/dsherret/ts-morph after ts-proto does its thing to resolve output issues that aren't fixable with ts-proto.

  • adds a transformers directory that's meant to be a home for adjusting and/or re-exporting generated exports from ts-proto. only does rpc for now since that's relevant for the original issue. general idea is for the rest of the library code to import from that file instead of the generated one directly (although maybe with some exceptions)

@gmaclennan
Copy link
Member Author

Replaced by #260

@gmaclennan gmaclennan closed this Sep 12, 2023
@gmaclennan gmaclennan deleted the fix/invite-encryption-keys branch October 26, 2023 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invite encryptionKeys should be required
2 participants