Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#1756] Add JSON validation and migrate api.js to TypeScript #1903

Merged
merged 15 commits into from
Feb 21, 2023

Conversation

vvidday
Copy link
Contributor

@vvidday vvidday commented Feb 10, 2023

Fixes #1756
Should not be merged before #1905

Proposed commit message

Currently, the objects produced by reading from summary.json,
authorship.json and commits.json are not validated against any schema.
This results in these objects having an implicit any type, which might
lead to type errors during runtime.

Let's add support for JSON validation with the zod library and migrate
the file where the JSON files are parsed, api.js, to TypeScript. This
will increase type safety and enable the migration of other files.

Other information

Organization

The files containing types are in the src/types folder, with the following layout:

└── types/
    ├── zod/
    │   ├── authorship-type.ts
    │   ├── commits-type.ts
    │   └── summary-type.ts
    ├── types.ts
    └── window.ts

The files in the zod subdirectory are zod schemas that are used to validate the parsed objects from authorship.json, commits.json and summary.json, whereas the other two files contain regular typescript interfaces, hence the distinction. window.ts contains type definitions for all the global variables stored in the window object (window.XXX), and all the other 'normal' types are in types.ts. Please let me know if you have any suggestions for better organization!

api.ts

Changes to this file were mostly related to adding types/typecasts to make it work with the typescript compiler. The only functional change is the three xxxSchema.parse(json) under api.loadSummary, api.loadCommits and api.loadAuthorship. This change will make it so that if the object doesn't match the schema (which can be either due to a modified .json file or a change to the schema in the backend of any of the three .json files), the app will throw an error instead of passing on the data. This will make it so the error is surfaced immediately instead of being passed on to other components and silently breaking elsewhere.

The reason why I'm migrating the whole file in this PR is because adding JSON validation requires the migration of most of the file (everything in window.api), and the only 'extra' aspects are just the type definitions of the other global variables (window.XXX). If we want to separate this into 2 PRs, it will be a bit weird since we will be migrating half the file, and as far as I can tell there isn't an easy way to get typescript to ignore a section of the file (see microsoft/TypeScript#19573)

Changes to eslint

The plugin added in line 9 allows proper resolving of typescript file imports, the change of rule from line 51 onwards makes the linter allow us to import typescript files like import ... from ../file instead of import ... from ../file.ts (which throws an error). The side effect is that the typescript imports in the .vue files must also be changed to remove the .ts, which is why this PR also modifies vue files. Although, if it is better, I can temporarily ignore the rule here and open a separate PR to change this.

@vvidday vvidday marked this pull request as ready for review February 10, 2023 18:28
@vvidday vvidday requested a review from a team February 10, 2023 18:29
Copy link
Member

@ckcherry23 ckcherry23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Learned a lot from your code as I don't have much prior experience using TypeScript! Regarding file structuring, I think we could move the files user.ts and segment.ts from the utils directory to the types directory for better organization.

frontend/src/types/types.ts Outdated Show resolved Hide resolved
frontend/src/utils/api.ts Outdated Show resolved Hide resolved
frontend/src/utils/api.ts Outdated Show resolved Hide resolved
frontend/src/utils/api.ts Outdated Show resolved Hide resolved
Copy link
Member

@sikai00 sikai00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSON validation addition looks good to me! Although, I am of the opinion that the changes to eslint should be in a separate PR following the good practice of limiting each PR to tackling a single issue

@vvidday
Copy link
Contributor Author

vvidday commented Feb 11, 2023

Learned a lot from your code as I don't have much prior experience using TypeScript! Regarding file structuring, I think we could move the files user.ts and segment.ts from the utils directory to the types directory for better organization.

I agree, that would definitely help the organization/structure of the files! Since it will also require changing some .vue files due to imports, I think it will be better to do the move in a separate PR to limit the scope of this one.

@vvidday
Copy link
Contributor Author

vvidday commented Feb 11, 2023

The JSON validation addition looks good to me! Although, I am of the opinion that the changes to eslint should be in a separate PR following the good practice of limiting each PR to tackling a single issue

Thanks, I'll open a separate PR for the eslint changes. For now, I added two lines in .eslintrc.json to ignore the import errors, to let CI continue with the cypress tests/deployments. I'll remove those lines once that PR is merged and I can merge the changes into this branch.

@vvidday vvidday requested a review from a team February 11, 2023 04:26
Copy link
Contributor

@zhoukerrr zhoukerrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really impressive work given how fast you did it. I like the organisation of the files that you added. Codes are also arranged in an easy-to-follow manner.

One small request will be to add the function parameter types wherever appropriate. That way when the functions are called, we can avoid any type error directly. Do let me know your thoughts as well.

frontend/src/utils/api.ts Outdated Show resolved Hide resolved
frontend/src/utils/api.ts Show resolved Hide resolved
Copy link
Contributor

@zhoukerrr zhoukerrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work. LGTM!

frontend/src/utils/api.ts Show resolved Hide resolved
@dcshzj
Copy link
Member

dcshzj commented Feb 21, 2023

@vvidday Sorry, there are some file conflicts in this branch. Are you able to resolve them?

@vvidday
Copy link
Contributor Author

vvidday commented Feb 21, 2023

@dcshzj Sorry, have just resolved the conflicts!

@dcshzj dcshzj merged commit ebd6f81 into reposense:master Feb 21, 2023
@github-actions
Copy link
Contributor

The following links are for previewing this pull request:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for json validation
7 participants