GitHub Test Annotations

[samchungy]

Resolves #638

Description

Adds a custom Jest Reporter which implements the new GitHub annotations API. This adds annotations to the skuba test command.

This will create a separate check run per jest.config displayName

[changeset-bot]

🦋 Changeset detected

Latest commit: ec0ce9e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
skuba	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[samchungy]

There's a slight issue with this approach and will need some good documentation around it.

At the moment if a project uses only a single jest.config file but uses different CLI commands for different tests. eg. yarn test --testPatterns *.int.test.ts and yarn test --testPatterns *.unit.test.ts. There will only be one check run with the name skuba test created with the latter skuba test call overwriting the results of the first. I cannot think of a simple workaround for this unless we change our approach and call to get all the check runs beforehand and create one with a different name eg. skuba test 2. What would happen if they ran two tests at the same time?

I would like to encourage users to create separate jest.config files for different test types eg. for the above scenario we could create 2 separate jest config files.

Eg.
jest.config.int.ts

export default Jest.mergePreset({
    displayName: 'integration',
    testPatterns: ['*.int.test.ts']
})

jest.config.ts

export default Jest.mergePreset({
    displayName: 'unit',
    testPatterns: ['*.unit.test.ts']
})

This will create two separate check runs skuba/test (integration) and skuba/test (unit).

Alternative, you could choose to not label one with a displayName.

Eg.
jest.config.int.ts

export default Jest.mergePreset({
    displayName: 'integration',
    testPatterns: ['*.int.test.ts']
})

jest.config.ts

export default Jest.mergePreset({
    testPatterns: ['*.unit.test.ts']
})

This will alsos create two separate check runs skuba/test (integration) and skuba/test.

[72636c]

Exciting 🙌

[72636c]

I think this displayName-based approach makes sense. We can either encourage users to have multiple config files with distinct displayNames, or a single config file with multiple projects and associated displayNames (if the latter, we should think about how we can better support that style in Jest.mergePreset).

We could hash process.cwd() and process.argv if we wanted to completely avoid clobbering check runs, but I wouldn't bother with that unless we see this displayName approach fall over in practice. Keep in mind this won't enable itself on existing repos until peeps update them to propagate the necessary prerequisites, and they can fix up configs while they are there anyway.

[72636c]

Not for this PR, but I'd be interested to see whether we can extend this with coverage in future. It would be neat if we could compare against the default branch a la Codecov; could we try to retrieve details of the equivalent check run on the head commit for comparison?

[samchungy]

Might be able to grab it from the test result https://github.com/facebook/jest/blob/8f2cdad7694f4c217ac779d3f4e3a150b5a3d74d/packages/jest-test-result/src/types.ts#L95

[72636c]

🙇

[72636c]

Just double checking that this was the intent of running in series?

[samchungy]

Correcto, you wake up early as 👀