-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom reporters need the way to provide metadata for tests #1951
Comments
Can you be more specific about usecase, and what exactly do you want? I do not know what Can you use context? |
@sheremet-va Hello. We just need something to put metadata during tests and then consume this metadata in our custom reporter. How i think it would look. import { it, addMeta } from 'vitest'
it('should work', ({ foo }) => {
addMeta("metaString")
addMeta({type:"metaObj"})
}) export class CustomReporter implements Reporter {
onFinished(files?: File[]) {
console.log(files[0].tasks[0].meta) // ["metaString", {type:"metaObj"}]
}
} |
We cannot do this with a global function, since we don't now what test is running with Why can't you use context? it('some test', ({ context }) => {
context.type = 'meta_obj'
}) We can technically add it('some test', ({ setContext }) => {
setContext({ type: 'myObj' })
}) Right now context is not transferred to reporter, of course. But it can be implemented. |
I definitely can. I thought this solution was limited by the fact that context is not accessible in the reporter 😅
🔥🔥🔥 |
Although I think it might be a breaking change, because you wouldn't be able to add a function on context (functions cannot be transferred between worker and main thread). So maybe indeed we need a new concept for this. I like the name |
i like the way it implemented in playwright. there you can do thing like test("basic test", async (testArgs, testInfo) => {
testInfo.attachments.push({
name: "screenshot",
body: await testArgs.page.screenshot(),
contentType: "image/png",
});
testInfo.annotations.push({
type: "issue-link",
description: "https://github.com/vitest-dev/vitest/issues/1951",
});
}); Wonder if we can do same thing like it("basic test", (context, testInfo) => {
testInfo.metadata.push("something")
}); |
This also allow us to access vite and vitest config from test. it("basic test", (context, testInfo) => {
testInfo.viteConfg
testInfo.vitestConfg
}); |
We already provide test('some test', ({ meta }) => {
console.log('metadata', meta)
})
Vitest config is part of Vite config. It's also not fully possible, because it needs to be serializable, but some properties, like |
Is it not internal-only thing? It's ok to modify this meta from inside of test? |
ofc we also need to expose the meta to reporter |
Well, it's not used internally. Vitest has direct access to it without needing context, but I guess the contents should be readonly (although it's not imposed). |
We have a similar use-case for "sending test information to a custom reporter". We're using Vitest for a university course, and all unit tests have a lot of helpful information for the students. We basically have an array of "steps", and then the reporter prints both the succeeded steps, and the (frequently customized) error message. Currently, the only way of getting such information to the reporter is by doing horrible hacks. We've resorted to
In an ideal world, we could use the context to send over that information. |
We need to expose meta to the reporter. Currently only the "result" is transported to the reporter. We cannot reuse Also, it's only possible to pass down clonable data. I'm not sure if we can impose it with typescript. |
Is this resolved by #3449? |
Yes, you are right! |
Wow!!! That's awesome will check later |
Clear and concise description of the problem
In allure team want to create custom reporter for vitest but there is no way to provide metadata to each test such as linked issue on another important data.
Also we need the way to consume that metadata from reporter API.
Suggested solution
for example playwright provide custom annotation API
Alternative
Maybe we can provide way to send messages to reporter.
Additional context
No response
Validations
The text was updated successfully, but these errors were encountered: