-
Notifications
You must be signed in to change notification settings - Fork 460
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: partial reports can be submitted #3179
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -121,7 +121,7 @@ comply with the following rules: | |
- `mode`: it is the mode of the implementation used to run the tests. If different | ||
from `default`, it must correspond to a specific setup of the implementation. | ||
- Test result: in order to have a report valid to be accepted, all the profiles | ||
need to have the `result` fields (core and extended) set to `success`. It means | ||
that all the core conformance tests have been successfully run as well as all | ||
the tests related to the supported extended features. No reports with partial | ||
or failing results can be accepted. | ||
need to have the `result` fields (core and extended) set to `success` or `partial`. | ||
It means that all the core conformance tests have been successfully run or properly | ||
skipped as well as all the tests related to the supported extended features. No | ||
reports with failing results can be accepted. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think there's another reason for partial conformance - the tests may all pass, but the reproduction required some kind of additional unexpected steps. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 I reworded this section to include this information |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like we probably need to include some guidelines for how we handle
partial
conformance results. Maybe it's sufficient to just say that they'll be displayed separately?