Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect and warn about incorrectly used ignore-comments #325

Merged
merged 9 commits into from
May 30, 2024

Conversation

RKushnir
Copy link
Contributor

@RKushnir RKushnir commented Mar 27, 2024

This PR addresses the issue of accidentally ignoring too much code coverage. The issue happens when you add coveralls-ignore-start but forget to add the corresponding -ignore-stop. With this change, excoveralls is going to discard the un-closed ignore-block (i.e. not ignore the coverage) and give you a warning, so that you can more easily spot the mistake.

Besides the main goal, this change also adds a few other useful warnings to let you know about redundant or invalid ignore-comments, which I previously outlined. This is an optional part, and we could remove it or split it into a separate PR if that would be more appropriate.

The most interesting part for review is ignore.ex and ignore_test.exs.

Here's what the warnings look like in the console output
image

The PR is unfinished, it's missing some of the final touches: some tests, changelog entry, version bump, readme update. But it's feature-complete and so I would love to get early feedback, in case I need to change anything before I polish it. I'm looking forward to your thoughts and suggestions.

Resolves #197.

@parroty
Copy link
Owner

parroty commented Apr 1, 2024

Thank you for the PR 🙇 . I think it makes sense to emit warning for avoiding unexpected behavior. It would be beneficial as long as existing behavior is maintained (just providing warning and doesn't throw error?)

@RKushnir
Copy link
Contributor Author

RKushnir commented Apr 1, 2024

@parroty Thanks for the feedback. What about the coverage? Should we ignore coverage if there is only ignore-start without ignore-stop? (existing behavior)

@parroty
Copy link
Owner

parroty commented Apr 15, 2024

@RKushnir I believe it's already a large change, so I am wondering taking a reasonably small step (while maintaining existing behavior) would be an option? (rather than extending coverage?), but looking for your opinion, too.

@RKushnir
Copy link
Contributor Author

@parroty I agree. I think it's already a good progress if we just show the warnings. I'll try to update the PR soon.

Later we can add some config option e.g. --warnings-as-errors to fail the build if there are any warnings.

@RKushnir
Copy link
Contributor Author

@parroty I updated the PR: reverted the behavior change as we discussed, added more tests and a changelog entry. Unless I missed something, it should be ready now. Thank you for your help.

@parroty
Copy link
Owner

parroty commented May 27, 2024

Thanks and very sorry about taking time to merge. I'll try merge and release during this week 🙇 .

@parroty parroty merged commit a530769 into parroty:master May 30, 2024
2 checks passed
tonyrud added a commit to Vetspire/excoveralls that referenced this pull request Sep 27, 2024
* Cobertura now handles defprotocol and defimpl definitions (parroty#306)

* Cobertura now handles defprotocol and defimpl definitions

* Cobertura, catch all if module type is unknown

* Bump version and update CHANGELOG

* Add Cobertura docs to README.md (parroty#312)

* Update Elixir requirement to 1.11+ (parroty#316)

* Update Elixir requirement to 1.11+

* Update deps

* Replace hackney with httpc (parroty#311)

* Replace hackney with httpc

* SSL options

* FIXUP

* Cache fixed

* Aaaah, caching again

* FIXUP

* Add missing apps to :extra_applications

* Add better check for :public_key

* Bump version and update CHANGELOG

* Fix lcov 2.0 source file handling (parroty#315)

Prior to 2.0 being released, `genhtml` was much better about handling
source files from the following paths:

* `test/support/some_helper.ex`
* `lib/foo/bar.ex`

But after 2.0 was released, when rendering with `genhtml` the paths
would be mangled and look like the following:

* `test/support/test/support/some_helper.ex`
* `lib/foo/lib/foo/bar.ex`

I have tried in vain with many permutations of `--prefix` when running
`genhtml` but the ultimate fix that made all of this go away was using
the absolute path for the source file (`SF`).

* Update CHANGELOG

* Remove erroneous line in ExCoveralls.poster (parroty#318)

* Import `.coverdata` after test run and improve documentation (parroty#309)

* Import `.coverdata` after test run and improve docs

* Update README table of contents

* Update README based on review suggestion

Co-authored-by: Alberto Sartori <alberto.sartori.as@gmail.com>

---------

Co-authored-by: Alberto Sartori <alberto.sartori.as@gmail.com>

* Update CHANGELOG

* Accept custom http options (parroty#319)

* Accept custom http options

* Add HTTP options docs to README

* Bump version and update CHANGELOG

* Always floor coverage instead of rounding (parroty#310)

* Always floor coverage instead of rounding

We do not want to report a 100% coverage when there are lines
that are not covered.

* Add option to restore previous ceil coverage behaviour

* Bump version and update CHANGELOG

* Update README examples (parroty#320)

* Use explicit steps to remove 1.16 deprecation warning (parroty#322)

* Update CHANGELOG and bump version

* Detect and warn about incorrectly used ignore-comments (parroty#325)

* Detect and warn about incorrectly used ignore-comments

Resolves parroty#197.

* Keep existing ignoring behavior

* Improve formatting

* Test more ignore-related warnings

* Remove warning in the case of ignore-next-line at the EOF

* Test the warning output

* Add a changelog entry

* Adjust test descriptions

---------

Co-authored-by: Roman <205906+RKushnir@users.noreply.github.com>

* Fix Elixir 1.17 single-quoted string warning (parroty#327)

* Bump version and update CHANGELOG

* add missing step for Cobertura's range (parroty#329)

Without this change, the output is full of warnings such as
```
warning: negative steps are not supported in String.slice/2, pass 44..-1//1 instead
```

* Update CHANGELOG and version

* Revert "add missing step for Cobertura's range (parroty#329)" (parroty#330)

This reverts commit 00a96c4.

* Revert version change

* Update Range to use function syntax (parroty#332)

* add missing step for Cobertura's range

* Update Range to use function syntax

* run tests in elixir 1.17

* fix test with relative path

---------

Co-authored-by: parroty <parroty@users.noreply.github.com>

* Update CHANGELOG and version

---------

Co-authored-by: Rodrigue Villetard <rodrigue@villetard.tech>
Co-authored-by: parroty <parroty@users.noreply.github.com>
Co-authored-by: Artem Solomatin <artem00298@gmail.com>
Co-authored-by: Andrea Leopardi <an.leopardi@gmail.com>
Co-authored-by: Matthew Johnston <warmwaffles@gmail.com>
Co-authored-by: Zach Allaun <zach.allaun@gmail.com>
Co-authored-by: Alberto Sartori <alberto.sartori.as@gmail.com>
Co-authored-by: Victor Rodrigues <rodrigues@users.noreply.github.com>
Co-authored-by: gitneko <67227083+gitneko@users.noreply.github.com>
Co-authored-by: Roman <RKushnir@users.noreply.github.com>
Co-authored-by: Roman <205906+RKushnir@users.noreply.github.com>
Co-authored-by: Hans Krutzer <git@pixelspaceships.com>
Co-authored-by: Juan Peri <eternoperegrino@gmail.com>
Co-authored-by: Kenta Nakase <1172471+parroty@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fail coverage if any file contains an unmatched coveralls-ignore-start
2 participants