Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor AUPRO metric #991

Merged
merged 9 commits into from
Apr 5, 2023

Conversation

triet1102
Copy link
Contributor

@triet1102 triet1102 commented Mar 29, 2023

Description

Hi, this is a proposal PR for the disscusion here #974

  • Before, we could not provide a mask for AUPRO calculation.
  • Now, a binary inspection_mask can be provided during initialization to filter out points outside the mask.

Provide a summary of the modification as well as the issue that has been resolved. List any dependencies that this modification necessitates.

  • Split the _compute() function into 2 sub-functions: perform_cca() and compute_pro()
    • The perform_cca() function performs Connected Component Analysis on the target
    • The compute_pro() function computes the PRO curve, with the ability to filter unnecessary points by using the inspection_mask

Changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which refactors the code base)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist

  • My code follows the pre-commit style and check guidelines of this project.
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing tests pass locally with my changes
  • I have added a summary of my changes to the CHANGELOG (not for minor changes, docs and tests).

@codecov
Copy link

codecov bot commented Mar 29, 2023

Codecov Report

Patch coverage: 75.00% and project coverage change: +0.13 🎉

Comparison is base (27adbae) 81.19% compared to head (8d0433a) 81.32%.

❗ Current head 8d0433a differs from pull request most recent head 77376d9. Consider uploading reports for the commit 77376d9 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #991      +/-   ##
==========================================
+ Coverage   81.19%   81.32%   +0.13%     
==========================================
  Files         176      176              
  Lines        6818     6813       -5     
==========================================
+ Hits         5536     5541       +5     
+ Misses       1282     1272      -10     
Impacted Files Coverage Δ
src/anomalib/utils/metrics/aupro.py 85.04% <75.00%> (-2.46%) ⬇️

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@ORippler ORippler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes beyond the pure refactoring discussed here. To achieve the refactor:

  1. Remove inspection_mask (keep it in your private class that inherits from AUPRO)
  2. Shift the flattening into _compute.

Doing so, you will be able to overload the __init__ and the _compute functions when inheriting from AUPRO and anomalib does not need to support masking.

@triet1102
Copy link
Contributor Author

@ORippler Got it, thank you a lot for the feedback! 😊

Copy link
Contributor

@ORippler ORippler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now

Copy link
Contributor

@samet-akcay samet-akcay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your contribution

@triet1102 triet1102 changed the title Compute AUPRO metric with inspection_mask Refactor AUPRO metric Apr 4, 2023
Copy link
Contributor

@djdameln djdameln left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@djdameln djdameln merged commit 730bdb6 into openvinotoolkit:main Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants