Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] cucim v22.08 #385

Merged
merged 44 commits into from
Aug 17, 2022
Merged

[RELEASE] cucim v22.08 #385

merged 44 commits into from
Aug 17, 2022

Commits on May 19, 2022

  1. DOC

    raydouglass committed May 19, 2022
    Configuration menu
    Copy the full SHA
    394574a View commit details
    Browse the repository at this point in the history

Commits on May 20, 2022

  1. Merge pull request #281 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored May 20, 2022
    Configuration menu
    Copy the full SHA
    a0be50d View commit details
    Browse the repository at this point in the history
  2. Merge pull request #282 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored May 20, 2022
    Configuration menu
    Copy the full SHA
    e443dde View commit details
    Browse the repository at this point in the history
  3. Merge pull request #285 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored May 20, 2022
    Configuration menu
    Copy the full SHA
    105b21c View commit details
    Browse the repository at this point in the history

Commits on May 23, 2022

  1. Merge pull request #287 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored May 23, 2022
    Configuration menu
    Copy the full SHA
    d50487c View commit details
    Browse the repository at this point in the history
  2. Merge pull request #288 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored May 23, 2022
    Configuration menu
    Copy the full SHA
    ca9a83d View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2022

  1. Merge pull request #298 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored Jun 1, 2022
    Configuration menu
    Copy the full SHA
    74c63ff View commit details
    Browse the repository at this point in the history
  2. Merge pull request #299 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored Jun 1, 2022
    Configuration menu
    Copy the full SHA
    5ce486a View commit details
    Browse the repository at this point in the history
  3. Merge pull request #300 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored Jun 1, 2022
    Configuration menu
    Copy the full SHA
    5167f84 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #301 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored Jun 1, 2022
    Configuration menu
    Copy the full SHA
    3b850f2 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2022

  1. Merge pull request #304 from rapidsai/branch-22.06

    [gpuCI] Forward-merge branch-22.06 to branch-22.08 [skip gpuci]
    GPUtester authored Jun 6, 2022
    Configuration menu
    Copy the full SHA
    7fe1580 View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2022

  1. fix RawKernel bug for canny filter when quantiles are used (#310)

    This PR fixes a bug in `canny` that was leading to failures in `test_use_quantiles`. It also expands that test case via `pytest.parametrize` to cover additional dtypes
    
    The changes to other test classes are merely refactoring to avoid use of `unittest.TestCase` so that `pytest.parametrize` can be used
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #310
    grlee77 authored Jun 16, 2022
    Configuration menu
    Copy the full SHA
    56804d2 View commit details
    Browse the repository at this point in the history
  2. Incorporate bug fixes from skimage 0.19.3 (#312)

    This PR incorporates the two bug fixes from skimage v0.19.3 (released on June 12th) that are relevant to cuCIM.
    
    - bug fix to ensure all color channel share the same histogram bins in `cucim.skimage.exposure.histogram`
    - clipping in warp functions should respect user-specified `cval` during clipping even if it is outside the original image range
    
    There was also a fix to Canny edge detection, but we had already resolved that issue in cuCIM. I did go ahead and add the new test case for it to #310.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #312
    grlee77 authored Jun 16, 2022
    Configuration menu
    Copy the full SHA
    3b4b9eb View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2022

  1. Remove custom compiler environment variables (#307)

    With the PR below merged, we no longer set the `CUDAHOSTCXX` variable in any of our CI images. This PR cleans up some references to `CUDAHOSTCXX`.
    
    
    - rapidsai/gpuci-build-environment#265
    
    Authors:
      - AJ Schmidt (https://github.com/ajschmidt8)
    
    Approvers:
      - Sevag H (https://github.com/sevagh)
      - Gigon Bae (https://github.com/gigony)
      - https://github.com/jakirkham
    
    URL: #307
    ajschmidt8 authored Jun 21, 2022
    Configuration menu
    Copy the full SHA
    c0feac3 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2022

  1. Configuration menu
    Copy the full SHA
    0341bb3 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2022

  1. Configuration menu
    Copy the full SHA
    e62d41d View commit details
    Browse the repository at this point in the history
  2. Merge pull request #344 from grlee77/branch-22.08-merge-22.06

    replaces #306 having manually fixed the conflict in `CHANGELOG.md`
    
    Authors:
       - Ray Douglass (https://github.com/raydouglass)
       - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
       - Mark Sadang (https://github.com/msadang)
    ajschmidt8 authored Jul 26, 2022
    Configuration menu
    Copy the full SHA
    d486ea9 View commit details
    Browse the repository at this point in the history
  3. benchmarks: avoid use of deprecated pandas API (#339)

    This is an update of the previously-approved #290 from branch-22.06 to branch-22.08
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/gbae
      - Gigon Bae (https://github.com/gigony)
      - https://github.com/jakirkham
    
    URL: #339
    grlee77 authored Jul 26, 2022
    Configuration menu
    Copy the full SHA
    e5006ff View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2022

  1. parameterization of filters and features benchmarks (v2) (#322)

    Update of #278 for branch-22.08 with minor additional fixes. I have tested this locally and it seems to be operating as expected.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
      - https://github.com/aasthajh
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #322
    grlee77 authored Jul 27, 2022
    Configuration menu
    Copy the full SHA
    3f7069f View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2022

  1. complete parametrization of cucim.skimage benchmarks (#324)

    This PR expands the approach of #322 to the remaining `cucim.skimage` benchmarks as well
    
    This PR is built on top of a few other open benchmarking PRs, so please review and merge those first: (#322, #323, #290)
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
      - https://github.com/aasthajh
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #324
    grlee77 authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    d03e03f View commit details
    Browse the repository at this point in the history
  2. Update README.md (#315)

    Incorporated definition for cuCIM; cuCIM benefits; applicable file formats; and links to blogs, documentation, and webinars.
    
    Authors:
      - https://github.com/HesAnEasyCoder
      - Gigon Bae (https://github.com/gigony)
    
    Approvers:
      - Gigon Bae (https://github.com/gigony)
      - Gregory Lee (https://github.com/grlee77)
      - https://github.com/jakirkham
    
    URL: #315
    HesAnEasyCoder authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    99919ff View commit details
    Browse the repository at this point in the history
  3. Stain extraction: use a less strict condition across channels when th…

    …resholding (#316)
    
    I think we want to allow pixels with substantial attenuation in **any** of the channels rather than requiring substantial attenuation in **all** channels. The Macenko et. al. reference does not actually explain how the threshold was applied (min, max or mean across channels). However, qualitatively, results looked closer to expected with the `any` condition proposed here when I was working on a pending stain normalization example/demo.
    
    I have also seen third party implementations where there is an RGB->LUV conversion and thresholding is done on the L channel rather than in luminance space. That seems reasonable and perhaps is more intuitive in choice of the threshold number (easier to think in fraction of brightness rather than absorbance units). However, making that change would be a bigger deviation from the publication and would involve some small additional overhead of the RGB->luminance conversion.
    
    I have marked this as breaking as it will affect the estimated stain vectors produced by `stain_extraction_pca`.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - Gigon Bae (https://github.com/gigony)
      - https://github.com/jakirkham
    
    URL: #316
    grlee77 authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    19741b7 View commit details
    Browse the repository at this point in the history
  4. Add Euclidean distance transform for images/volumes (#318)

    ### Overview
    
    closes #319
    
    This PR adds an implementation of `scipy.ndimage.distance_transform_edt`. For each foreground pixel in a binary image, this function computes the minimal Euclidean distance to reach a background pixel. This is a SciPy function rather than a scikit-image one so I have put it under `cucim.core` instead of `cucim.skimage`. Longer term we could move this upstream to CuPy, but I think we want to have a copy here so that we can use it in the near term to implement some of the missing scikit-image functionality. This function is used by the following functions in the scikit-image API, so this PR will help us implement these in future PRs:
    
    `skimage.morphology.medial_axis` (**update**: see #342)
    `skimage.segmentation.expand_labels` (**update**: see #341)
    `skimage.segmentation.chan_vese` (**update**: see #343)
    It is also used in examples related to `skimage.segmentation.watershed` and is needed for implementation of ITK's `SignedMaurerDistanceMapImageFilter` for [itk-cucim](InsightSoftwareConsortium/itk_cucim#13).
    
    The algorithm used here is adapted from the C++ code in the [PBA+ repository](https://github.com/orzzzjq/Parallel-Banding-Algorithm-plus) (MIT-licensed)
    
    ### extensions made to PBA+ kernel source
    - larger sizes can be supported due to runtime code generation of the integer datatypes and encode/decode functions.
    
    ### current known limitations
    - 2D and 3D only (this should cover the most common use cases)
    - The `sampling` argument is not fully supported. This can likely be done with a bit of effort, but will require modifications to the CUDA kernels
    - User-specified output arguments are not currently supported. We can potentially add this in the future
    - The indices returned by `return_indices=True` are equally valid, but not always identical to those returned by SciPy. This is because there can be multiple indices with an identical distance, so which one gets chosen in that case is implementation-dependent.
    
    ### initial benchmarks relative to SciPy
    
    Here `% true` is the percentage of the image that corresponds to foreground pixels. Even for fairly small images there is some benefit, with the benefit becoming two orders of magnitude at larger sizes.
    
    shape      | % true |  cuCIM |  SciPy  | acceleration
    -----------------|--------|---------|---------|--------------
    (256, 256) | 5 | 0.00108 | 0.00353 | 3.25
    (512, 512) | 5 | 0.00108 | 0.01552 | 14.42
    (2048, 2048) | 5 | 0.00252 | 0.34434 | 136.86
    (4096, 4096) | 5 | 0.00765 | 1.58948 | 207.88
    (32, 32, 32) | 5 | 0.00103 | 0.00305 | 2.98
    (128, 128, 128) | 5 | 0.00153 | 0.30103 | 196.26
    (256, 256, 256) | 5 | 0.00763 | 3.17872 | 416.37
    (384, 384, 384) | 5 | 0.02460 | 14.28779 | 580.89
    (256, 256) | 50 | 0.00107 | 0.00430 | 4.01
    (512, 512) | 50 | 0.00109 | 0.01878 | 17.30
    (2048, 2048) | 50 | 0.00299 | 0.39304 | 131.60
    (4096, 4096) | 50 | 0.00896 | 1.84686 | 206.19
    (32, 32, 32) | 50 | 0.00102 | 0.00361 | 3.53
    (128, 128, 128) | 50 | 0.00163 | 0.31657 | 194.66
    (256, 256, 256) | 50 | 0.00914 | 3.35914 | 367.49
    (384, 384, 384) | 50 | 0.03005 | 13.83219 | 460.30
    (256, 256) | 95 | 0.00107 | 0.00344 | 3.22
    (512, 512) | 95 | 0.00108 | 0.01638 | 15.17
    (2048, 2048) | 95 | 0.00314 | 0.36996 | 117.81
    (4096, 4096) | 95 | 0.00943 | 1.90475 | 202.05
    (32, 32, 32) | 95 | 0.00102 | 0.00314 | 3.07
    (128, 128, 128) | 95 | 0.00180 | 0.28843 | 159.80
    (256, 256, 256) | 95 | 0.01073 | 3.23450 | 301.41
    (384, 384, 384) | 95 | 0.03577 | 12.40526 | 346.82
    
    ### other comments
    - can likely reduce memory overhead and improve performance a bit by refactoring some of the pre/post-processing code into elementwise kernels. (e.g. `encode3d`/`decode3d`, etc.)
    
    - (JK) This may be able to leverage CuPy in the future ( cupy/cupy#6919 )
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - Gigon Bae (https://github.com/gigony)
      - https://github.com/jakirkham
    
    URL: #318
    grlee77 authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    1d784ab View commit details
    Browse the repository at this point in the history
  5. Reduce memory overhead and improve performance of normalize_colors_pca (

    #328)
    
    In this PR we change the following line:
    ```Python
    conc_norm = conc_raw * normalization_factors[:, cp.newaxis]
    ```
    
    to instead modify `conc_raw` in-place. The `conc_raw` array is an intermediate array of concentrations, so changing to an in-place operation will not effect user-facing code. The benefit is avoiding an extra copy of the concentrations array. It also avoids an unintended promotion to float64 due to multiplication by a `normalization_factors` array that was potentially higher precision than `conc_raw`.
    
    For a fairly large uint8 input array of shape `(26420, 19920, 3)`, the `conc_raw` array ends up as an ~4GB float32 array. Without the change in this PR there will be a new 8GB float64 `conc_norm` array created.  By avoiding this we substantially reduce memory use. 
    
    We also make sure the `ref_max_conc` and `ref_stain_coeff` inputs to `_normalized_from_concentration` match the precision of the image. Due to this, a 25% improvement in performance was observed (due to float32 rather than float64 computations in the remainder of the `_normalized_from_concentrations` function).
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #328
    grlee77 authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    c3e5a9a View commit details
    Browse the repository at this point in the history
  6. Update PyPI package documentation for v22.06.00 (#311)

    - Update CHANGELOG.md and README.md for PyPI v22.06.00 release
    
    Authors:
      - Gigon Bae (https://github.com/gigony)
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - Gregory Lee (https://github.com/grlee77)
      - https://github.com/jakirkham
    
    URL: #311
    gigony authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    9c29827 View commit details
    Browse the repository at this point in the history
  7. remove unimplemented functions from See Also and fix version numbers …

    …in deprecation warnings (#356)
    
    closes #325
    
    TODO:
    - [x] update DeprecationWarning strings
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #356
    grlee77 authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    3befda7 View commit details
    Browse the repository at this point in the history
  8. Allow CuPy 11 (#357)

    Allows cuCIM to be installed with CuPy 11.
    
    xref: rapidsai/integration#508
    
    Authors:
      - https://github.com/jakirkham
    
    Approvers:
      - Gregory Lee (https://github.com/grlee77)
      - Sevag H (https://github.com/sevagh)
    
    URL: #357
    jakirkham authored Jul 28, 2022
    Configuration menu
    Copy the full SHA
    c961066 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2022

  1. Update README.md (#361)

    Incorporating latest blog that dropped 7/27.
    
    Authors:
      - https://github.com/HesAnEasyCoder
    
    Approvers:
      - https://github.com/jakirkham
      - AJ Schmidt (https://github.com/ajschmidt8)
    
    URL: #361
    HesAnEasyCoder authored Jul 29, 2022
    Configuration menu
    Copy the full SHA
    64e3bb4 View commit details
    Browse the repository at this point in the history
  2. Add __init__.py in cucim.core (#359)

    Hopefully addresses some `import` issues seen around modules in this package.
    
    cc @grlee77
    
    Authors:
      - https://github.com/jakirkham
    
    Approvers:
      - Gregory Lee (https://github.com/grlee77)
    
    URL: #359
    jakirkham authored Jul 29, 2022
    Configuration menu
    Copy the full SHA
    d1b64ad View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2022

  1. Configuration menu
    Copy the full SHA
    397c428 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2022

  1. Configuration menu
    Copy the full SHA
    6f22f2e View commit details
    Browse the repository at this point in the history
  2. Add cucim.skimage.segmentation.expand_labels (#341)

    closes #334
    This adds a simple function that can be used to expand a set of labels in a label image.
    
    It should be reviewed after #318 is merged. The new commits only start from 6241bd2.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #341
    grlee77 authored Aug 1, 2022
    Configuration menu
    Copy the full SHA
    5fee5cd View commit details
    Browse the repository at this point in the history
  3. Performance optimizations to morphological segmentation functions (#340)

    This PR is purely a performance refactor without a change in behavior. The main changes to `morphological_chan_vese` are:
    
    - Use a simplified version of `cupy.gradient` that does not promote all integer types to float64 and omits a couple of division operations by assuming unit spacing. Currently we don't have any places where non-uniform spacing or non first-order edges are needed, but we can always still use `cupy.gradient` for those if they come up in the future. 
    
    - apply kernel fusion via `cupy.fuse()` decorators.
    
    - avoid repeated transfer of footprints to the GPU and reuse the same workspace array for repeated `_curveop` calculations.
    
    Overall, these changes give around 1/3 reduction in run-time for `morphological_chan_vese`.
    
    All other places using `cp.gradient` were also changed to use the simplified `gradient` defined here. These include:
    `cucim.skimage.features.hessian_matrix`
    `cucim.skimage.features.multiscale_basic_features`
    `cucim.skimage.registration.optical_flow_tvl1`
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #340
    grlee77 authored Aug 1, 2022
    Configuration menu
    Copy the full SHA
    31049d9 View commit details
    Browse the repository at this point in the history
  4. Protect against obscure divide by zero error in edge case of `normali…

    …ze_colors_pca` (#327)
    
    This PR updates `normalize_colors_pca` to raise a more helpful error message in the case that there is only a single pixel above the specified `beta` threshold. Currently this leads to a not very helpful `DivideByZeroError` because the degrees of freedom during an internal covariance calculation becomes zero in this case.
    
    This does also change the error type when there are no pixels above the threshold from `ValueError` to `RuntimeError` which seems more appropriate to me.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
      - https://github.com/jakirkham
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #327
    grlee77 authored Aug 1, 2022
    Configuration menu
    Copy the full SHA
    e7f0877 View commit details
    Browse the repository at this point in the history
  5. create SimilarityTransform using CuPy 9.x-compatible indexing (#365)

    closes #364
    
    This tiny fix resolves the local test failures I saw with CuPy 9.6. I also think the updated code is clearer in any case.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #365
    grlee77 authored Aug 1, 2022
    Configuration menu
    Copy the full SHA
    8d7a016 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. add cucim.core.morphology to API docs + other docstring fixes (#367)

    make sure `cucim.core.morphology` module is included in the core API docs
    
    fix typo in references section of `distance_transform_edt` docstring
    
    setup intersphinx link to SciPy docs (used in `distance_transform_edt`)
    
    add `sphinx.ext.mathjax` for LaTeX rendering
    
    fix sphinx warning/typo related to `image_to_absorbance` docstring
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #367
    grlee77 authored Aug 2, 2022
    Configuration menu
    Copy the full SHA
    1f853dc View commit details
    Browse the repository at this point in the history
  2. Add segmentation with the Chan-Vese active contours method (#343)

    closes #335
    
    This PR adds a function for segmentation based on active-contours. Test cases are directly from scikit-image, but the code itself was heavily refactored to make extensive use of kernel fusion via `cupy.fuse()`. I will add benchmark results here soon.
    
    It should be reviewed after #318 is merged. The new commits only start from 502f41a
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - https://github.com/jakirkham
    
    URL: #343
    grlee77 authored Aug 2, 2022
    Configuration menu
    Copy the full SHA
    4814905 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2022

  1. Add a fast histogram-based median filter (#317)

    ### Background
    
    closes #320
    
    This PR adds a fast histogram-based median filter. It is based on a paper by Oded Green [1] as available in [`opencv_contrib`](https://github.com/opencv/opencv_contrib/blob/4.x/modules/cudafilters/src/cuda/median_filter.cu). It is a GPU-based adaptation of an earlier CPU-based publication [2].
    
    This new functionality has been added to the existing `median` function and will be automatically used when possible (although the user can override the automated choice via a new keyword-only `algorithm` parameter.
    
    The CUDA file [here](https://github.com/opencv/opencv_contrib/blob/4.x/modules/cudafilters/src/cuda/median_filter.cu) has a BSD 3-clause license up top, although the repository states that it is Apache 2.0 licensed, so that is likely out of date. We do already list OpenCV's extra modules and the Apache 2.0 license for it in our 3rdparty license folder.
    
    ### References
    
    [1] O. Green, "Efficient Scalable Median Filtering Using Histogram-Based Operations," in IEEE Transactions on Image Processing, vol. 27, no. 5, pp. 2217-2228, May 2018, https://doi.org/10.1109/TIP.2017.2781375.
    
    [2] S. Perreault and P. Hebert, "Median Filtering in Constant Time," in IEEE Transactions on Image Processing, vol. 16, no. 9, pp. 2389-2394, Sept. 2007, https://doi.org/10.1109/TIP.2007.902329.
    
    ### Improvements
    
    There are a number of generalizations in the implementation here as compared to the one in OpenCV:
    - supports odd-sized rectangular footprints as well as square ones
    - supports int8, uint16 and int16 in addition to uint8
    - the dtype used to store the histogram counts is adapted to the filter's footprint size
    - handles many additional boundary extension modes via explicit pre-pad and post-crop (in principle could adapt the kernel for each case to avoid this overhead, but likely not worth the substantial additional effort)
    - automatically switches between the sorting-based and histogram-based implementations depending on kernel size (sorting is much faster for very small kernels while the histogram-based approach is much faster for large ones)
    
    ### Notes/Limitations
    
    The GPU I tested this on has a static shared memory limit of 48k per block. This means that the `HFine` shared memory array can exceed the available GPU memory once there are more than ~12000 gray levels in the histogram. So, for int16 or uint16, we I can only run the algorithm on images where the min/max range does not exceed that value. One fairly common case in medical imaging is DICOM images with 12-bit range (0-4095) that are stored at 16-bit int. This shared memory limit is compatible with this case.
    
    ### Performance
    
    see summary in the next comment
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - Gigon Bae (https://github.com/gigony)
      - https://github.com/jakirkham
    
    URL: #317
    grlee77 authored Aug 3, 2022
    Configuration menu
    Copy the full SHA
    db0026b View commit details
    Browse the repository at this point in the history
  2. more efficient separable convolution (#355)

    closes #321 
    
    This PR adds a different implementation of separable convolution that was adapted from `opencv_contrib`. It is not yet ready for review.
    
    Still needs:
    - [x] support other boundary modes
    - [x] tests
    - [x] ~extension to nD~  extension to 3D
    - [x] wrappers for existing filters to call this one instead of CuPy's
    
    The key changes in this PR are the new files:
    `skimage/filters/_separable_filtering.py`
    `skimage/filters/tests/test_separable_filtering.py`
    
    It is based on the approach taken in OpenCV-contrib's [row](https://github.com/opencv/opencv_contrib/blob/4.6.0/modules/cudafilters/src/cuda/row_filter.hpp) and [column]([https://github.com/opencv/opencv_contrib/blob/4.6.0/modules/cudafilters/src/cuda/column_filter.hpp] ) filters, but also supports:
    - 3D
    - additional dtypes (e.g. complex64)
    - all boundary modes from SciPy
    - not restricted to kernel size <= 32
    - casting behavior to the output matches SciPy rather than OpenCV conventions.
     
    A simpler version of the same approach has long been in the CUDA samples [convolutionSeparable.cu example](https://github.com/NVIDIA/cuda-samples/blob/master/Samples/2_Concepts_and_Techniques/convolutionSeparable/convolutionSeparable.cu). The basic idea is:
    1.) First stage loads the current patch of the image and its boundaries into shared memory
    2.) After synchronization, convolution is performed on the shared memory array.
    
    A lot of CuPy's ndimage code is vendored here for the following reasons:
    - We need any `cupyx.scipy.ndimage` functions we call, like `gaussian_filter`, `uniform_filter`, etc. to dispatch to the new convolution implementation when possible.
    - the `_get_weights_dtype` utility was changed to promote 8 and 16-bit integers to float32 instead of float64 during convolutions.
    - the `_get_output` utility was changed to allocate the output arrays as `empty` rather than `zeros` which is more efficient.
    - the `_run_1d_filters` utility was improved so that it avoids an extra copy when the number of filters is even
    
    I will submit PRs for these to CuPy. The `_get_output` change in particular also impacts other morphology and interpolation functions that we use in cuCIM, so I have vendored those here as well. For reference regarding these non-filtering changes, there is modest performance improvement. Grayscale `erosion` on a 4k image became 10% faster and `resize` of a 4k image to HD became 20% faster.
    
    It is not easy to review kernels based on generated code strings, so I have tried to add pretty comprehensive tests covering many kernel sizes, all boundary modes, various dtype combinations, etc.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - Gigon Bae (https://github.com/gigony)
      - https://github.com/jakirkham
    
    URL: #355
    grlee77 authored Aug 3, 2022
    Configuration menu
    Copy the full SHA
    553418b View commit details
    Browse the repository at this point in the history
  3. Add cucim.skimage.morphology.medial_axis (#342)

    closes #336
    
    This PR adds a function for skeletonization of 2D images via the medial axis transform. 
    
    It should be reviewed after #318 is merged. The new commits only start from 19a6fed.
    
    There is one sequential component to this algorithm that still must be run on the CPU, but the majority of the computations are on the GPU and acceleration is good. I will add benchmark results here soon.
    
    Authors:
      - Gregory Lee (https://github.com/grlee77)
      - https://github.com/jakirkham
    
    Approvers:
      - https://github.com/jakirkham
      - Gigon Bae (https://github.com/gigony)
    
    URL: #342
    grlee77 authored Aug 3, 2022
    Configuration menu
    Copy the full SHA
    c834039 View commit details
    Browse the repository at this point in the history
  4. Support resolution and spacing metadata (#349)

    Add 'tiff.resolution_unit' and 'tiff.x_resolution', 'tiff.y_resolution' metadata to the CuImage object.
    (See https://www.awaresystems.be/imaging/tiff/tifftags/resolutionunit.html)
    
    If `resolution_unit` is not 1 (1 = No absolute unit of measurement), `spacing_units` is `['micrometer', 'micrometer', 'color']` and spacing unit is calculated based on `micrometer`.
    
    **spacing_test.py**
    ```python
    from pprint import pprint
    from cucim import CuImage
    img = CuImage("TCGA-18-3406-01Z-00-DX1_tissue.tif")
    pprint(img.metadata, indent=2, compact=True)
    
    ```
    
    ```bash
    ❯ python spacing_test.py
    { 'cucim': { 'associated_images': [],
                 'channel_names': ['R', 'G', 'B', 'A'],
                 'coord_sys': 'LPS',
                 'dims': 'YXC',
                 'direction': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]],
                 'dtype': {'bits': 8, 'code': 1, 'lanes': 1},
                 'ndim': 3,
                 'origin': [0.0, 0.0, 0.0],
                 'path': '/home/gbae/Downloads/TCGA-18-3406-01Z-00-DX1_tissue.tif',
                 'resolutions': { 'level_count': 6,
                                  'level_dimensions': [ [16034, 10970],
                                                        [8017, 5485], [4008, 2742],
                                                        [2004, 1371], [1002, 685],
                                                        [501, 342]],
                                  'level_downsamples': [ 1.0, 2.0,
                                                         4.000614166259766,
                                                         8.001228332519531,
                                                         16.008296966552734,
                                                         32.040008544921875],
                                  'level_tile_sizes': [ [512, 512], [512, 512],
                                                        [512, 512], [512, 512],
                                                        [512, 512], [512, 512]]},
                 'shape': [10970, 16034, 4],
                 'spacing': [2.0159945487976074, 2.0159945487976074, 1.0],
                 'spacing_units': ['micrometer', 'micrometer', 'color'],
                 'typestr': '|u1'},
      'tiff': { 'model': '',
                'resolution_unit': 'centimeter',
                'software': '',
                'x_resolution': 4960.3310546875,
                'y_resolution': 4960.3310546875}}
    ```
    
    **Test install**
    ```bash
    python -m pip install --force-reinstall --extra-index-url https://test.pypi.org/simple/ cucim==0.0.333
    ```
    
    Address #333 
    
    Signed-off-by: Gigon Bae <gbae@nvidia.com>
    
    Authors:
      - Gigon Bae (https://github.com/gigony)
      - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
      - Gregory Lee (https://github.com/grlee77)
    
    URL: #349
    gigony authored Aug 3, 2022
    Configuration menu
    Copy the full SHA
    39f7c01 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2022

  1. Defer loading of custom.js (#383)

    This PR switches the loading of `custom.js` to `defer` because we will need the entire page to be loading until the methods in this script can even execute correctly.
    
    xref: rapidsai/cudf#11465
    
    Authors:
      - GALI PREM SAGAR (https://github.com/galipremsagar)
    
    Approvers:
      - AJ Schmidt (https://github.com/ajschmidt8)
    
    URL: #383
    galipremsagar authored Aug 4, 2022
    Configuration menu
    Copy the full SHA
    ff7b27b View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2022

  1. bump scikit-image requirement to >= 0.19 (#388)

    closes #363
    by bumping the minimum required scikit-image
    
    Authors:
       - Gregory Lee (https://github.com/grlee77)
    
    Approvers:
       - AJ Schmidt (https://github.com/ajschmidt8)
       - https://github.com/jakirkham
    grlee77 authored Aug 16, 2022
    Configuration menu
    Copy the full SHA
    d673581 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2022

  1. update changelog

    raydouglass committed Aug 17, 2022
    Configuration menu
    Copy the full SHA
    8136f07 View commit details
    Browse the repository at this point in the history