Fix wheel relocation issues

[andfoy]

No description provided.

[codecov]

Codecov Report

Merging #2777 into master will decrease coverage by 0.08%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2777      +/-   ##
==========================================
- Coverage   73.31%   73.22%   -0.09%     
==========================================
  Files          99       96       -3     
  Lines        8724     8446     -278     
  Branches     1373     1320      -53     
==========================================
- Hits         6396     6185     -211     
+ Misses       1909     1859      -50     
+ Partials      419      402      -17     

Impacted Files	Coverage Δ
torchvision/ops/feature_pyramid_network.py	75.82% <0.00%> (-15.39%)	⬇️
torchvision/models/detection/rpn.py	93.16% <0.00%> (-0.71%)	⬇️
torchvision/models/detection/faster_rcnn.py	81.17% <0.00%> (-0.22%)	⬇️
torchvision/io/__init__.py	89.65% <0.00%> (ø)
torchvision/ops/__init__.py	100.00% <0.00%> (ø)
torchvision/models/detection/__init__.py	100.00% <0.00%> (ø)
torchvision/ops/focal_loss.py
torchvision/models/detection/retinanet.py
torchvision/models/detection/anchor_utils.py
torchvision/models/detection/backbone_utils.py	100.00% <0.00%> (+5.71%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2831f11...f5917e2. Read the comment docs.

[andfoy]

I have bad news here, we need to compile FFmpeg from scratch on the manylinux image in order to have manylinux wheel compatibility

[fmassa]

Ok. I propose that we first fix the issues with libjpeg / libpng, merge this, and then fix ffmpeg in a follow-up PR

[andfoy]

We need to compile libPNG ourselves as well

[andfoy]

Regarding the Windows testing errors, it seems that we're picking the global conda ffmpeg binaries instead of the wheel ones

[andfoy]

My solution on this case would be the following:

1. Compile the pyd for video_reader
2. Copy the FFmpeg libraries and rename them
3. Update the pyd to point to the new binaries
4. Include those as part of both the conda recipe and the wheels

This seems relevant: https://github.com/njsmith/machomachomangler

[andfoy]

I tried to rename avcodec-58.dll to avcodec-asdasd-50.dll on video_reader.pyd, however, it seems that PyAV dlls are picking avcodec-58.dll from conda, that's because C:\Users\circleci\project\env\Library\bin is first on PATH, and PyAV wheels are not renaming the packaged DLLs inside the wheels. That means that PyAV wheels are the culprit here, since those should be loading their own binaries to prevent clashes

[vfdev-5]

PyAV is not a hard dependency, right ? For windows, we can put "experimentatl" mention in the docs, put try/catch while it is imported and include a detailed instruction on what to do instead.

[fmassa]

My take on this: let's separate this into two PRs: one with only libjpeg / libpng, and one with FFmpeg. This way we can start merging the libjpeg / libpng bits and testing it, while we iterate on FFmpeg.

EDIT: I'm instead disabling FFmpeg on Windows in this PR to move forward, and we fix the rest in a follow-up PR

[fmassa]

Test failures are unrelated, merging this to unblock. We should still fix FFmpeg on Windows and Linux

[vfdev-5]

Manual tests:

• Colab linux+pip, io.image works fine : https://colab.research.google.com/drive/1DQbZfdA9N1IGuuRVS37hvy9BKtLboxxI
• MacOSX 10.13 pip binaries, io.image works fine
• MacOSX 10.13 conda binaries, io.image works fine

pip_install_req.sh

#!/bin/bash

python -m pip uninstall -y torchvision
python -m pip uninstall -y torch
conda uninstall -y torchvision
conda uninstall -y pytorch

python -m pip install --pre --upgrade torch torchvision -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html

python -m pip list | grep torch

pip install opencv-python

conda_install_req.sh

#!/bin/bash

pip uninstall -y torchvision
pip uninstall -y torch
conda uninstall -y torchvision
conda uninstall -y pytorch

conda install -y pytorch torchvision -c pytorch-nightly

conda list | grep torch

pip install opencv-python

test_io_image.py

import cv2
from PIL import Image

import torch, torchvision
from torchvision.io.image import read_image

print("torch.ops.load_library(torchvision.io.image.ext_specs.origin):",
      torch.ops.load_library(torchvision.io.image.ext_specs.origin))

im1 = cv2.imread("test-image.jpg")
print("Opencv:", im1.shape, im1.dtype)

im2 = Image.open("test-image.jpg")
print("PIL:", im2.size, im2.mode)

tensor_image = read_image("test-image.jpg")
print("tensor image info: ", tensor_image.shape, tensor_image.dtype)

[fmassa]

Awesome, thanks a lot for checking this @vfdev-5 !