Provider parameter needs to be set when instantiating InferenceSession in ONNXAdaptiveModel

[cjb06776]

Describe the bug
More recent versions of Onnx require the provider parameter to be explicitly set when creating a onnxruntime.InferenceSession, which results in errors currently when loading an ONNXAdaptiveModel.

Error message

  File "/usr/local/lib/python3.6/dist-packages/farm/modeling/adaptive_model.py", line 653, in load
    onnx_session = onnxruntime.InferenceSession(str(load_dir / "model.onnx"), sess_options)
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 335, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 364, in _create_inference_session
    "onnxruntime.InferenceSession(..., providers={}, ...)".format(available_providers))

ValueError: This ORT build has ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'] enabled. Since ORT 1.9, you 
are required to explicitly set the providers parameter when instantiating InferenceSession. For example, onnxruntime.InferenceSession(..., providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'], ...)

I've made the below change to ONNXAdaptiveModel.load in my local environment to workaround the problem, the change:

• Sets a default CUDAExecutionProvider or CPUExecutionProvider provider depending on the device type passed in, if no provider(s) are specified.
• Optionally allow passing in of a custom provider list

Can create a PR with this if you like...

   @classmethod
    def load(cls, load_dir: Union[str, Path], device: str, **kwargs):  # type: ignore
        """
        Loads an ONNXAdaptiveModel from a directory.

        :param load_dir: Location where the ONNXAdaptiveModel is stored.
        :param device: The device on which this model will operate. Either "cpu" or "cuda".
        """
        load_dir = Path(load_dir)
        import onnxruntime
        sess_options = onnxruntime.SessionOptions()
        # Set graph optimization level to ORT_ENABLE_EXTENDED to enable bert optimization.
        sess_options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
        # Use OpenMP optimizations. Only useful for CPU, has little impact for GPUs.
        sess_options.intra_op_num_threads = multiprocessing.cpu_count()
-       onnx_session = onnxruntime.InferenceSession(str(load_dir / "model.onnx"), sess_options)
+       providers = kwargs.get("providers", ["CPUExecutionProvider"] if device.type == "cpu" else ["CUDAExecutionProvider"])
+       onnx_session = onnxruntime.InferenceSession(str(load_dir / "model.onnx"), sess_options, providers=providers)

[julian-risch]

Hi @cjb06776 thanks for pointing out that issue and for suggesting a fix already. We'd really appreciate it if you could create a PR, yes please. 👍 Would these changes be backwards compatible in the sense that users with an older ONXX version could also work with the changed code? Could you please check that in your PR? Thank you!