Parameters are optimized out even if it is a needed return value

[ganler]

Describe the issue

The simplified model exported from PyTorch:

"""
graph torch_jit (
  %i0[FLOAT, 1x1x2x2]
) initializers (
  %/Cast_1_output_0[INT64, 8]
  %o2[FLOAT, 1]
) {
  %o0 = Pad[mode = 'reflect'](%i0, %/Cast_1_output_0)
  %o1 = Min(%o2, %i0)
  return %o0, %o1, %o2
}
"""

When running in ONNXRuntime with default optimization, the result o2 is missing where it returns None instead of an actual tensor. While disabling all ORT optimizations by setting ort.GraphOptimizationLevel.ORT_DISABLE_ALL gets this model correctly executed. So I think it might be due to some miss-/over-optimization in some graph passes.

To reproduce

import onnxruntime as ort
import onnx

import numpy as np

model = onnx.load("test.onnx")
onnx.checker.check_model(model, full_check=True)

x = np.array([[[[5.69269, 3.9831784], [5.8404264, 5.823447]]]], dtype=np.float32)

sess = ort.InferenceSession(
    model.SerializeToString(),
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
results = sess.run(["o0", "o1", "o2"], {"i0": x})

print(f"{results[0] = }")
print(f"{results[1] = }")
print(f"{results[2] = }") # Results are incorrectly optimized out as `None`

assert results[2] is None

options = ort.SessionOptions()
options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
sess = ort.InferenceSession(
    model.SerializeToString(),
    sess_options=options,
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
results = sess.run(["o0", "o1", "o2"], {"i0": x})

print(f"{results[0] = }")
print(f"{results[1] = }")
print(f"{results[2] = }")  # It works without optimization

assert results[2] is not None

The model "test.onnx" is attached: test.zip

Urgency

None. But it is very common to cache all intermediate outputs when using ONNXRuntime as a reference backend in testing (say in PyTorch and other ONNX supported engines).

Platform

Linux

OS Version

Ubuntu 22.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.14.0 (ort-nightly)

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response