Upgrade grpcio-tools to use protobuf 5.26.x

[alexloginov]

Is your feature request related to a problem?
Currently OpenTelemetry-SDK is using protobuf version 4.x which makes impossible to use OTEL and latest Protobuf 5.26 in the same client application. Version 4.x is going to be EOL 31 March 2025 : https://protobuf.dev/support/version-support/.

As far as I can see, issue is related to grpcio-tools dependency: https://github.com/open-telemetry/opentelemetry-python/blob/main/gen-requirements.txt#L5

Describe the solution you'd like
Upgrade grpcio-tools to a newer version that uses newer protobuf version: https://github.com/grpc/grpc/blob/master/tools/distrib/python/grpcio_tools/setup.py#L332

Describe alternatives you've considered
Currently, for us, the only alternative is to stick with Protobuf 4.x

Additional context
Github thread on migrating grpcio-tools to use latest protobuf version: grpc/grpc#36142

[aabmass]

Unfortunately protobuf generated code is supposed to match the runtime protobuf library version. When 4.x was released we had a lot of issues and I opened protocolbuffers/protobuf#11123 to get some clarification. We know have some docs https://protobuf.dev/support/cross-version-runtime-guarantee/ which make it clear New Gencode + Old Runtime = Never Allowed. I.e. regenerating code with grpcio-tools 5 will break compatibility with protobuf 4.

This puts us in a tricky spot of choosing which major version to support. The good news is

Starting with the 2025Q1 release, the protobuf project will adopt a rolling compatibility window for major versions..

That doesn't help right now, but hopefully will for the next major version. Because they are planning to fix things, I'm hesitant on making our own solution like keeping two copies of generated code and dynamically choosing between them at import time.

[aabmass]

Describe alternatives you've considered
Currently, for us, the only alternative is to stick with Protobuf 4.x

@alexloginov is this a big deal right now? Once the ecosystem has majority moved over to Protobuf 5, we can switch over as well. I don't have a good signal on when that is. It would helpful if people can share what version of Protobuf you are using and if this is a critical blocker for other version upgrades.

[aabmass]

I've opened an issue to track upgrading #3958. Please leave your comments there