Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix proto import paths #2920

Merged
merged 6 commits into from
Aug 19, 2024
Merged

Fix proto import paths #2920

merged 6 commits into from
Aug 19, 2024

Conversation

joshklop
Copy link
Contributor

@joshklop joshklop commented Aug 14, 2024

A proto file's *import path* is relative to one of the `--proto-path`s.

Previously, the proto files were compiled separately. Some invocations
used different values for the `--proto_path`, which led to inconsistent
import paths in proto file descriptors.

Typically, this wouldn't be a problem. However, if a downstream project
uses `protoregistry.GlobalFiles` to inspect proto dependencies, it will fail
to find a dependency's file descriptor when the dependency was compiled
with a different `--proto_path`.

By using a single script to generate all protobuf files, we can ensure
the `--proto_path` is always set to the same sane value (the root of the
project, as suggested in the [official documentation]).

[official documentation]: https://protobuf.dev/programming-guides/proto2/#importing

Cosmos SDK modules may inspect protoregistry.GlobalFiles in SDK v0.50.8 and later. As a result, this appears to be necessary to use libp2p in the same binary as those modules.

A proto file's *import path* is relative to one of the `--proto-path`s.

Previously, the proto files were compiled separately. Some invocations
used different values for the `--proto_path`, which led to inconsistent
import paths in proto file descriptors.

Typically, this wouldn't be a problem. However, if a downstream project
uses `protoregistry.GlobalFiles` to inspect proto dependencies, it will fail
to find a dependency's file descriptor when the dependency was compiled
with a different `--proto_path`.

By using a single script to generate all protobuf files, we can ensure
the `--proto_path` is always set to the same sane value (the root of the
project, as suggested in the [official documentation]).

[official documentation]: https://protobuf.dev/programming-guides/proto2/#importing
@joshklop
Copy link
Contributor Author

joshklop commented Aug 14, 2024

The commits start with a failing test and builds up to a fully working solution. They should probably be squashed into a single commit before merging to master.

joshklop added a commit to polymerdao/monomer that referenced this pull request Aug 14, 2024
We recently expunged Pulsar from our repo (yay!). In the process, we
needed to upgrade to the next patch version of the Cosmos SDK (v0.50.8).

Version v0.50.8 has dependencies on modules that rely on `gogoproto`'s
`MergedRegistry` function. Thus, it's not possible to comment out the
line (as we were doing [before]), unless we wanted to multiply the
number of `go replace` directives.

Instead, we address the root cause in the `go-libp2p` package. That's
the only new replace directive we need. We have [upstreamed the fix],
but that may not be used by the op-node for quite some time. A single
`replace` is fine for now.

This commit updates the docs with the correct instructions for the
latest Monomer changes.

[before]: https://github.com/polymerdao/monomer/blob/c7ecbee46914c943bfbecaf2bd5ded3076df09f2/doc/docs/learn/create-an-app-with-monomer.md?plain=1#L150
[upstreamed a fix]: libp2p/go-libp2p#2920
joshklop added a commit to polymerdao/monomer that referenced this pull request Aug 14, 2024
We recently expunged Pulsar from our repo (yay!). In the process, we
needed to upgrade to the next patch version of the Cosmos SDK (v0.50.8).

Version v0.50.8 has dependencies on modules that rely on `gogoproto`'s
`MergedRegistry` function. Thus, it's not possible to comment out the
line (as we were doing [before]), unless we wanted to multiply the
number of `go replace` directives.

Instead, we address the root cause in the `go-libp2p` package. That's
the only new replace directive we need. We have [upstreamed the fix],
but that may not be used by the op-node for quite some time. A single
`replace` is fine for now.

This commit updates the docs with the correct instructions for the
latest Monomer changes.

[before]: https://github.com/polymerdao/monomer/blob/c7ecbee46914c943bfbecaf2bd5ded3076df09f2/doc/docs/learn/create-an-app-with-monomer.md?plain=1#L150
[upstreamed the fix]: libp2p/go-libp2p#2920
joshklop added a commit to polymerdao/monomer that referenced this pull request Aug 14, 2024
We recently expunged Pulsar from our repo (yay!). In the process, we
needed to upgrade to the next patch version of the Cosmos SDK (v0.50.8).

Version v0.50.8 has dependencies on modules that rely on `gogoproto`'s
`MergedRegistry` function. Thus, it's not possible to comment out the
line (as we were doing [before]), unless we wanted to multiply the
number of `go replace` directives.

Instead, we address the root cause in the `go-libp2p` package. That's
the only new replace directive we need. We have [upstreamed the fix],
but that may not be used by the op-node for quite some time. A single
`replace` is fine for now.

This commit updates the docs with the correct instructions for the
latest Monomer changes.

[before]: https://github.com/polymerdao/monomer/blob/c7ecbee46914c943bfbecaf2bd5ded3076df09f2/doc/docs/learn/create-an-app-with-monomer.md?plain=1#L150
[upstreamed the fix]: libp2p/go-libp2p#2920
joshklop added a commit to polymerdao/monomer that referenced this pull request Aug 14, 2024
We recently expunged Pulsar from our repo (yay!). In the process, we
needed to upgrade to the next patch version of the Cosmos SDK (v0.50.8).

Version v0.50.8 has dependencies on modules that rely on `gogoproto`'s
`MergedRegistry` function. Thus, it's not possible to comment out the
line (as we were doing [before]), unless we wanted to multiply the
number of `go replace` directives.

Instead, we address the root cause in the `go-libp2p` package. That's
the only new replace directive we need. We have [upstreamed the fix],
but that may not be used by the op-node for quite some time. A single
`replace` is fine for now.

This commit updates the docs with the correct instructions for the
latest Monomer changes.

[before]: https://github.com/polymerdao/monomer/blob/c7ecbee46914c943bfbecaf2bd5ded3076df09f2/doc/docs/learn/create-an-app-with-monomer.md?plain=1#L150
[upstreamed the fix]: libp2p/go-libp2p#2920
joshklop added a commit to polymerdao/monomer that referenced this pull request Aug 14, 2024
We recently expunged Pulsar from our repo (yay!). In the process, we
needed to upgrade to the next patch version of the Cosmos SDK (v0.50.8).

Version v0.50.8 has dependencies on modules that rely on `gogoproto`'s
`MergedRegistry` function. Thus, it's not possible to comment out the
line (as we were doing [before]), unless we wanted to multiply the
number of `go replace` directives.

Instead, we address the root cause in the `go-libp2p` package. That's
the only new replace directive we need. We have [upstreamed the fix],
but that may not be used by the op-node for quite some time. A single
`replace` is fine for now.

This commit updates the docs with the correct instructions for the
latest Monomer changes.

[before]: https://github.com/polymerdao/monomer/blob/c7ecbee46914c943bfbecaf2bd5ded3076df09f2/doc/docs/learn/create-an-app-with-monomer.md?plain=1#L150
[upstreamed the fix]: libp2p/go-libp2p#2920
Copy link
Collaborator

@MarcoPolo MarcoPolo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a reasonable change. Thanks for doing it.

@MarcoPolo
Copy link
Collaborator

Interop test is failing, but it's not this PR's fault. It's just running out of space on the runner.

@MarcoPolo MarcoPolo merged commit 0fb7dac into libp2p:master Aug 19, 2024
8 of 9 checks passed
MarcoPolo added a commit that referenced this pull request Sep 4, 2024
* Add failing proto test

* Add a new proto compilation script

A proto file's *import path* is relative to one of the `--proto-path`s.

Previously, the proto files were compiled separately. Some invocations
used different values for the `--proto_path`, which led to inconsistent
import paths in proto file descriptors.

Typically, this wouldn't be a problem. However, if a downstream project
uses `protoregistry.GlobalFiles` to inspect proto dependencies, it will fail
to find a dependency's file descriptor when the dependency was compiled
with a different `--proto_path`.

By using a single script to generate all protobuf files, we can ensure
the `--proto_path` is always set to the same sane value (the root of the
project, as suggested in the [official documentation]).

[official documentation]: https://protobuf.dev/programming-guides/proto2/#importing

* Add go_package options so scripts/gen-proto.sh succeeds

* Remove undesirable `go:generate protoc` directives

* Run `go generate ./...`

* Script uses arrays, I think we need bash

---------

Co-authored-by: Marco Munizaga <git@marcopolo.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants