Define supported mappings (proto->JSON or JSON->proto)

[webmaster128]

The current document speaks about encoding proto->JSON. I think we should get clarity if this is the only supported mapping or not. There are other places that imply a usage of JSON->proto (let's call this "(JSON) decoding" in this context).

For signing and signature verification purposes, only encoding is needed.

• Sign: put m = hash(serialize(proto)) and privkey into ECDSA secp256k1 sign
• Verify: out m = hash(serialize(proto)), pubkey and signature into secp256k1 verify

What cosmos/cosmos-sdk#6031 suggests is that there is a reverse mapping (decode) as well. However, such a decoding would be lossy in the current implementation (i.e. decode(encode(original)) != original), e.g. because JSON Canonical Form changes strings to a normalized form, i.e. manipulates content, not only structure.

I think it would be important to know the goals of those two mappings in order to verify if the spec achives them. Obviously, things get much easier by explicitely disallowing the decoding.

[aaronc]

However, such a decoding would be lossy in the current implementation (i.e. decode(encode(original)) != original), e.g. because JSON Canonical Form changes strings to a normalized form, i.e. manipulates content, not only structure.

How is this true? What changes in the content?

The proto3 spec does define JSON decoding: https://developers.google.com/protocol-buffers/docs/proto3#json. But I think it is out of the scope of this document because decoding wouldn't be used in the signature verification process. cosmos/cosmos-sdk#6031 implies a JSON API via https://github.com/grpc-ecosystem/grpc-gateway, but that doesn't relate to signing. Or am I missing something?

[webmaster128]

How is this true? What changes in the content?

Oh, damn, I misunderstood

MUST represent all strings (including object member names) in their minimal-length UTF-8 encoding

I confused this with unicode normalization. Sorry for the noise! But I'll keep searching for incompatibilities. This proto-JSON bridge was not designed for crypto.

But I think it is out of the scope of this document because decoding wouldn't be used in the signature verification process.

If decoding is in-scope or not really depends on the question if and to which degree JSON->proto must be supported and which guarantees are needed. If we need decode(encode(original)) == original, then we need to be much more careful compared to having a deterministic one way function.

cosmos/cosmos-sdk#6031 implies a JSON API via https://github.com/grpc-ecosystem/grpc-gateway, but that doesn't relate to signing. Or am I missing something?

Not only a JSON API (that could consume basse64 encoded protobuf documents). But that JSON encoded transaction need to be decoded back to proto without access to the original protobuf document.

[aaronc]

If decoding is in-scope or not really depends on the question if and to which degree JSON->proto must be supported and which guarantees are needed. If we need decode(encode(original)) == original, then we need to be much more careful compared to having a deterministic one way function.

No, decoding the JSON is irrelevant for this scope.

Not only a JSON API (that could consume basse64 encoded protobuf documents). But that JSON encoded transaction need to be decoded back to proto without access to the original protobuf document.

Yeah, I think that's out of the scope of this particular canonicalization document which is more about signing and state encoding than RPC/REST interfaces.