Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Map decoding in presence of optional long byte-length marker #193

Merged
merged 3 commits into from
Nov 10, 2023

Conversation

fieldstrength
Copy link

I encountered a problem where messages that have an Avro Map produced by certain other Avro libraries were not decoding, and I noticed that we are not decoding maps properly in the presence of this optional byte-length indicator.

If a block’s count is negative, its absolute value is used, and the count is followed immediately by a long block size indicating the number of bytes in the block. This block size permits fast skipping through data, e.g., when projecting a record to a subset of its fields.

> If a block’s count is negative, its absolute value is used, and the count is followed immediately by a long block size indicating the number of bytes in the block. This block size permits fast skipping through data, e.g., when projecting a record to a subset of its fields.

- https://avro.apache.org/docs/1.11.1/specification/#complex-types-1
@HuwCampbell
Copy link
Collaborator

HuwCampbell commented Oct 2, 2023

Thanks!

That looks ok, but, the documentation also states this is so for arrays as well as maps.

It you could please:

  • Include the fixup for arrays as well as maps
  • Add the prose from the documentation to the code where it makes sense

That would be great and I'll merge

@HuwCampbell HuwCampbell merged commit eb1e45e into haskell-works:main Nov 10, 2023
12 of 17 checks passed
@HuwCampbell
Copy link
Collaborator

Ta

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants