Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when roundtripping BJData #3519

Closed
2 tasks done
nlohmann opened this issue Jun 6, 2022 · 1 comment · Fixed by #3523
Closed
2 tasks done

Error when roundtripping BJData #3519

nlohmann opened this issue Jun 6, 2022 · 1 comment · Fixed by #3523
Assignees
Labels
aspect: binary formats BSON, CBOR, MessagePack, UBJSON kind: bug solution: proposed fix a fix for the issue has been proposed and waits for confirmation

Comments

@nlohmann
Copy link
Owner

nlohmann commented Jun 6, 2022

Description

Given the following BJData input

00000000: 5b24 4d23 5b49 0020 4d00 0000 0000 0020  [$M#[I. M...... 
00000010: ff5d                                     .]

the library can successfully parse it to

{
    "_ArrayData_": [],
    "_ArraySize_": [
        8192,
        -63050394783186944
    ],
    "_ArrayType_": "uint64"
}

However, when then serializing this JSON as BJData, the following exception is raised:

[json.exception.parse_error.113] parse error at byte 17: syntax error while parsing BJData size: count in an optimized container must be positive

The error was found by OSS-Fuzz, and is unrelated to the latest fixes.

Reproduction steps

int main() {
    auto vec = readFile("clusterfuzz-testcase-minimized-parse_bjdata_fuzzer-5159110460440576.bjdata");

    auto j = json::from_bjdata(vec);
    std::vector<uint8_t> vec2 = json::to_bjdata(j, false, false);

    json::from_bjdata(vec2);
}

Expected vs. actual results

  • Expected: Either reject the original input or (also) accept the created byte vector.
  • Actual: Accepting the input, but rejecting the byte vector.

I would also be fine with adding a note to https://json.nlohmann.me/features/binary_formats/bjdata/, but then we also need to adjust https://github.com/nlohmann/json/blob/develop/tests/src/fuzzer-parse_bjdata.cpp#L65

Minimal code example

See above.

Error messages

[json.exception.parse_error.113] parse error at byte 17: syntax error while parsing BJData size: count in an optimized container must be positive

Compiler and operating system

OSS-Fuzz

Library version

develop

Validation

@nlohmann nlohmann added kind: bug aspect: binary formats BSON, CBOR, MessagePack, UBJSON labels Jun 6, 2022
@nlohmann
Copy link
Owner Author

nlohmann commented Jun 6, 2022

CC @fangq

@nlohmann nlohmann added the solution: proposed fix a fix for the issue has been proposed and waits for confirmation label Jun 9, 2022
@nlohmann nlohmann self-assigned this Jun 9, 2022
@nlohmann nlohmann added this to the Release 3.11.0 milestone Jun 9, 2022
nlohmann pushed a commit that referenced this issue Jun 9, 2022
…d 32bit unit test (#3523)

* Fix ndarray dimension signness, fix ndarray length overflow, close #3519

* detect size overflow in ubjson and bjdata

* force reformatting

* Fix MSVC compiler warning

* Add value_in_range_of trait

* Use value_in_range_of trait

* Correct 408 parse_errors to out_of_range

* Add 32bit unit test

The test can be enabled by setting JSON_32bitTest=ON.

* Exclude unreachable lines from coverage

Certain lines are unreachable in 64bit builds.

Co-authored-by: Qianqian Fang <fangqq@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
aspect: binary formats BSON, CBOR, MessagePack, UBJSON kind: bug solution: proposed fix a fix for the issue has been proposed and waits for confirmation
Projects
None yet
1 participant