Iterate over leafs of a JSON data structure: enrich the JSON pointer API

[SylvainCorlay]

Enriching the json_pointer API

I noticed that the json_pointer class stored a std::vector<std::string> internally.

With private inheritance instead of composition, and a bunch of using statements to forward the API, you would enable a JSON pointer with a richer API, such as

• initialize with initializer lists.
• push / pop -pack for new values in the paths.

The use case

My use-case is that I would like to iterate over all the leafs (number, boolean, null, strings) of a nested JSON data structure..

In order to do so, I would keep a vector of strings for the current position to which new elements are pushed back when meeting an array or object, and popped when leaving a level.

If this "key" is a JSON pointer, it could be used for access later, while right now, I would need to compose a string from it and pass it to the constructor of json_pointer, which is not very efficient.

[nlohmann]

Related: #837

[nlohmann]

Could you please provide more information on the described use case? Is the use of JSON Pointers fixed or would a different solution also be helpful? On iteration of nested elements, this may be helpful: https://stackoverflow.com/questions/45934851/c-nlohmann-json-how-to-iterate-find-a-nested-object/46630394#46630394

[SylvainCorlay]

Could you please provide more information on the described use case? Is the use of JSON Pointers fixed or would a different solution also be helpful? On iteration of nested elements, this may be helpful: https://stackoverflow.com/questions/45934851/c-nlohmann-json-how-to-iterate-find-a-nested-object/46630394#46630394

Use case

OK, so the context is the implementation of the widgets protocol for the C++ Jupyter kernel (see e.g. xwidgets). You can try it out live here: https://beta.mybinder.org/v2/gh/QuantStack/xwidgets/0.9.0?filepath=notebooks/xwidgets.ipynb

The Jupyter widget protocol allows communicating binary frames in the messages to and from the frontend. The first frame is generally a JSON string and includes references to the binary frames

{
    "foo": {
        "bar": @buffer_reference@1
    },
    "baz": 3,
    "bat": @buffer_reference@0
}

From which I need to extract an array of buffer paths:

[["bat"], ["foo", "bar"]]

Reversely, from the buffer paths, and the JSON {"baz": 3}, I need to produce the original JSON.

Using JSON pointers?

I could basically iterate in a similar way as in your example, but I figured that if the paths were encoded as json pointers, they it would be more elegant, especially for inserting back the @buffer_reference@ into the JSON.

So IMO, buffer paths would be more amenable if they had the full vector API.

• we could easily push / pop back, insert items

• more generally, use the rich vector API to do things such as

  jp.insert(jp.end(), {
      "few",
      "more",
      "items"
  });

In practice, using private inheritance with std::vector<std::string> instead of composition, and a few using statements to export the API.

[nlohmann]

I am hesitant on adding an operator[] with a string access. This is exactly what JSON Pointers are for - including a well-defined syntax and semantics. If further manipulation of the path is required, then this would be possible with #837.

I would be happy for further discussion on this!

[SylvainCorlay]

I am hesitant on adding an operator[] with a string access. This is exactly what JSON Pointers are for - including a well-defined syntax and semantics. If further manipulation of the path is required, then this would be possible with #837.

Hum, this is not what I suggested! I merely suggested to give JSON pointer a general std::vector API.

[nlohmann]

So this issue could be seen as a duplicate of #837?

[SylvainCorlay]

Maybe, although the idea of a push_back with a slash-separated string is weird, since the STL has APIs to manipulate sequences.

Something like

jp.insert(jp.end(), {
    "few",
    "more",
    "items"
});

is more natural than

jp.push_back("few/more/items");

also, having a special API based on operator/= and such would make it impossible to use STL generic algorithms on json_pointers.

[nlohmann]

Ok, I reopened #837. Let's join the discussion there.

[nlohmann]

Duplicate of #837.

[SylvainCorlay]

Thanks