Current notebook format version

[jasongrout]

There seems to be some discrepancy between places where the current notebook format version is defined

• this package version: 4.4 currently released

  nbformat/nbformat/_version.py

  Line 2 in 35a60ed

  version_info = (4, 5, 0, 'dev')

• the comment in the json schema: implying 4.2

  nbformat/nbformat/v4/nbformat.v4.schema.json

  Line 3 in 35a60ed

  "description": "Jupyter Notebook v4.2 JSON schema.",

• The minor version specified in the schema: implying 4.0

  nbformat/nbformat/v4/nbformat.v4.schema.json

  Line 85 in 35a60ed

  "minimum": 0

Given that there have been schema changes between package version 4.3.0 and 4.4.0, I think the json schema should be updated to reflect it being notebook format 4.4, changing both the comment and the minor version specified.

I also think we should have in the schema definition directory schema files for the notebook format 4.0, 4.1, 4.2, and 4.3 to help people with different formats.

[willingc]

@jasongrout So now I'm confused...

The released version of nbformat is: 4.4

The front ends (jupyterlab, notebook, nteract) are all using 4.2??

[jasongrout]

Apparently we are confused as well. I'm right now implementing several fields from version 4.4 in jlab, but apparently we are saying in the jlab codebase that we only understand 4.2. I'm trying to untangle this now.

[jasongrout]

My current updates to jlab are in commit jupyterlab/jupyterlab@97bf7ba in jupyterlab/jupyterlab#5968

[captainsafia]

Thanks for opening this, @jasongrout.

Following from the conversation on nteract/nteract#4290, should we conclude that each of the front-ends should update to support 4.4 as the latest default format.

It looks like you've got a handle on that for JLab and I can make the necessary changes in nteract. Perhaps we should open an in issue in notebook (if there isn't already one) to track this?

[blois]

The schema additions between 4.2 and 4.4 are cell.metadata.jupyter.source_hidden and cell.metadata.jupyter.cell_hidden: 4.2.0...jupyter:4.4.0

If those fields are written then yes, I do think that frontends should use 4.4 to indicate that the fields will be conforming to the schema (primarily, they will have the correct type).

Typically for file formats, tools are written to support reading older versions but reject versions higher than what is known to be supported. Because of this it seems safer to default to writing the lowest version which is required.

We removed the minor version constraint in Colab, but the scope of the allowed changes did raise some discussions about whether newer versions could truly be considered supported in advance.

[jasongrout]

See also the PR fixing this issue: #139

[jasongrout]

The schema additions between 4.2 and 4.4 are cell.metadata.jupyter.source_hidden and cell.metadata.jupyter.cell_hidden

jupyter.source_hidden and jupyter.outputs_hidden. See also #137, where I think jupyter.outputs_hidden is ambiguous (or redundant?) with the top-level collapsed metadata.

[jasongrout]

@captainsafia - @MSeal pointed out that jupyter/notebook#534 is the notebook issue.

I still think we need to clarify the difference between collapsed and jupyter.outputs_hidden for notebook format 4.4, and which takes precedence when they conflict. That's the subject of #137.

[minrk]

the nbformat package version and the nbformat schema version do not describe the same thing, and shouldn't be assumed to have the same value. It's mostly a confusing coincidence that they happen to be close to the same value at the moment.

For example, an nbformat package release that dropped Python 3 support and/or changed Python API without changing the spec would be version 5.0 while keeping the spec at 4.x with no change.

[willingc]

Let's add a note with @minrk's clarification to the docs to reduce any future confusion.