Update C domain docs to Sphinx 3 syntax

[AA-Turner]

Sphinx would like to remove support for the pre-v3 C domain syntax. We have confirmed that the Linux kernel is no longer using it, so Python is the only project we are blocked on.

I have a branch where I've done the work, but it is a large number of files affected (~50) and the majority of the C-API documentation. The changes are almost all mechanical search-and-replace, but there are a few where I've had to use judgement.

Questions:

1. Is this change welcome?
2. How should I proceed with PRs? One per data type, per file, etc?
3. I would want to backport these changes to 3.10/3.11 (to allow future docs backports without conflicts), does anybody have objections?
4. Does anybody have other concerns?

cc: @vstinner @serhiy-storchaka @encukou @erlend-aasland @JelleZijlstra @hugovk (Selection of C-API/documentation people, if I've missed anyone please mention them).

A

Tasks

• gh-93738: Documentation C syntax (:c:data:0 -> 0) #97771
• gh-93738: Documentation C syntax (:c:type:Py_UNICODE* -> :c:expr:Py_UNICODE*) #97784
• gh-93738: Documentation C syntax (:c:type:PyUnicodeObject* -> :c:expr:PyUnicodeObject*) #97783
• gh-93738: Documentation C syntax (:c:type:PyBytesObject* -> :c:expr:PyBytesObject*) #97782
• gh-93738: Documentation C syntax (:c:type:PyTupleObject* -> :c:expr:PyTupleObject*) #97780
• gh-93738: Documentation C syntax (:c:data:view->obj -> :c:expr:view->obj) #97773
• gh-93738: Documentation C syntax (:c:type:FILE -> :c:expr:FILE) #97769
• gh-93738: Documentation C syntax (:c:type:TYPE -> :c:expr:TYPE) #97770
• gh-93738: Documentation C syntax (:c:type:PyInterpreterState * -> :c:expr:PyInterpreterState *) #97777
• gh-93738: Documentation C syntax (:c:type:PyObject -> :c:expr:PyObject) #97776
• gh-93738: Documentation C syntax (:c:type: to :c:expr:, misc. cases) #97775
• gh-93738: Documentation C syntax (:c:type:PyTypeObject* -> :c:expr:PyTypeObject*) #97778
• gh-93738: Documentation C syntax (Use c:struct) #97772
• gh-93738: Documentation C syntax (Function glob patterns -> literal markup) #97774
• gh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) #97768
• gh-93738: Disallow pre-v3 syntax in the C domain #97962

[serhiy-storchaka]

What directives and roles can be used and what are gone?

1. Is this change welcome?

We do not have a choice, isn't?

2. How should I proceed with PRs? One per data type, per file, etc?

Could manual and automatic changes be separated, so intermediate results are still working? Say, first some preparing manual changes, then automatic changes, then final manual changes?

3. I would want to backport these changes to 3.10/3.11 (to allow future docs backports without conflicts), does anybody have objections?

Do not have objections if systems with 3.10 support Sphinx 3.

[AA-Turner]

What directives and roles can be used and what are gone?

Gone in 3.0-3.2:

Relaxed parsing. In Sphinx 2, a definition like :c:type:`const char*` was OK, but in Sphinx 3 this fails -- it should be a :c:expr:. :c:type:`PyObject*` is currently rendered in the docs with a link only to the "PyObject" part of the role text -- it should change to either having the * outside of the role (:c:type:`PyObject`\*) or being an expression (:c:expr:`PyObject*` ). Places where there are references to groups of functions using the type role are invalid (and currently don't work), these should be replaced with literal syntax (e.g. :c:func:`PyErr_Set\*` ).

New in 3.0-3.2, taken from Sphinx's C domain docs:

Directives:

• .. c:macro:: name(arg list) (Function style macro variant)
• .. c:struct:: name
• .. c:union:: name
• .. c:enum:: name
• .. c:enumerator:: name

Cross-referencing roles:

• :c:var: (alias of both :c:member: and :c:data:)
• :c:struct:
• :c:union:
• :c:enum:
• :c:enumerator:

Documentation support for anonymous entities (https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#anonymous-entities)

Support for re-printing declarations in a list, e.g. for overviews of interfaces (https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#aliasing-declarations)

Support for inline expresssions (https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#inline-expressions-and-types):

• :c:expr:
• :c:texpr: ("text-expression")

Support for documentation-namespaces (https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#namespacing)

A

[hugovk]

cc: @vstinner @serhiy-storchaka @encukou @erlend-aasland @JelleZijlstra @hugovk (Selection of C-API/documentation people, if I've missed anyone please mention them).

cc also @JulienPalard

[vstinner]

If I recall correctly, in 2020, I asked Sphinx to add an option to opt-in for the legacy C syntax. When it was discussed, the problem was that Sphinx supporting the new syntax was not widely available in Linux distributions. I guess that the situation changed and now it's perfectly fine to switch to the new syntax.

Doc/conf.py gives some context and links to bpo-40204:

# bpo-40204: Allow Sphinx 2 syntax in the C domain
c_allow_pre_v3 = True

# bpo-40204: Disable warnings on Sphinx 2 syntax of the C domain since the
# documentation is built with -W (warnings treated as errors).
c_warn_on_allowed_pre_v3 = False

I would want to backport these changes to 3.10/3.11 (to allow future docs backports without conflicts)

I agree to backport for the reason that you give: make future backports simpler.

[encukou]

I've looked at the proposed branch. The commits there would make good individual PRs, IMO.
IMO we've now made it clear that documentation discussions live in the Discuss category. Perhaps mention there, and if no one objects, go ahead with PRs.

Some specific points:

Please keep/turn as many of these as possible as/into links. Many times this would just mean checking that the links are generated (and are correct).

AFAIK, Sphinx will parse :c:expr: expressions and linkify any types it finds, so :c:expr:const PyObject* will link to PyObject. If that's right, IMO the :c:type:PyObject* should be changed to expr, which is much easier to type than the ecaped star.
Or is there some disadvantage to :c:expr:?

References to a family of functions (glob patterns) like :c:func:PyArg_Parse\* functions link to the section (especially for PyArg_Parse where not all of them actually use the prefix); other cases could use wording like PyArg_Parse* functions (such as :c:func:PyArg_VaParse)

[AA-Turner]

Created at https://discuss.python.org/t/16500

A

[encukou]

@AA-Turner, do you still want to do this, or would it be better if someone else took over?