PEP 733: An Evaluation of Python's Public C API

[iritkatriel]

Any author who has no more comments can go ahead and approve.
I'll make some efforts to chase those who don't approve, but if not successful I may have to remove them as authors.

Approving authors

• Erlend E. Aasland @erlend-aasland
• Domenico Andreoli @cavokz
• Stefan Behnel @scoder
• Carl Friedrich Bolz-Tereick @cfbolz
• Simon Cross @hodgestar
• Steve Dower @zooba
• Tim Felgentreff @timfel
• David Hewitt @davidhewitt
• Shantanu Jain @hauntsaninja
• Wenzel Jakob @wjakob
• Irit Katriel
• Marc-Andre Lemburg @malemburg
• Donghee Na @corona10
• Karl Nelson @Thrameos
• Ronald Oussoren @ronaldoussoren
• Antoine Pitrou @pitrou
• Neil Schemenauer @nascheme
• Mark Shannon @markshannon
• Stepan Sindelar @steve-s
• Gregory P. Smith @gpshead
• Eric Snow @ericsnowcurrently
• Victor Stinner @vstinner
• Guido van Rossum @gvanrossum
• Petr Viktorin @encukou
• Carol Willing @willingc
• William Woodruff @woodruffw
• David Woods @da-woods
• Jelle Zijlstra @JelleZijlstra

Basic requirements (all PEP Types)

• Read and followed PEP 1 & PEP 12
• File created from the latest PEP template
• PEP has next available number, & set in filename (pep-NNNN.rst), PR title (PEP 123: <Title of PEP>) and PEP header
• Title clearly, accurately and concisely describes the content in 79 characters or less
• Core dev/PEP editor listed as Author or Sponsor, and formally confirmed their approval
• Author, Status (Draft), Type and Created headers filled out correctly
• PEP-Delegate, Topic, Requires and Replaces headers completed if appropriate
• Required sections included
  • Abstract (first section)
  • Copyright (last section; exact wording from template required)
• Code is well-formatted (PEP 7/PEP 8) and is in code blocks, with the right lexer names if non-Python
• PEP builds with no warnings, pre-commit checks pass and content displays as intended in the rendered HTML
• Authors/sponsor added to .github/CODEOWNERS for the PEP

---

📚 Documentation preview 📚: https://pep-previews--3491.org.readthedocs.build/pep-0733/

[cfbolz]

great draft!

In my opinion it's limiting to not list the CPython core developers as an explicit stakeholder group. CPython is a producer of the API, but it also consumes the API itself, and it has quite specific needs for what the API should look like that are different from everyone else. We should think about and list those specific needs and not leave them out as implicitly understood (I certainly don't completely understand them).

[JelleZijlstra]

You can use PEP 732 733 (#3489 can take 732 as it was submitted first).

There is a very large number of authors (Victor also mentioned this on the PEP). I feel this means we should wait for all authors to confirm that they are OK with being listed as authors, since even if they contributed something to the problems repo, they may not necessarily want their name on this PEP.

An alternative could be to have only you (Irit) and maybe a few others as an author, and list everyone else in the Acknowledgments section.

[timfel]

great draft!

In my opinion it's limiting to not list the CPython core developers as an explicit stakeholder group. CPython is a producer of the API, but it also consumes the API itself, and it has quite specific needs for what the API should look like that are different from everyone else. We should think about and list those specific needs and not leave them out as implicitly understood (I certainly don't completely understand them).

@cfbolz there was some discussion around this here: capi-workgroup/problems#73

[timfel]

Thanks for the great write up @iritkatriel! For Stepan and me the Email addresses are Stepan Sindelar stepan.sindelar@oracle.com and myself tim.felgentreff@oracle.com.

An alternative could be to have only you (Irit) and maybe a few others as an author, and list everyone else in the Acknowledgments section.

I would also think that's a good alternative if many others is too uncommon or not considered useful for other reasons.

[hugovk]

I've updated the top post with the checklist, please go through and check off things as they're done.

[hugovk]

Thank you for putting this together!

[iritkatriel]

Any author who has no more comments can go ahead and approve.
I'll make some efforts to chase those who don't approve, but if they are not successful I may have to remove them as authors.

[JelleZijlstra]

Looks good to me, with a number of minor suggestions.

[cfbolz]

@cfbolz there was some discussion around this here: capi-workgroup/problems#73

Thanks @timfel, I had indeed not seen that. I don't want to rehash that discussion (even if I ultimately don't completely agree), so you can ignore my comment.

[gpshead]

Overall this is in great shape, thanks for writing it up! I left a few comments, though many of them are informational rather than things in need of change here.

[markshannon]

Looks good.
A few comments, mostly about reference terminology.

[malemburg]

@malemburg @Thrameos @nascheme @markshannon @gpshead @serhiy-storchaka @woodruffw

This is getting close to being merged. If you won't review and approve in the next few days I will remove you from the author list. (You can still review later and asked to be put back on).

I'll review this weekend. I'm missing a few things from the "Common Actions" section, in particular accessing existing objects and calling methods/functions on them (rather than only creating them and then calling them). I'm also missing a statement to address the concern I raised about the general focus of the C API in terms of consistency, completeness and usefulness (see the discussion in capi-workgroup/problems#71), but more on that in the next two days.

[Thrameos]

An items that I don't see captured that were relevant to this PEP were the lack of ability to construct derived classes for PyLong (which internally does hacks in which it constructs a PyLong and then copies it contents). This same hack has to be done to extend when with other language binds (such as JInt).

In general all types that one can extend from within Python should also be extendable within the CAPI with the same ease.

[Thrameos]

@malemburg sorry for the slow review. My laptop died 3 months ago and was unable to be replaced at work due to supply chain issues until this weekend. And as I am not permitted to bring my cell phone into perform the two factor authentication, I was locked out for participating.

[malemburg]

Thank you for working through all those "problems" issues in the capi-workgroup repo and creating this writeup, @iritkatriel. I guess this will not be only PEP that'll get written on the topic, but it's definitely a good start in the right direction.

I left a few comments on the PEP, which may help improve it some more.

[Thrameos]

Then why select a illustration for project that did not contribute to the conversations of the PEP over one who did? There is already the perception problem for JPype that PyJNIus is a solution for Java. They currently serve different roles on paper, but they are incompatible thus just make for incompatible modules. I tried unsuccessfully in the past to merge the projects setting up a full demo of JPype that worked on Android that Gabriel Pettier and the KIvy project did not take action on. PyJNIus is a bare bones Java wrapper that can be easily seg fault through user actions vs JPype which is fully integrated with CPython and NumPy and very good stability (though not perfect stability because of defects this PEP is addressing). Hence many topics ( concrete vs abstract strings, stack frame reporting, memory management) in the PEP were problems found in JPype because it is integrated. None of these are relevant to PyJNIus because it just has the minimal required to use Java on Android. If I had the time and support from the other Java wrappers to unify JPype, JEP, and PyJNIus I would but I was unable to get interest from others. Hence a divided community. I would have no problem if PyJNIus was listed as the binding for “Android”, but listing it as the binding for “Java” is a bit much as it does minimal exposure and doesn’t integrate anything like Python buffers or IO which are required to use complex Python packages like numpy or matplotlib with Java.

[iritkatriel]

I'll give this another day or two and if no more comments I'll remove the (few) authors who have not approved and merge.

[woodruffw]

LGTM! I left one comment about stable_abi.toml being a noteworthy source of machine-readable information about the C API.

[iritkatriel]

Thank you everyone!

[vstinner]

Congrats @iritkatriel for managing to drive the writing of this very important document!