Expand the stakeholders section #73
Conversation
capi_problems.rst
Outdated
|
|
||
| Because the CPython code is by definition intended to run only on given | ||
| version of CPython and is always recompiled, the following requirements | ||
| *do not apply* to CPython developers: |
There was a problem hiding this comment.
Can we reword this somehow? Everyone's requirement apply to cpython core devs because they are the stewards of the language.
There was a problem hiding this comment.
yes, absolutely valid point. What about "...do not apply to the C API usage in CPython core codebase"?
|
Fixed the review comment and two other minor things/typos I spotted in the meanwhile |
capi_problems.rst
Outdated
| There are two main groups of the users of the C API: | ||
|
|
||
| * External users. They only consume the API. | ||
| * CPython developers. They define, implement, and consume the API inside the CPython code. |
There was a problem hiding this comment.
In terms of the structure of this section, I think this binary division into "us" and "them" gives too much weight to the internal usage in cpython. I tried in the section to include a list of types of users, and this lumps everything other than cpython core into one category, with the others being sub-categories.
Can we, instead of talking about "two main types of users", just add the internal usage in cpython core as one of the stakeholders, and in the cpython core section talk about what they do or don't need?
capi_problems.rst
Outdated
| version of CPython and is always recompiled, the following requirements | ||
| *do not apply* to the C API usage in CPython core codebase: | ||
|
|
||
| * Compatibility with alternative Python implementations |
There was a problem hiding this comment.
The cpython test suite is actually maintained to support alternative implementations. There is a decorator to indicate which tests are testing implementation details that other implementations don't need to care about.
There was a problem hiding this comment.
Sure, that's the public contract of Python, but from the point of view of CPython developer I don't think that it matters whether the API that is used to implement CPython itself supports alternative Pythons or not (and should not be visible in Python's public contract). For alternative Pythons it matters whether the C API that extensions use is implementable with their internal structure. Those two APIs (for CPython core, for 3rd party extensions) are one thing currently (or overlap a lot), but they do not have to be.
Maybe it should clarify that in the section "CPython developers" we focus on the development of CPython core and the C API's requirements for that purpose only. If those requirements are in contradiction with requirements for other use-cases, it's fine. I think that's the point of collecting the requirements.
There was a problem hiding this comment.
Is it even correct to say that CPython Core (excluding stdlib extensions) is using the cpython API? It can access whatever it wants.
There was a problem hiding this comment.
Perhaps we should just focus on the requirements that different stakeholders have, and omit statements about what requirements they don't care about. So rather than explaining how python core doesn't need to support multiple versions, we mention that many extension writers do.
There was a problem hiding this comment.
This is a good point. The document actually does not specify what exactly is "C API" in its context. Is it limited API, things exposed in Python.h, things exposed in any header in Python's include directory? In any case, those header files and the functions(, structs, macros, ...) declared in them are shared between external code (extensions, embedders) and the interpreter itself. Also the introduction of this document mentions:
It [C API] evolved from what was initially the internal API between the C code of the interpreter and the Python language and libraries
I think that it is a real issue that the current C API (even if we think of it as the most restricted version: the limited API) hinders the development of the internals of CPython itself. The linked discussion demonstrates that clearly I believe. There is also discussion about stable ABI and PEP 703, because it needs to hide implementation details of reference counting behind actual function call as opposed to C macros, i.e., break the stable ABI. I think all that is evidence that there should be a stakeholder section for CPython and it should list those issues.
My specific point here is that whatever CPython uses to implement itself, does not actually need to work for other Pythons and does not need any stability guarantees. Whether that is the same API, some superset, or something completely different belongs to the design/implementation of whatever comes up from assessing the requirements described here.
Admittedly, I believe that C API for 3rd party extensions that hides the implementation details enough to be really future proof not only for other Pythons but future development of CPython, must be inherently different from what CPython uses internally. The fact that the current C API can somehow serve both as public C API for external code and internal API actually shows that it is not abstract enough. However, this is a conclusion/solution, which is what I am trying to avoid in this document, and I am rather trying to describe the problems that lead me to this conclusion :-)
I will think about rephrasing this section bit more. Any suggestions are welcome :-)
Exactly. However, the "internal C API" and the "extension module C API" overlap substantially, so it seems important to note this distinction clearly somewhere. In the past I have been in some confusing discussions because I was thinking of the C API as "for extension modules" and others in the discussion have been thinking of them as "for writing CPython". Given the overlap, it's hard to criticize this latter view as entirely incorrect. I'm happy for this to be resolved by defining what we mean by "C API" in these documents to exclude the internal use instead. As already noted by @iritkatriel, the internal API can, to zeroth approximation, be whatever it wants because it is internal. |
Which parts of the addition do you see as "debating" rather then "describing"? I think the additions really boil down to:
Plus this PR also expands on different use-cases of why and how users choose to develop C extensions. Let's put that aside for now to keep this discussion more focused. Do we agree on points 1. - 3.? I can make the CPython developers section shorter. I would like to keep the compatibility requirements in the stakeholders section (I think that's already short). I'd leave out the use-cases for extension writing. Does that sound better? |
|
Ceretainly this comment feels like debating. :-) I feel it's redundant to list the CPython developers as stakeholders -- that's implied. When they use the API, they are just users (like extension authors). When they design parts of the API, they consult this document (or its successors, etc.). None of that needs to be said or described explicitly, that just makes the document longer. I would be very surprised if backward and forward compatibility wasn't already mentioned in the document. It's an obvious requirement (without it, we wouldn't be so stressed about changes to the C API), we don't need to use many words. Indeed the Stable ABI is a constraint, if it isn't mentioned at all in the document we could add a sentence mentioning it. |
|
Ok, point taken. CPython developers as stakeholders: that's indeed implied. I wanted to emphasize it. If this document is intended for CPython developers (and not extension authors, for example), than there's no point in emphasizing that. Points 2. and 3. are indeed mentioned later in the document. I wanted to emphasize the connection between problems and the stakeholders. I still feel that there is ambiguity in what we actually mean by C API and whether the document is describing problems with the API that is public and intended to be consumed by external project (extensions) or also problems with CPython's internal abstractions that are AFAICS mixed together with the public C API, which I think increases the possibility of confusion. Similar to @hodgestar, I have also occasionally found that in certain discussions, it isn't always clear which API each party is referring to. Now I suggest just changing the introduction: as long as this is what this document should describe? |
Given that you somehow were confused about whether that's what was meant by "C API", I can't argue against this change (if it's the only change you now propose). To be clear, the intended readers of this document are those core developers who are in charge of maintaining and (re)designing the (public) C API (including the Steering Council, which has the ultimate authority -- though they very much prefer rough consensus among core developers to appear without their intervention). We want to hear from stakeholders, to make sure that they are heard; but this document is not intended for extension authors who just want to know about the C API or its future directions (as the document itself takes no position on the latter -- its intention is to inform the deciders). Considerations about priorities, for example, is in our future, but should not be included in this document. (The problem is definitely over-constrained, so setting priorities will be essential if we want to make decisions; but this document is not the place to state those priorities.) |
|
I've created a new PR for the change that we seem to be in agreement on |
Following my #66 (comment)