Document contributors convention to respect skeleton and distutils

[abravalheri]

Summary of changes

I created these paragraphs based on the comment in #4212. The idea is to document to setuptools developers how the compat/pyXX convention works.

Closes

Pull Request Checklist

• Changes have tests
• News fragment added in newsfragments/.
  (See documentation for details)

[Avasam]

I have two questions. If these are implementation details, should the module be named _compat? And should it include _importlib.py which are just conditional imports based on Python version?

[abravalheri]

Yes, I think they could be moved to _compat. @jaraco would that be OK?

Probably importlib is treated as a special case, the file used to be much more complex than it is nowadays.

[jaraco]

Thanks for doing this!

I have two questions. If these are implementation details, should the module be named _compat?

I find the _ prefix for modules particularly abrasive. It looks like lint, like a typo. I don't want to be forced to add weird characters to modules. Especially because in many packages, all of the public interfaces are available from the top-level package, so everything else is de facto private. I'd much rather have simple, clean names like compat.

Presumably to address this concern, I've seen other projects put everything under a separate, top-level package (e.g. pytest [although even there, they have private packages below]) or move everything private into an underscore-prefixed private package (e.g. pip).

I've resisted these approaches because they add extra indirection and top-level names. I'd much rather be able to name modules with a clean, natural form and flag visibility using some orthogonal signal. Since the language doesn't provide a good orthogonal signal, I'm fine with documenting the expectation.

I do think downstream consumers should have an instinct to avoid using functionality from a package that's not part of the advertised interface. That is, I don't think users would be particularly tempted to use these modules.

In cases where the functionality might be valuable across more than a couple of projects, I've worked to expose those under jaraco.compat.

And should it include _importlib.py which are just conditional imports based on Python version?

In my opinion, yes. In fact, it might be worthwhile refactoring those imports into py39 and py310 modules, but it'd be fine to call it compat/importlib.py (or compat/_importlib.py) and use as-is. It would also be fine to leave it where it is. It's not crucial that a module go there, but if you find yourself creating a module called macos_compat, it should probably be compat.macos.

I also use this namespace for things unrelated to the Python version, such as keyring.compat.properties or zipp.compat.overlay. In the case of zipp.compat.overlay, I do expect it to be used by external consumers.

Yes, I think they could be moved to _compat. @jaraco would that be OK?

Yes, it would be okay, but then it will deviate from the convention I've followed in most other projects. IMO, it'd be better to keep it consistent across projects.

Document internal convention on compat modules

Thinking about this concept in general, I apply it across many projects, so it feels a little weird to document it here. It will be a little unsustainable to copy this documentation to each and every project that employs this approach. Better would be to have a blog entry or (even better) have documentation in a shared system (like skeleton or coherent) that can possibly be linked from Setuptools if needed.

What do you think about contributing this guidance instead to a "conventions" section in https://github.com/jaraco/skeleton/blob/gh-pages/index.md?

[jaraco]

In cases where the functionality might be valuable across more than a couple of projects, I've worked to expose those under jaraco.compat.

The only reason I haven't added shims for importlib resources and importlib metadata to jaraco.compat is because there's no one-size-fits-all approach - each consumer can have different compatibility expectations. Although, it occurs to me that some package could present a compat.pyNNN.importlib.{resources,metadata} for each pertinent value of NNN. Maybe I'll explore that at some point.

[abravalheri]

Thank you very much for the feedback @jaraco, I have extracted the "compat" docs into jaraco/skeleton#145.

I have also repurposed this PR to talk about how setuptools uses skeleton and that setuptools/_distutils should not be changed directly.