gh-101659: Isolate "obmalloc" State to Each Interpreter #101660

ericsnowcurrently · 2023-02-07T18:58:07Z

This is strictly about moving the "obmalloc" runtime state from _PyRuntimeState to PyInterpreterState. Doing so improves isolation between interpreters, specifically most of the memory (incl. objects) allocated for each interpreter's use. This is important for a per-interpreter GIL, but such isolation is valuable even without it.

FWIW, a per-interpreter obmalloc is the proverbial canary-in-the-coalmine when it comes to the isolation of objects between interpreters. Any object that leaks (unintentionally) to another interpreter is highly likely to cause a crash (on debug builds at least). That's a useful thing to know, relative to interpreter isolation.

Benchmarking indicates the performance impact is negligible. (I'll re-run soon to double-check.)

Issue: Isolate the Default Object Allocator between Interpreters #101659

ericsnowcurrently · 2023-02-09T16:54:24Z

FYI, the current CI failures are due to existing code. Basically, that code (in _PyImport_FixupExtensionObject()) tries to delete a [currently] global object that was created by a different interpreter than the "current" one. (There may be a few other similarly problematic global objects. I'll have to check.) As to why only some of the jobs are failing: the failure is a check by the debug allocator, which is only used in Py_DEBUG builds.

FWIW, the failure is exactly what we should expect to happen when per-interpreter data breaks isolation. I actually spent a while trying to figure out what I was doing wrong in my branch before realizing that everything was working as it should. 😄

UPDATE: I've opened gh-101758 to address this.

ericsnowcurrently · 2023-02-21T18:12:42Z

Given what I've determined via gh-101758, I'll probably need to make non-isolated interpreters share the obmalloc state with the main interpreter.

ericsnowcurrently · 2023-02-27T16:33:30Z

I'm tabling this until we've isolated all non-static objects.

… init extensions.

The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on gh-101660.)

bedevere-bot · 2023-04-06T00:43:28Z

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit 22758a3 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

In pythongh-102744 we added is_core_module() (in Python/import.c), which relies on get_core_module_dict() (also added in that PR). The problem is that_PyImport_FixupBuiltin(), which ultimately calls is_core_module(), is called on the builtins module before interp->builtins_copyis set. Consequently, the builtins module isn't considered a "core" module while it is getting "fixed up" and its module def m_copy erroneously gets set. Under isolated interpreters this causes problems since sys and builtins are allowed even though they are still single-phase init modules. (This was discovered while working on pythongh-101660.) The solution is to stop relying on get_core_module_dict() in is_core_module().

pythongh-103287) Using the raw allocator for any of the global state makes sense, especially as we move to a per-interpreter obmalloc state (pythongh-101660).

The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on pythongh-101660.)

…ules (pythongh-102661) It doesn't make sense to use multi-phase init for these modules. Using a per-interpreter "m_copy" (instead of PyModuleDef.m_base.m_copy) makes this work okay. (This came up while working on pythongh-101660.) Note that we might instead end up disallowing re-load for sys/builtins since they are so special. python#102660

…02658) The error-handling code in new_interpreter() has been broken for a while. We hadn't noticed because those code mostly doesn't fail. (I noticed while working on pythongh-101660.) The problem is that we try to clear/delete the newly-created thread/interpreter using itself, which just failed. The solution is to switch back to the calling thread state first. python#98608

…102663) Aside from sys and builtins, _io is the only core builtin module that hasn't been ported to multi-phase init. We may do so later (e.g. pythongh-101948), but in the meantime we must at least take care of the module's static types properly. (This came up while working on pythongh-101660.) python#94673

…rpreters (pythongh-102925) This is effectively two changes. The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.). The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly. Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict). _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters. This is especially important as we move the obmalloc state to each interpreter (pythongh-101660), as well as, potentially, the GIL (PEP 684). python#100227

…Interpreters (pythongh-103084) Sharing mutable (or non-immortal) objects between interpreters is generally not safe. We can work around that but not easily. There are two restrictions that are critical for objects that break interpreter isolation. The first is that the object's state be guarded by a global lock. For now the GIL meets this requirement, but a granular global lock is needed once we have a per-interpreter GIL. The second restriction is that the object (and, for a container, its items) be deallocated/resized only when the interpreter in which it was allocated is the current one. This is because every interpreter has (or will have, see pythongh-101660) its own object allocator. Deallocating an object with a different allocator can cause crashes. The dict for the cache of module defs is completely internal, which simplifies what we have to do to meet those requirements. To do so, we do the following: * add a mechanism for re-using a temporary thread state tied to the main interpreter in an arbitrary thread * add _PyRuntime.imports.extensions.main_tstate` * add _PyThreadState_InitDetached() and _PyThreadState_ClearDetached() (pystate.c) * add _PyThreadState_BindDetached() and _PyThreadState_UnbindDetached() (pystate.c) * make sure the cache dict (_PyRuntime.imports.extensions.dict) and its items are all owned by the main interpreter) * add a placeholder using for a granular global lock Note that the cache is only used for legacy extension modules and not for multi-phase init modules. python#100227

Decref the key in the right interpreter in _extensions_cache_set(). This is a follow-up to pythongh-103084. I found the bug while working on pythongh-101660.

In pythongh-102744 we added is_core_module() (in Python/import.c), which relies on get_core_module_dict() (also added in that PR). The problem is that_PyImport_FixupBuiltin(), which ultimately calls is_core_module(), is called on the builtins module before interp->builtins_copyis set. Consequently, the builtins module isn't considered a "core" module while it is getting "fixed up" and its module def m_copy erroneously gets set. Under isolated interpreters this causes problems since sys and builtins are allowed even though they are still single-phase init modules. (This was discovered while working on pythongh-101660.) The solution is to stop relying on get_core_module_dict() in is_core_module().

pythongh-103287) Using the raw allocator for any of the global state makes sense, especially as we move to a per-interpreter obmalloc state (pythongh-101660).

The function is like Py_AtExit() but for a single interpreter. This is a companion to the atexit module's register() function, taking a C callback instead of a Python one. We also update the _xxinterpchannels module to use _Py_AtExit(), which is the motivating case. (This is inspired by pain points felt while working on pythongh-101660.)

vstinner · 2023-10-30T15:17:06Z

This change broke PYTHONMALLOCSTATS: see issue #111499.

ericsnowcurrently added 2 commits February 7, 2023 10:02

Pass PyInterpreterState to pymalloc_*().

07a09d4

Move the object arenas to the interpreter state.

ca75048

ericsnowcurrently added the skip news label Feb 7, 2023

bedevere-bot added the awaiting core review label Feb 7, 2023

bedevere-bot mentioned this pull request Feb 7, 2023

Isolate the Default Object Allocator between Interpreters #101659

Closed

ericsnowcurrently added 2 commits February 7, 2023 12:17

Drop an errant #define.

4ee199b

Leave dump_debug_stats in the global state.

2768fa4

erlend-aasland previously approved these changes Feb 7, 2023

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Feb 7, 2023

Dynamically initialize obmalloc for subinterpreters.

bf9425f

ericsnowcurrently mentioned this pull request Feb 14, 2023

gh-101758: Add a Test For Single-Phase Init Modules in Multiple Interpreters #101920

Merged

ericsnowcurrently marked this pull request as draft February 27, 2023 16:33

ericsnowcurrently added 14 commits March 8, 2023 17:44

Merge branch 'main' into per-interpreter-alloc

d5da34b

Pass around struct _obmalloc_state* instead of PyInterpeterState*.

6c3111c

Add _PyInterpreterConfig.use_main_obmalloc.

4dc087d

Add a comment about why per-interpreter obmalloc requires multi-phase…

1ae33a0

… init extensions.

Add a TODO comment.

5b54d63

Optionally use the main interpreter's obmalloc state.

9f4f8f3

Pass use_main_obmalloc to run_in_subinterp() in test_import.

aa10204

_Py_GetAllocatedBlocks() -> _Py_GetGlobalAllocatedBlocks().

69d9a2d

Errors from _Py_NewInterpreterFromConfig() are no longer fatal.

25378f8

Chain the exceptions.

1c5b109

Swap out the failed tstate.

f36426b

Remaining static builtin types must be fixed.

54b9f09

Add PyInterpreterState.sysdict_copy.

2358a42

Set m_copy to None for sys and builtins.

b6502e1

Merge branch 'atexit-c-callback' into per-interpreter-alloc

df77a64

ericsnowcurrently requested a review from a team as a code owner April 6, 2023 00:09

Merge branch 'main' into per-interpreter-alloc

22758a3

ericsnowcurrently added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 6, 2023

bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 6, 2023

Merge branch 'main' into per-interpreter-alloc

0fd74a9

ericsnowcurrently requested review from brettcannon, encukou, ncoghlan and warsaw as code owners April 24, 2023 19:54

ericsnowcurrently merged commit df3173d into python:main Apr 24, 2023

bedevere-bot removed the awaiting review label Apr 24, 2023

ericsnowcurrently deleted the per-interpreter-alloc branch April 24, 2023 23:24

ksunden mentioned this pull request Nov 8, 2023

[Bug]: Segmentation fault when resizing on Python 3.12 and MacOS 14 matplotlib/matplotlib#27262

Closed

UTsweetyfish mentioned this pull request Jul 10, 2024

【deepin_V23_Release】【一般】【立即】【集成测试】【系统】onboard报错“段错误” linuxdeepin/developer-center#9622

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-101659: Isolate "obmalloc" State to Each Interpreter #101660

gh-101659: Isolate "obmalloc" State to Each Interpreter #101660

ericsnowcurrently commented Feb 7, 2023 •

edited

Loading

ericsnowcurrently commented Feb 9, 2023 •

edited

Loading

ericsnowcurrently commented Feb 21, 2023

ericsnowcurrently commented Feb 27, 2023

bedevere-bot commented Apr 6, 2023

vstinner commented Oct 30, 2023

gh-101659: Isolate "obmalloc" State to Each Interpreter #101660

gh-101659: Isolate "obmalloc" State to Each Interpreter #101660

Conversation

ericsnowcurrently commented Feb 7, 2023 • edited Loading

ericsnowcurrently commented Feb 9, 2023 • edited Loading

ericsnowcurrently commented Feb 21, 2023

ericsnowcurrently commented Feb 27, 2023

bedevere-bot commented Apr 6, 2023

vstinner commented Oct 30, 2023

ericsnowcurrently commented Feb 7, 2023 •

edited

Loading

ericsnowcurrently commented Feb 9, 2023 •

edited

Loading