Segfault on frame.f_back when frame is created with PyFrame_New()

[jpe]

Python segfaults when frame.f_back is accessed on a frame created with PyFrame_New() c api. Calling the PyFrame_GetBack() c api also segfaults, at least in debug builds and on win32 (it depends on the contents of uninitialized memory). Tested with 3.11.0 and git 3.11 branch as of Nov 4, 2022

Cause is that the ->previous field of the _PyInterpreterFrame is never set to NULL and when PyFrame_GetBack() runs, it tries to dereference the pointer value of ->previous and segfaults. A test case using ctypes is attached.

Adding a frame->previous = NULL; line to init_frame() in frameobject.c fixes this, though I don't know if it's the best place for it.

f_back_segfault.py.txt

Linked PRs

• gh-99110: Initialize frame->previous in init_frame to fix segmentation fault #100182
• [3.11] gh-99110: Initialize frame->previous in init_frame to fix segmentation fault (GH-100182) #100478

[brandtbucher]

Thanks, I'll take a closer look at this soon.

[byllyfish]

I am also seeing this issue when using the grpc extension which has an asyncio task written in cython. This crash is new to 3.11.0 and 3.11.1; there was no problem with 3.10.x.

Here's a valgrind stack trace from Python 3.11.1 on Linux. The program test.py is the same as f_back_segfault.py.txt mentioned above. (There are other stack traces in the linked byllyfish/finsy issue.). The address 0xcdcdcd... indicates the code tried to dereference an uninitialized pointer.

$ valgrind python -X dev test.py 
==75321== Memcheck, a memory error detector
==75321== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==75321== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==75321== Command: python -X dev test.py
==75321== 
<frame at 0x52bfe90, file '/home/bfish/code/finsy/test.py', line 21, code greet>
==75321== Invalid read of size 1
==75321==    at 0x2859F0: _PyFrame_IsIncomplete (pycore_frame.h:147)
==75321==    by 0x2859F0: PyFrame_GetBack (frameobject.c:1326)
==75321==    by 0x2859F0: frame_getback (frameobject.c:103)
==75321==    by 0x2BBFAE: _PyObject_GenericGetAttrWithDict (object.c:1278)
==75321==    by 0x2BB46A: PyObject_GetAttr (object.c:916)
==75321==    by 0x1F9B17: _PyEval_EvalFrameDefault (ceval.c:3464)
==75321==    by 0x36987E: _PyEval_EvalFrame (pycore_ceval.h:73)
==75321==    by 0x36987E: _PyEval_Vector (ceval.c:6435)
==75321==    by 0x36987E: PyEval_EvalCode (ceval.c:1154)
==75321==    by 0x3B90AB: run_eval_code_obj (pythonrun.c:1714)
==75321==    by 0x3B90AB: run_mod (pythonrun.c:1735)
==75321==    by 0x3BA801: pyrun_file (pythonrun.c:1630)
==75321==    by 0x3BA801: _PyRun_SimpleFileObject (pythonrun.c:440)
==75321==    by 0x3BADBE: _PyRun_AnyFileObject (pythonrun.c:79)
==75321==    by 0x3DCEF7: pymain_run_file_obj (main.c:360)
==75321==    by 0x3DCEF7: pymain_run_file (main.c:379)
==75321==    by 0x3DCEF7: pymain_run_python.constprop.0 (main.c:601)
==75321==    by 0x3DD5EF: Py_RunMain (main.c:680)
==75321==    by 0x3DD5EF: pymain_main (main.c:710)
==75321==    by 0x3DD5EF: Py_BytesMain (main.c:734)
==75321==    by 0x4973D8F: (below main) (libc_start_call_main.h:58)
==75321==  Address 0xcdcdcdcdcdcdce12 is not stack'd, malloc'd or (recently) free'd
==75321== 
Fatal Python error: Segmentation fault
... (rest elided)

frame->previous is not initialized in _PyFrame_InitializeSpecials(). Setting it to NULL there fixes the issues with grpc/cython and also jpe's original complaint.

diff --git a/Include/internal/pycore_frame.h b/Include/internal/pycore_frame.h
index ecc8346..6e73e1f 100644
--- a/Include/internal/pycore_frame.h
+++ b/Include/internal/pycore_frame.h
@@ -107,6 +107,7 @@ _PyFrame_InitializeSpecials(
     frame->f_locals = Py_XNewRef(locals);
     frame->stacktop = nlocalsplus;
     frame->frame_obj = NULL;
+    frame->previous = NULL;
     frame->prev_instr = _PyCode_CODE(frame->f_code) - 1;
     frame->is_entry = false;
     frame->owner = FRAME_OWNED_BY_THREAD;

If frame->previous should be initialized somewhere else, I would still put a comment in _PyFrame_InitializeSpecials to explain why it wasn't initialized there, because it looked weird to me that it was missing. Thanks.

[jpe]

Would it help to submit a pull request for this? I didn't submit one initially because it's only one line and there's two places it could go.

[byllyfish]

I will submit a pull request to the main branch (3.12) tomorrow. Thanks.

[illia-v]

I am not sure how much this issue is related to #100126.
It looks like a new segfault was introduced in recent b72014c, it is related to frames too.

[jpe]

Did the fix get committed to the 3.11 branch?

[zyv]

@brandtbucher could you please have a look at getsentry/sentry-python#2386 ?

We are facing further segfaults in _PyFrame_IsIncomplete / PyFrame_GetBack on Python 3.11.5, where it seems multiple issues of that sort should be already fixed. So it seems that there is something still going on... Should we report the issue again CPython?

Here is our backtrace from one of the core dumps:

#0  0x0000ffffa0263a8c in _PyFrame_IsIncomplete (frame=0xffff9fa9b2f8) at ./Include/internal/pycore_frame.h:151
#1  PyFrame_GetBack (frame=<optimized out>) at Objects/frameobject.c:1328
#2  0x0000ffffa027073c in frame_getback (f=<optimized out>, closure=<optimized out>) at Objects/frameobject.c:103
#3  0x0000ffffa0195730 in _PyObject_GenericGetAttrWithDict
    (obj=Frame 0xffff9fa9b378, for file /usr/local/lib/python3.11/site-packages/typing_inspect.py, line 232, in is_union_type (tp=<type at remote 0xffffa04568d8>), name='f_back', dict=0x0, suppress=0) at ./Include/object.h:133
#4  0x0000ffffa017915c in PyObject_GenericGetAttr (name='f_back', obj=Frame 0xffff9fa9b378, for file /usr/local/lib/python3.11/site-packages/typing_inspect.py, line 232, in is_union_type (tp=<type at remote 0xffffa04568d8>))
    at Objects/object.c:1368
#5  PyObject_GetAttr (v=Frame 0xffff9fa9b378, for file /usr/local/lib/python3.11/site-packages/typing_inspect.py, line 232, in is_union_type (tp=<type at remote 0xffffa04568d8>), name='f_back') at Objects/object.c:916
#6  0x0000ffffa0182870 in _PyEval_EvalFrameDefault (tstate=0xffff90001ed0, frame=0xffff9fe18428, throwflag=-1614065056) at Python/ceval.c:3466
#7  0x0000ffffa01805d4 in _PyEval_EvalFrame (throwflag=0, frame=0xffff9fe18230, tstate=0xffff90001ed0) at ./Include/internal/pycore_ceval.h:73
#8  _PyEval_Vector (tstate=0xffff90001ed0, func=func@entry=0xffff9d1ecae0, locals=locals@entry=0x0, args=args@entry=0xffff8fffe4f8, argcount=argcount@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:6439
#9  0x0000ffffa01b8b1c in _PyFunction_Vectorcall (kwnames=0x0, nargsf=1, stack=0xffff8fffe4f8, func=<function at remote 0xffff9d1ecae0>) at Objects/call.c:393
#10 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0xffff8fffe4f8, callable=<function at remote 0xffff9d1ecae0>, tstate=0xffff90001ed0) at ./Include/internal/pycore_call.h:92
#11 method_vectorcall (method=<optimized out>, args=0xffffa04886f8 <_PyRuntime+58944>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:67
#12 0x0000ffffa0184414 in _PyEval_EvalFrameDefault (tstate=0xffff90001ed0, frame=0xffff9fe181b8, throwflag=-1614065056) at Python/ceval.c:5381
#13 0x0000ffffa01805d4 in _PyEval_EvalFrame (throwflag=0, frame=0xffff9fe181b8, tstate=0xffff90001ed0) at ./Include/internal/pycore_ceval.h:73
#14 _PyEval_Vector (tstate=0xffff90001ed0, func=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=<optimized out>) at Python/ceval.c:6439
#15 0x0000ffffa0184414 in _PyEval_EvalFrameDefault (tstate=0xffff90001ed0, frame=0xffff9fe18110, throwflag=-1614065056) at Python/ceval.c:5381
#16 0x0000ffffa01805d4 in _PyEval_EvalFrame (throwflag=0, frame=0xffff9fe18020, tstate=0xffff90001ed0) at ./Include/internal/pycore_ceval.h:73
#17 _PyEval_Vector (tstate=0xffff90001ed0, func=func@entry=0xffff9f68a020, locals=locals@entry=0x0, args=args@entry=0xffff8fffe8c8, argcount=argcount@entry=1, kwnames=kwnames@entry=0x0) at Python/ceval.c:6439
#18 0x0000ffffa01b8b1c in _PyFunction_Vectorcall (kwnames=0x0, nargsf=1, stack=0xffff8fffe8c8, func=<function at remote 0xffff9f68a020>) at Objects/call.c:393
#19 _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0xffff8fffe8c8, callable=<function at remote 0xffff9f68a020>, tstate=0xffff90001ed0) at ./Include/internal/pycore_call.h:92
#20 method_vectorcall (method=<optimized out>, args=0xffffa04886f8 <_PyRuntime+58944>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/classobject.c:67
#21 0x0000ffffa026b230 in thread_run (boot_raw=0xffff95890f90) at ./Modules/_threadmodule.c:1092
#22 0x0000ffffa026b1d0 in pythread_wrapper (arg=<optimized out>) at Python/thread_pthread.h:241
#23 0x0000ffff9fe9ee18 in start_thread (arg=0xffff9555d537) at ./nptl/pthread_create.c:442
#24 0x0000ffff9ff07e9c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79

/cc @IamAbbey