Modify _PyObject_GC_TRACK() to ensure that newly tracked object is valid

[vstinner]

BPO	40142
Nosy	@vstinner
Files	track.patch gc_track.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-05-14.23:00:43.056>
created_at = <Date 2020-04-01.21:55:41.922>
labels = ['interpreter-core', 'type-feature', '3.9']
title = 'Modify _PyObject_GC_TRACK() to ensure that newly tracked object is valid'
updated_at = <Date 2021-06-29.02:25:22.979>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-06-29.02:25:22.979>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-05-14.23:00:43.056>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-04-01.21:55:41.922>
creator = 'vstinner'
dependencies = []
files = ['49021', '50130']
hgrepos = []
issue_num = 40142
keywords = ['patch']
message_count = 5.0
messages = ['365515', '368873', '396693', '396697', '396698']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40142'
versions = ['Python 3.9']

[vstinner]

In bpo-38392, I modified PyObject_GC_Track() to ensure that the object newly tracked is valid: call its traverse function.
=> commit 1b18455

I propose to now also attempt to implement the same check in _PyObject_GC_TRACK() which is part of the internal C API.

PyType_GenericAlloc() allocates memory for a type allocated on the heap... and then immediately track it in the GC. Problem: this type is not initialized yet, all fields are set to zero. Calling type_traverse() at this point fails with an assertion error:
---
Objects/typeobject.c:3570: type_traverse: Assertion failed: type_traverse() called on non-heap type '(null)'
Enable tracemalloc to get the memory block allocation traceback

object address : 0x840860
object refcount : 1
object type : 0x7e0900
object type name: type
object repr :
---

By the way, Python crash in _PyObject_Dump() on PyObject_Repr() call: type_repr() crash when accessing type->tp_name which is NULL.

type_call() should only track the newly created type when it's fully initialized.

Try attached track.patch to reproduce the crash.

[vstinner]

While it might be doable, I don't have the bandwidth to investigate this issue and so I prefer to close it as out of date.

[vstinner]

PyType_GenericAlloc() cannot traverse the object since the object members are not initialized yet. For example, dict_traverse() can only be called when PyDict_New() completes.

A different approach would be to:

• (1) Add PyType_AllocNoTrack(), use it in built-in types, and call _PyObject_GC_TRACK() on the instance once it is fully initialized.
• (2) Modify PyType_GenericAlloc() to use a new variant of _PyObject_GC_TRACK() which will not traverse the instance.
• (3) Modify _PyObject_GC_TRACK() to traverse the instance.

In short, PyType_GenericAlloc() cannot be modified for backward compatibility.

Moreover, _PyObject_GC_TRACK() should only be used inside Python internals, since it's part the internal C API.

[vstinner]

I created bpo-44531 "Add _PyType_AllocNoTrack() function: allocate without tracking in the GC".

[vstinner]

Attached gc_track.patch: my latest attempt to implement this idea.