Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-31333: Re-implement ABCMeta in C #5273

Merged
merged 103 commits into from
Feb 18, 2018
Merged
Changes from 1 commit
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
5c34508
Initial work on C implementation of ABCMeta
ilevkivskyi Jul 23, 2017
cb7ffcf
Basic implementation of ABCMeta.__new__
ilevkivskyi Jul 25, 2017
b83ee80
Bare-bone implementation of register and subclass checks
ilevkivskyi Jul 25, 2017
181e83f
Fix mock failure and silence compiler warnings
ilevkivskyi Jul 26, 2017
c084a7f
Provide nicer dump of registry
ilevkivskyi Jul 26, 2017
a192d5d
Add better docstrings
ilevkivskyi Jul 26, 2017
35a2472
Expose the internal cavhes and registry (backward compatibility)
ilevkivskyi Jul 27, 2017
4812450
Merge remote-tracking branch 'upstream/master' into c-abc
ilevkivskyi Sep 3, 2017
b9038e2
Merge remote-tracking branch 'upstream/master' into c-abc
Jan 20, 2018
bbee578
Add _abc to Setup.dist
Jan 20, 2018
a3464fd
Fix _abc in Setup.dist
Jan 20, 2018
947bf7d
Update a comment
Jan 20, 2018
7ffc59e
Settle the .py version
Jan 20, 2018
41287a7
Fix some TODOs and refleaks
Jan 21, 2018
569cc44
Fix some more refleaks; use weak refs in registry
Jan 21, 2018
34665a8
Some more fixes; add some caching
Jan 21, 2018
576acac
Fix a crash due to erroneous DECREF
Jan 21, 2018
30098b4
Simplify some code; reorganize TODOs
Jan 22, 2018
51ede5d
Finish caches; add more comments
Jan 22, 2018
5263e1a
Use Py_RETURN_TRUE/FALSE and fix refleak
methane Jan 24, 2018
1f7aee9
Use Py_RETURN_NONE
methane Jan 24, 2018
11fea70
Use _PyObject_IsAbstract()
methane Jan 24, 2018
7ff3fbb
Use _PySet_NextEntry
methane Jan 24, 2018
9b4eb2f
Minor review comments
Jan 25, 2018
ab20a33
Sketch the new API
Jan 25, 2018
39f2692
Merge remote-tracking branch 'upstream/master' into c-abc
Jan 25, 2018
493d0ec
Use _PyObject_LookupAttr
Jan 25, 2018
2fe2c54
Some more progress
Jan 26, 2018
9476af6
Implement weakref callbacks and guarded iteration
Jan 26, 2018
ab68cdb
Fix some errors
Jan 26, 2018
3eb0a60
Fix two review comments
Jan 26, 2018
ed36b76
More fixes, test_abc passes
Jan 27, 2018
25fc5b9
Merge remote-tracking branch 'upstream/master' into c-abc
Jan 27, 2018
b2f75b9
Fix some remaining problems
Jan 27, 2018
cdb5cdf
Update TODO
Jan 27, 2018
a1a3a52
Add missing statics
Jan 27, 2018
a66b08c
Build on Windows
methane Jan 27, 2018
86af9ae
Refactor __abstractmethods__ calculation.
methane Jan 27, 2018
b22232a
Refactor via _abc_impl
Jan 27, 2018
e51c5ca
Fix some refleaks
Jan 27, 2018
bac7a43
Add docs (required by some tests) and initialization
Jan 27, 2018
4571649
Merge remote-tracking branch 'upstream/master' into c-abc
Jan 27, 2018
0d7513b
Outdated comment and detection of intrusions
Jan 27, 2018
c429f49
Minor fixes
Jan 27, 2018
357b56d
Few more refleaks
Jan 27, 2018
cd80fcb
Restore unwanted changes
Jan 28, 2018
34e13c3
Fix(?) some more refleaks
Jan 28, 2018
c5633b6
Reset caches between runs
Jan 28, 2018
3cbbc12
Fix abuse of borrowed reference
methane Jan 28, 2018
23bcb07
Fix remaining refleaks
Jan 28, 2018
0aab479
Updare TODO, switch to common result agreement, few more checks
Jan 28, 2018
86e0660
Fix typos
Jan 28, 2018
c55e482
Remove irrelevant TODOs, add few comments
Jan 28, 2018
5f9526a
Use Py_ssize_t for iterating
methane Jan 28, 2018
8174b61
Use c99 designated initializer.
methane Jan 28, 2018
bb8d623
Fix gset_new() and abc_data_new()
methane Jan 28, 2018
ef59e54
Massive refactoring...
methane Jan 28, 2018
22699fe
fixup
methane Jan 28, 2018
95cbf34
Review comments
Jan 28, 2018
4d596cc
More refactoring
Jan 28, 2018
fa3cba3
Typos and minor fixes
Jan 28, 2018
6f18293
Minor fixes and code style
Jan 28, 2018
9100891
Remove some unreachable code
Jan 28, 2018
36c5643
Review comments
Jan 28, 2018
dd2abda
Split the class into two separate implementations
Jan 28, 2018
99d950c
Add version independent (Py vs C) cache clearing
Jan 28, 2018
3762d49
Test the six-like tricky type.__new__(metaclass) with ABCMeta
Jan 28, 2018
404d1ce
Test both versions
Jan 28, 2018
0dc5fae
Add comment about testing
Jan 28, 2018
287b26a
Always DECREF after PyTuplr_Pack
Jan 28, 2018
ef34364
Use PySet_New() instead of calling copy() method.
methane Jan 29, 2018
d4d78a1
Copy set before iterating, and remove guarded set
methane Jan 29, 2018
f58822e
Add NULL check after PyWeakref_GetObject
methane Jan 30, 2018
3b74bdc
Add fast path for looking register
methane Jan 30, 2018
db1c852
Check negative cache version before cache lookup
methane Jan 30, 2018
6e62be7
Merge pull request #5 from methane/c-abc-no-guard-set
ilevkivskyi Feb 11, 2018
5384726
Fix nits pointed by pppery.
methane Feb 12, 2018
a48eecc
Create set lazily
methane Feb 14, 2018
16a8db1
Explicitly set NULL
methane Feb 14, 2018
5ad3ea8
Merge pull request #6 from methane/c-abc-lazyset
ilevkivskyi Feb 14, 2018
86a9b8d
Check PyObject_IsTrue() error
methane Feb 14, 2018
36c2013
Strip TODO comments.
methane Feb 14, 2018
09c5370
Add NEWS entry
methane Feb 14, 2018
207d8e9
Fix _reset_caches
methane Feb 15, 2018
a15377b
Add default: Py_UNREACHABLE()
methane Feb 15, 2018
d31da13
Rephrase NEWS entry
methane Feb 15, 2018
eaff1cb
Merge remote-tracking branch 'upstream/master' into c-abc
Feb 16, 2018
48de70e
Factor out Python version to a separate file
Feb 16, 2018
3bd0666
Factor out Python version to a separate file
Feb 16, 2018
702347a
Remove extra whitespace
Feb 16, 2018
e0c978b
Add more details to NEWS
Feb 16, 2018
b370dfe
Fix an import in refleak test
Feb 16, 2018
a1ae0a7
Restart tests
Feb 16, 2018
4746211
Make order of subclass checks in Python version stable and consistent…
Feb 16, 2018
001b416
Convert _abc to Argument Clinic
Feb 17, 2018
fc528df
Merge remote-tracking branch 'upstream/master' into c-abc
Feb 17, 2018
289c414
Regenerate clinic
Feb 17, 2018
ac0c639
Switch from Python invalidation counter to C long long
Feb 17, 2018
079e3be
The rest of the comments
Feb 17, 2018
9c49e5a
Merge remote-tracking branch 'upstream/master' into c-abc
Feb 17, 2018
4146588
Regenerate clinics.
Feb 17, 2018
c133605
Two more comments
Feb 17, 2018
f82e04d
Few more comments by Serhiy; add Whats New item
Feb 18, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 18 additions & 35 deletions Modules/_abc.c
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ _Py_IDENTIFIER(__subclasshook__);
negative cache to be cleared before its next use.
Note: this counter is private. Use `abc.get_cache_token()` for
external code. */
static PyObject *abc_invalidation_counter;
static unsigned long long abc_invalidation_counter = 0;

/* This object stores internal state for ABCs.
Note that we can use normal sets for caches,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is out of date

Expand All @@ -35,7 +35,7 @@ typedef struct {
PyObject *_abc_registry;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of registry, cache, and negative cache are normal sets of weak references; there should be comments stating that for either all of none of them

PyObject *_abc_cache; /* Normal set of weak references. */
PyObject *_abc_negative_cache; /* Normal set of weak references. */
PyObject *_abc_negative_cache_version;
unsigned long long _abc_negative_cache_version;
} _abc_data;

static void
Expand All @@ -44,7 +44,6 @@ abc_data_dealloc(_abc_data *self)
Py_XDECREF(self->_abc_registry);
Py_XDECREF(self->_abc_cache);
Py_XDECREF(self->_abc_negative_cache);
Py_DECREF(self->_abc_negative_cache_version);
Py_TYPE(self)->tp_free(self);
}

Expand All @@ -60,7 +59,6 @@ abc_data_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
self->_abc_cache = NULL;
self->_abc_negative_cache = NULL;
self->_abc_negative_cache_version = abc_invalidation_counter;
Py_INCREF(abc_invalidation_counter);
return (PyObject *) self;
}

Expand Down Expand Up @@ -239,7 +237,7 @@ static PyObject *
_abc__get_dump(PyObject *module, PyObject *self)
/*[clinic end generated code: output=9d9569a8e2c1c443 input=2c5deb1bfe9e3c79]*/
{
PyObject *registry, *cache, *negative_cache;
PyObject *registry, *cache, *negative_cache, *cache_version;
_abc_data *impl = _get_impl(self);
if (impl == NULL) {
return NULL;
Expand All @@ -262,13 +260,19 @@ _abc__get_dump(PyObject *module, PyObject *self)
Py_DECREF(cache);
return NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add Py_DECREF(cache)

}
Py_INCREF(impl->_abc_negative_cache_version); /* PyTuple_Pack doesn't do this. */
PyObject *res = PyTuple_Pack(4,
registry, cache, negative_cache, impl->_abc_negative_cache_version);
cache_version = PyLong_FromUnsignedLongLong(impl->_abc_negative_cache_version);
if (cache_version == NULL) {
Py_DECREF(impl);
Py_DECREF(registry);
Py_DECREF(cache);
Py_DECREF(negative_cache);
return NULL;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add Py_DECREF(cache)
Add Py_DECREF(negative_cache)

}
PyObject *res = PyTuple_Pack(4, registry, cache, negative_cache, cache_version);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be written simpler as:

PyObject *res = Py_BuildValue("NNNK",
                              PySet_New(impl->_abc_registry),
                              PySet_New(impl->_abc_cache),
                              PySet_New(impl->_abc_negative_cache),
                              impl->_abc_negative_cache_version);

Py_DECREF(registry);
Py_DECREF(cache);
Py_DECREF(negative_cache);
Py_DECREF(impl->_abc_negative_cache_version);
Py_DECREF(cache_version);
Py_DECREF(impl);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add all remaining decrefs

return res;
}
Expand Down Expand Up @@ -475,16 +479,7 @@ _abc__abc_register_impl(PyObject *module, PyObject *self, PyObject *subclass)
Py_DECREF(impl);

/* Invalidate negative cache */
PyObject *one = PyLong_FromLong(1);
if (one == NULL) {
return NULL;
}
PyObject *next_version = PyNumber_Add(abc_invalidation_counter, one);
Py_DECREF(one);
if (next_version == NULL) {
return NULL;
}
Py_SETREF(abc_invalidation_counter, next_version);
abc_invalidation_counter++;

Py_INCREF(subclass);
return subclass;
Expand Down Expand Up @@ -525,11 +520,7 @@ _abc__abc_instancecheck_impl(PyObject *module, PyObject *self,
}
subtype = (PyObject *)Py_TYPE(instance);
if (subtype == subclass) {
int r = PyObject_RichCompareBool(
impl->_abc_negative_cache_version,
abc_invalidation_counter, Py_EQ);
assert(r >= 0); // Both should be PyLong
if (r > 0) {
if (impl->_abc_negative_cache_version == abc_invalidation_counter) {
incache = _in_weak_set(impl->_abc_negative_cache, subclass);
if (incache < 0) {
goto end;
Expand Down Expand Up @@ -614,19 +605,13 @@ _abc__abc_subclasscheck_impl(PyObject *module, PyObject *self,
}

/* 2. Check negative cache; may have to invalidate. */
int r = PyObject_RichCompareBool(impl->_abc_negative_cache_version,
abc_invalidation_counter, Py_LT);
assert(r >= 0); // Both should be PyLong
if (r > 0) {
if (impl->_abc_negative_cache_version < abc_invalidation_counter) {
/* Invalidate the negative cache. */
if (impl->_abc_negative_cache != NULL &&
PySet_Clear(impl->_abc_negative_cache) < 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PEP 7 requires { on a separate line in this case.

goto end;
}
/* INCREF the new value of cache version,
then carefully DECREF the old one. */
Py_INCREF(abc_invalidation_counter);
Py_SETREF(impl->_abc_negative_cache_version, abc_invalidation_counter);
impl->_abc_negative_cache_version = abc_invalidation_counter;
}
else {
incache = _in_weak_set(impl->_abc_negative_cache, subclass);
Expand Down Expand Up @@ -811,8 +796,7 @@ static PyObject *
_abc_get_cache_token_impl(PyObject *module)
/*[clinic end generated code: output=c7d87841e033dacc input=1d49ab7218687f59]*/
{
Py_INCREF(abc_invalidation_counter);
return abc_invalidation_counter;
return PyLong_FromUnsignedLongLong(abc_invalidation_counter);
}

static struct PyMethodDef module_functions[] = {
Expand Down Expand Up @@ -848,6 +832,5 @@ PyInit__abc(void)
}
_abc_data_type.tp_doc = abc_data_doc;

abc_invalidation_counter = PyLong_FromLong(0);
return PyModule_Create(&_abcmodule);
}