Stale entries in MRO cache if MRO contains non-base classes

[colesbury]

Bug report

For example the following program fails with either assert WeirdClass.value == 2 or assert WeirdClass.value == 3 in recent Python versions:

import sys

class Base:
    value = 1

class Meta(type):
    def mro(cls):
        return (cls, Base, object)

class WeirdClass(metaclass=Meta):
    pass

assert Base.value == 1
assert WeirdClass.value == 1

Base.value = 2
assert Base.value == 2
assert WeirdClass.value == 2

Base.value = 3
assert Base.value == 3
assert WeirdClass.value == 3

Adding intervening calls to sys _clear_internal_caches() makes the test pass.

Version	Result
3.7	OK
3.8	OK
3.9	OK
3.10	AssertionError: assert WeirdClass.value == 2
3.11	AssertionError: assert WeirdClass.value == 2
3.12	AssertionError: assert WeirdClass.value == 2
3.13	AssertionError: assert WeirdClass.value == 3
main	AssertionError: assert WeirdClass.value == 3

We have code that checks for this case, but it hasn't worked properly in Python 3.10+:

cpython/Objects/typeobject.c

Lines 1107 to 1112 in 5c89adf

	static void
	type_mro_modified(PyTypeObject *type, PyObject *bases) {
	/*
	Check that all base classes or elements of the MRO of type are
	able to be cached. This function is called after the base
	classes or mro of the type are altered.

We also have a test that partly covers this case, but doesn't detect the bug:

cpython/Lib/test/test_capi/test_type.py

Lines 40 to 66 in 5c89adf

	def test_freeze_meta(self):
	"""test PyType_Freeze() with overridden MRO"""
	type_freeze = _testcapi.type_freeze
	class Base:
	value = 1
	class Meta(type):
	def mro(cls):
	return (cls, Base, object)
	class FreezeThis(metaclass=Meta):
	"""This has `Base` in the MRO, but not tp_bases"""
	self.assertEqual(FreezeThis.value, 1)
	with self.assertRaises(TypeError):
	type_freeze(FreezeThis)
	Base.value = 2
	self.assertEqual(FreezeThis.value, 2)
	type_freeze(Base)
	with self.assertRaises(TypeError):
	Base.value = 3
	type_freeze(FreezeThis)
	self.assertEqual(FreezeThis.value, 2)

Linked PRs

• gh-127773: Disable attribute cache on incompatible MRO entries #127924

[colesbury]

cc @encukou - this broke in a4760cc (GH-27260), which removed Py_TPFLAGS_HAVE_VERSION_TAG.

The reason for removing Py_TPFLAGS_HAVE_VERSION_TAG was because "the field is always present in the type struct", but the flag indicates whether a type supports the method cache (i.e, supports having a valid version tag) and types with MRO entries that aren't superclasses don't support the method cache.

[cfbolz]

Great (if very annoying) find, @colesbury! The bug even affects pypy, which has an entirely different architecture for its caches 😳

[colesbury]

I think this was fixed by @encukou

[encukou]

In the main branch. I don't know if I'll get to older branches in time.