gh-155752: Do not crash when GenericAlias parameters change during substitution

Project: python/cpython | Language: Python | Concepts: error-handling, type-systems | Difficulty: mid
Link: https://github.com/python/cpython/pull/155761

The problem

When using Python's types.GenericAlias, if an argument gains a __typing_subst__ attribute after the alias's __parameters__ tuple has already been cached, the internal substitution logic would crash (segfault) instead of raising a controlled error. This could happen if the type parameters of a generic alias change after its creation, leading to an out-of-bounds lookup.

Why it's non-trivial

A naive fix might assume that the presence of __typing_subst__ is always in sync with the cached __parameters__ tuple. However, because Python objects are mutable, it's possible for __typing_subst__ to be added after the parameters are cached. The code must defensively handle this mismatch and not assume that every such argument is present in __parameters__.

The approach

1. Defensive check in C implementation

Previously, the code assumed that tuple_index would always return a valid index (>= 0) for arguments with __typing_subst__. This was enforced with an assert, which would crash if violated. The fix adds a check to handle the case where the index is not found:

-            assert(iparam >= 0);
-            arg = PyObject_CallOneArg(subst, argitems[iparam]);
+            if (iparam < 0) {
+                // __parameters__ may be stale if an argument gained
+                // __typing_subst__ after the tuple was computed.
+                PyErr_Format(PyExc_TypeError,
+                             "argument %R with __typing_subst__ was not found "
+                             "in __parameters__",
+                             arg);
+                arg = NULL;
+            }
+            else {
+                arg = PyObject_CallOneArg(subst, argitems[iparam]);
+            }

What this does:

• Checks if the argument is not found in __parameters__ (iparam < 0).
• If not found, raises a TypeError with a clear message, instead of crashing.
• If found, proceeds as before.

2. Regression test for the edge case

A new test is added to ensure that if an argument gains __typing_subst__ after the parameters are cached, the error is raised (not a crash):

+    def test_parameter_added_after_parameters_cached(self):
+        # gh-155752: GenericAlias parameters are cached before substitution, so
+        # an argument can gain __typing_subst__ after the tuple is calculated.
+        class Parameter:
+            pass
+
+        first = Parameter()
+        first.__typing_subst__ = lambda value: value
+        late = Parameter()
+        alias = types.GenericAlias(dict, (first, late))
+        self.assertEqual(alias.__parameters__, (first,))
+        late.__typing_subst__ = lambda value: value
+
+        with self.assertRaisesRegex(TypeError, "not found in __parameters__"):
+            alias[0]

What this does:

• Creates two parameter objects, only one of which initially has __typing_subst__.
• Creates a GenericAlias with both, which only caches the first as a parameter.
• Adds __typing_subst__ to the second after the fact.
• Asserts that using the alias with the "late" parameter raises a TypeError with the expected message.

3. News entry for the fix

A news file is added to document the bugfix for users:

+Fix a crash when a :class:`types.GenericAlias` argument gains a
+``__typing_subst__`` hook after the alias parameters have been cached.

From the review thread

picnixz: Can you put the comment inside the test as it is usually the case (but not as a docstring please)

dariushoule: Not a problem, I see that now.

Takeaways

• Never assume cached or precomputed values are always in sync with mutable object state; always check invariants at use sites.
• When handling internal errors, prefer raising explicit exceptions with clear messages over relying on assertions that may crash the interpreter.
• Regression tests for edge cases involving object mutability help prevent subtle bugs and future regressions.