gh-154088: Count elided stacks consistently

Project: python/cpython | Language: Python | Concepts: refactoring, testing-strategy, observability | Difficulty: mid
Link: https://github.com/python/cpython/pull/154093

The problem

The profiler's differential flamegraph feature was inconsistently counting and displaying "elided" stacks—call stacks that were present in a baseline profile but missing in the current one. This led to inaccurate reporting of how many unique stacks disappeared between two profiling runs.

Why it's non-trivial

A naive fix might just count the number of missing call paths (elided paths), but this would undercount cases where both internal (non-leaf) and leaf nodes in the call stack have self samples. Additionally, the flamegraph rendering logic could filter out small stacks, causing some elided stacks to be omitted from the output, further reducing accuracy.

The approach

1. Make the minimum sample filter configurable and ensure elided stacks are always counted:

-    def _convert_to_flamegraph_format(self):
+    def _convert_to_flamegraph_format(self, *, min_samples=None):

This change allows the minimum sample threshold to be overridden, ensuring that even stacks with only one sample (which might otherwise be filtered out) can be included when necessary.

2. Only apply the minimum sample threshold if not explicitly set:

-        min_samples = max(1, int(total_samples * 0.001))
+        if min_samples is None:
+            min_samples = max(1, int(total_samples * 0.001))

This ensures that, by default, insignificant stacks are filtered, but tests or internal logic can request all stacks be included.

3. Count every elided path with self samples, not just leaves:

-        current_flamegraph["stats"]["elided_count"] = len(self._elided_paths)
+        # A sampled stack can end at an internal path that also has elided
+        # descendants.  Count every disappeared path with self samples, not
+        # just the leaves of the elided path tree.
+        elided_stacks = {
+            path
+            for path in self._elided_paths
+            if baseline_stats[path]["self"] > 0
+        }
+        current_flamegraph["stats"]["elided_count"] = len(elided_stacks)

This code ensures that the count reflects all missing stacks that had samples, not just the terminal nodes.

4. Always render elided stacks in the flamegraph, even if they would be filtered out by the significance threshold:

-            baseline_data = self._baseline_collector._convert_to_flamegraph_format()
+            baseline_data = self._baseline_collector._convert_to_flamegraph_format(
+                min_samples=1
+            )

Setting min_samples=1 guarantees that even tiny elided stacks are rendered in the differential flamegraph.

5. Add targeted tests to verify the new behavior:

+    def test_diff_flamegraph_counts_elided_stacks_not_paths(self):
+        """Internal and leaf stack endings are counted separately."""
+        ...
+        self.assertEqual(data["stats"]["elided_count"], 2)
+
+    def test_diff_flamegraph_renders_small_elided_stack(self):
+        """Elided stacks are not removed by the significance filter."""
+        ...
+        self.assertEqual(data["stats"]["elided_count"], 1)
+        self.assertIn("elided_flamegraph", data["stats"])
+        ...
+        self.assertIsNotNone(
+            find_child_by_name(elided.get("children", []), strings, "old_tiny")
+        )

These tests ensure that both internal and leaf elided stacks are counted, and that small stacks are not omitted from the output.

From the review thread

No notable review discussion on this PR.

Takeaways

When reporting differences between two profiles, count every unique stack with self samples, not just the leaves or unique paths.
Make filtering thresholds configurable to allow tests and internal logic to bypass significance filters when accuracy is more important than noise reduction.
Always add regression tests that cover subtle distinctions in counting or rendering logic, especially for observability and reporting features.