[mono][wasm][AOT] Pin `OP_MOVE` destinations in the GC pin area

Project: dotnet/runtime | Language: C | Concepts: garbage-collection, memory-management, compiler-internals | Difficulty: senior
Link: https://github.com/dotnet/runtime/pull/132306

The problem

On Mono's WebAssembly (wasm) AOT backend, certain live references held in local variables could be prematurely collected by the garbage collector (GC). Specifically, when a reference-typed value was copied via an OP_MOVE instruction, the destination was not always written into the "GC pin area"—a region in linear memory used to keep references visible to the GC. This could result in the GC moving or collecting objects that were still in use, breaking object lifetime guarantees and causing subtle, hard-to-debug failures.

Why it's non-trivial

A naive fix would be to always pin all OP_MOVE destinations, but this would increase code size and memory usage unnecessarily. The correct solution must only pin moves that actually need it—i.e., where the source reference might be unrooted before the destination is dead, which is a subtle property depending on variable liveness and the internal IR structure. The fix must also avoid pinning locals that are already handled by other mechanisms (like address-taken variables or constants).

The approach

The PR changes the Mono LLVM AOT codegen to conditionally pin OP_MOVE destinations only when necessary. Below are the key code snippets and their roles:

1. The original exclusion of OP_MOVE from pinning:

-if (vreg_is_ref (cfg, ins->dreg) && ctx->values [ins->dreg] && ins->opcode != OP_MOVE && ins->opcode != OP_AOTCONST)
+if (vreg_is_ref (cfg, ins->dreg) && ctx->values [ins->dreg] && ins->opcode != OP_AOTCONST &&
+        (ins->opcode != OP_MOVE || move_needs_gc_pin (ctx, ins)))
         emit_gc_pin (ctx, builder, ins->dreg);

What it does:
Previously, OP_MOVE destinations were always excluded from being pinned. The new logic uses a helper (move_needs_gc_pin) to decide whether a move destination needs to be pinned, only pinning when necessary.

2. Counting vreg definitions to inform pinning decisions:

+	ctx->vreg_defcount = g_new0 (guint8, cfg->next_vreg);
+	for (MonoBasicBlock *bb = cfg->bb_entry; bb; bb = bb->next_bb) {
+		for (MonoInst *ins = bb->code; ins; ins = ins->next) {
+			if (ins->dreg >= 0 && (guint32)ins->dreg < cfg->next_vreg &&
+				LLVM_INS_INFO (ins->opcode) [MONO_INST_DEST] != ' ' &&
+				ctx->vreg_defcount [ins->dreg] < 2)
+				ctx->vreg_defcount [ins->dreg] ++;
+		}
+	}

What it does:
This pass counts how many times each virtual register is defined, saturating at 2. This information is used to determine if a move destination can become unrooted independently of its source.

3. The helper that decides if a move needs pinning:

+static gboolean
+move_needs_gc_pin (EmitContext *ctx, MonoInst *ins)
+{
+	MonoCompile *cfg = ctx->cfg;
+	MonoInst *var;
+
+	if (!ctx->vreg_defcount || ins->sreg1 < 0 || (guint32)ins->sreg1 >= cfg->next_vreg)
+		return TRUE;
+	if (ctx->vreg_defcount [ins->sreg1] > 1)
+		return TRUE;
+	var = get_vreg_to_inst (cfg, ins->sreg1);
+	return var && (var->flags & (MONO_INST_VOLATILE | MONO_INST_INDIRECT | MONO_INST_IS_DEAD));
+}

What it does:
This function determines whether a move destination needs to be pinned. It checks if the source vreg has multiple definitions or is address-taken/volatile/dead, in which case the move destination must be pinned to keep the object alive.

4. Ensuring the defcount computation runs at the right time:

+#ifdef TARGET_WASM
+	compute_vreg_defcounts (ctx);
+#endif

What it does:
This ensures the vreg definition counts are computed before code generation, so that the pinning logic has the necessary information.

5. Test coverage for the bug:

+        [Fact]
+        public static async Task MovedAliasOfAddressTakenLocalIsRootedAcrossGC()
+        {
+            // Resume on a shallow stack: under a deep caller frame a stale copy of the reference
+            // is often found by the conservative stack scan, which pins the object and hides the bug.
+            await Task.Yield();
+            RunMovedAliasLoop();
+        }

What it does:
This test ensures that a reference copied via a move remains rooted across a GC, catching regressions in the pinning logic.

From the review thread

BrzVlad: How is this pass related to the other llvm specific handling from above ? It simply operates on our own IR. Is this positioning here an artifact of a previous implementation that was doing this tracking only for indirect vars ? Which would have been incomplete fix, even though it addresses the scenario in the testcase ? In that case I don't believe it belongs in emit_entry_bb, maybe its own small function called from above.

pavelsavara: Moved out as compute_vreg_defcounts pass.

Takeaways

Correctness in memory management often requires precise, context-sensitive rooting logic—blanket solutions can be wasteful or insufficient.
Compiler transformations must account for platform-specific runtime constraints (e.g., wasm's lack of a scan-able stack), not just generic IR semantics.
Regression tests for subtle GC bugs should simulate real-world object lifetime patterns and verify invariants after forced collections.

[mono][wasm][AOT] Pin OP_MOVE destinations in the GC pin area