On SM100/SM103 GPUs, the DSv4.1 output projection ran four separate kernels per layer, causing latency overhead at decode batch sizes due to sequential kernel launches and data transfers.

The original design split inverse RoPE, FP8 quantization, and matrix multiplications into separate kernels, leading to redundant memory operations and synchronization delays, especially for small batch sizes common in decoding.

Introduced a fused CuTe-DSL kernel, FusedWoAKernel, that combines inverse RoPE, per-32 FP8 quantization, and the wo_a matrix multiplication with MXFP8 quantization into a single pipelined kernel. This kernel prefetches weights under producer dependency, overlaps RoPE coefficient loading, and uses split-K MMA with efficient synchronization to reduce latency and memory overhead.

Fusing multiple small kernels into a single pipelined GPU kernel can significantly reduce latency and improve throughput for small-batch workloads by minimizing memory traffic and synchronization costs, especially on advanced GPU architectures like SM100/SM103.