[Model] Add Reflection Beam
The Issue & Context
Before this PR, vLLM lacked support for running Reflection's Beam-1.0 model, a massive mixture-of-experts architecture with 23B active parameters, limiting users from leveraging this state-of-the-art model and its specialized decoding strategies.
Root Cause Analysis
The existing codebase had no implementations for Beam-1.0's unique architecture, such as its hybrid sliding-window RoPE and full-attention NoPE layers, per-head Q/K normalization with sigmoid gating, and DeepSeek-V3 style MoE routing, nor support for its DFlash2 speculative decoding and fused kernels optimized for NVIDIA hardware.
Solution Strategy & Architectural Pattern
This PR adds a new model implementation under vllm/models/beam/nvidia including BeamForCausalLM and DFlash2BeamDraftModel classes, integrating Beam-1.0's architectural features and decoding methods. It introduces fused CUDA kernels for attention and normalization, supports tensor, data, and expert parallelism, and implements DFlash2 speculative decoding with FP8 KV cache. The PR also includes comprehensive unit tests and config support for loading released Beam-1.0 configs.
Key Takeaway for Contributors
Supporting large-scale MoE models with specialized architectures requires tightly integrating model-specific layers, fused kernels, and decoding strategies while maintaining compatibility with existing parallelism frameworks and config schemas to enable scalable, efficient inference.