unlock unlimited code runs for $79, limited time →
NOSSLETTER.techLIVE
back to archive

model: add Beam (Reflection)

Language: C+++732 -11feature
concepts demonstrated
Mixture of Experts (MoE) architectureFeedforward Neural Networks with SwiGLU activationRotary Positional Embeddings (RoPE)Custom chat parsing with XML-like tool-call markers
The Issue & Context

Before this PR, the llama.cpp codebase lacked support for the Reflection Beam-1.0-501B-A23B model, which features a complex mixture-of-experts architecture, interleaved attention layers, and a new chat parser for tool-call formats; attempts to run or convert this model were not possible.

Root Cause Analysis

The existing codebase did not implement the specialized MoE routing with top-8 expert selection and sigmoid routing, nor the interleaved sliding-window and NoPE attention layers with power-law query scaling, and lacked a parser for the Beam-specific tool-call chat format; these architectural and format specifics required new components and extensions beyond existing llama.cpp models.

Solution Strategy & Architectural Pattern

This PR introduces a new model implementation in src/models/beam.cpp that handles the MoE FFN with grouped selection and routing biases, implements the interleaved attention layers with selective RoPE application, and adds power-law query scaling as an optional feature in llama-graph.cpp. It also adds a converter script (conversion/beam.py) for Beam weights, a new harmony-xml chat parser in common/parsers/harmony-xml.cpp to detect and parse Beam tool-call markers, and integrates these into the codebase with tests and GGUF format support to ensure correct loading and generation.

Key Takeaway for Contributors

Supporting large, specialized models with complex routing and attention patterns requires extending both model graph logic and input/output parsing, emphasizing the importance of modular architecture and format-aware parsers to handle unique tokenization and interaction protocols.