model: add Beam (Reflection)
The Issue & Context
Before this PR, the llama.cpp codebase lacked support for the Reflection Beam-1.0-501B-A23B model, which features a complex mixture-of-experts architecture, interleaved attention layers, and a new chat parser for tool-call formats; attempts to run or convert this model were not possible.
Root Cause Analysis
The existing codebase did not implement the specialized MoE routing with top-8 expert selection and sigmoid routing, nor the interleaved sliding-window and NoPE attention layers with power-law query scaling, and lacked a parser for the Beam-specific tool-call chat format; these architectural and format specifics required new components and extensions beyond existing llama.cpp models.
Solution Strategy & Architectural Pattern
This PR introduces a new model implementation in src/models/beam.cpp that handles the MoE FFN with grouped selection and routing biases, implements the interleaved attention layers with selective RoPE application, and adds power-law query scaling as an optional feature in llama-graph.cpp. It also adds a converter script (conversion/beam.py) for Beam weights, a new harmony-xml chat parser in common/parsers/harmony-xml.cpp to detect and parse Beam tool-call markers, and integrates these into the codebase with tests and GGUF format support to ensure correct loading and generation.
Key Takeaway for Contributors
Supporting large, specialized models with complex routing and attention patterns requires extending both model graph logic and input/output parsing, emphasizing the importance of modular architecture and format-aware parsers to handle unique tokenization and interaction protocols.