unlock unlimited code runs for $79, limited time →
NOSSLETTER.techLIVE
back to archive

add GLM-5.3-Flash (GLM5-Next) support

Language: C+++2377 -39feature
concepts demonstrated
Hybrid neural network architecturesMixture of Experts (MoE) patternLayer reuse and extension in C++Token indexing and caching mechanisms
The Issue & Context

The GLM-5.3-Flash model is a large 320B parameter hybrid architecture combining text and vision processing with complex layer types like KDA, DSA, mHC, and Deepseek-style mixture of experts. Prior to this PR, llama.cpp did not support this model variant, limiting users to older GLM versions or other architectures.

Root Cause Analysis

GLM-5.3-Flash introduced new layer types and indexing mechanisms, such as pooling over token groups in DSA and vision tower components with distinct preprocessing, which were not implemented or integrated in llama.cpp. Additionally, precision-sensitive tensors required careful handling to maintain accuracy.

Solution Strategy & Architectural Pattern

This PR integrated GLM-5.3-Flash by reusing existing implementations for KDA and mHC layers, adapting DSA indexer to pool over token groups without new backend ops, adding a new vision tower variant (glm5v) with appropriate preprocessing, and preserving small precision-sensitive tensors unquantized. It also refactored shared helpers for mHC to be reusable across models and validated outputs against transformer baselines.

Key Takeaway for Contributors

Extending llama.cpp to support complex hybrid models like GLM-5.3-Flash is feasible by modular reuse of existing components and careful handling of new indexing and vision features, enabling users to run state-of-the-art large multimodal models efficiently in this framework.