unlock unlimited code runs for $79, limited time →
NOSSLETTER.techLIVE
back to archive

(2.15) [IMPROVED] MQTT: Pipeline QoS2 PUBLISH and PUBREL processing

Language: Go+818 -201performance
The Issue & Context

Previously, MQTT QoS2 message handling on the server was stop-and-wait, blocking the publisher's read loop during JetStream store operations for each PUBLISH and PUBREL packet. This limited QoS2 throughput to a few hundred messages per second regardless of the client's in-flight window size.

Root Cause Analysis

The synchronous JetStream operations—loading, deleting, and storing message copies—caused blocking and multiple round trips per message, severely capping throughput and increasing latency for QoS2 publishers.

Solution Strategy & Architectural Pattern

The PR introduced asynchronous JetStream operations for QoS2, extending the existing QoS1 ack pipelining to QoS2. It pipelines PUBLISH and PUBREL processing, sends PUBREC and PUBCOMP acknowledgments as they arrive in order, and keeps necessary state in memory to avoid blocking reads, drastically increasing throughput and reducing latency.

Key Takeaway for Contributors

Asynchronous pipelining of message processing and acknowledgments can dramatically improve MQTT QoS2 throughput by eliminating blocking I/O and reducing round trips, especially when combined with in-memory state management to maintain protocol correctness.