A NULL payload column on an outbox row means "this aggregate was
deleted" -- Debezium's outbox convention routes that as a TOMBSTONE (a
record with the aggregate's key but value=None), which tells Kafka's
log compaction it can eventually drop every earlier record under that
key. Right now, deleting an aggregate silently produces a NORMAL event
with a None value instead of a properly flagged tombstone -- downstream
consumers that specifically check for tombstones (to react to a deletion,
say) never see it as one.
Your task: find the bug in route()'s tombstone check and fix it.
The topic derivation and the key (aggregateid, not the outbox row's own
id -- getting that part right is what keeps every event for one
aggregate on the same partition, in order) are already correct -- don't
change them.
Example that currently fails
router = OutboxEventRouter()
row = {"id": "evt-3", "aggregatetype": "Order", "aggregateid": "order-42",
"type": "OrderDeleted", "payload": None}
_, _, value, is_tombstone = router.route(row)
is_tombstone # should be True
# currently: False (a None payload isn't caught at all --
# it slips through the same path as a normal event)
Real-world grounding
- Debezium's outbox event router SMT:
debezium-connect-plugins/src/main/java/io/debezium/transforms/outbox/EventRouter.java
Real Debezium's outbox router additionally supports promoting extra outbox columns into Kafka message headers and configuring the topic naming pattern -- but "NULL payload becomes a same-key tombstone" is the specific mechanism this bug breaks.