3. Query Optimization

Fix the Bug: Broadcast vs. Shuffle Join Strategy Selection

midbig-data-processingquery-optimizationdebugging

A join where the RIGHT side is small enough to broadcast (and the left side is much larger) is falling back to an expensive shuffle join instead of broadcasting the right side. A join where the LEFT side is the small one works correctly -- it's specifically the "small right, large left" case that's broken.

Your task: find the bug in choose_join_strategy() and fix it. The overall decision structure (check both sides' eligibility, prefer the smaller eligible side, fall back to shuffle) is already correct -- don't restructure it.

Example that currently fails

choose_join_strategy(
    left_size_bytes=500 * 1024 * 1024,   # 500MB -- too big to broadcast
    right_size_bytes=1 * 1024 * 1024,    # 1MB -- easily broadcastable
    broadcast_threshold_bytes=10 * 1024 * 1024,
)
# should be "BROADCAST_RIGHT" (the right side is well under the
# threshold) -- currently returns "SHUFFLE" instead.

Real-world grounding

Real Spark's decision also accounts for the join type (an outer join can only broadcast the non-preserved side) and can be overridden with explicit broadcast hints -- this exercise implements the core size-threshold check both build on.

hints
join_strategy.pysign in to run
Loading...
test result

This is a preview. Sign in to run this exercise against the real tests.