Retrying a failing call immediately just hammers a struggling dependency
harder. Plain exponential backoff (wait base * 2^attempt, capped) fixes
that, but if every client backs off on the exact same schedule, they all
retry in lockstep and cause a synchronized burst anyway -- a "thundering
herd". The fix, used widely in production systems, is "Full Jitter": pick a
RANDOM delay between 0 and the capped exponential value, so retries spread
out instead of syncing up.
rand is injected as a zero-argument callable returning a float in [0, 1),
and sleep as a one-argument callable, so tests can supply fixed/fake values
for exact, reproducible results (real code lets them default to
random.random/time.sleep, which the starter already wires up).
Implement:
RetryPolicy.next_delay(attempt): ifattempt >= max_retries, returnNone(give up). Otherwise computeexp_backoff = min(cap, base * 2**attempt), then returnrand() * exp_backoff.RetryPolicy.execute(fn): callfn(). If it raises, usenext_delay()to decide the retry delay -- sleep (viaself._sleep) and callfn()again. Keep retrying until eitherfn()succeeds (return its value) ornext_delayruns out of budget, at which point the original exception from the most recent failed call must propagate to the caller -- retrying is invisible on success, and never invisible on final failure.
Example
policy = RetryPolicy(base=1.0, cap=100.0, max_retries=5, rand=lambda: 1.0)
policy.next_delay(0) # -> 1.0 (base * 2**0 = 1.0, capped at 100, rand=1.0)
policy.next_delay(1) # -> 2.0 (base * 2**1 = 2.0)
policy.next_delay(2) # -> 4.0 (base * 2**2 = 4.0)
calls = []
def flaky():
calls.append(None)
if len(calls) < 3:
raise ValueError("not yet")
return "ok"
policy.execute(flaky) # -> "ok", after 2 retries (3 total calls)
Constraints
min(cap, ...)must be applied BEFORE multiplying by the jitter factor -- jitter scales the already-capped delay, it isn't itself capped separately.executemust not swallow or replace the exception type -- iffnfinally fails for good, the caller sees exactly whatfnraised, not a generic "gave up" error.executemust callself._sleep, not a real clock -- tests inject a fake that just records delays, so a realtime.sleepcall would make the test suite itself slow (and Judge0 runs under a tight CPU budget).
Real-world grounding
- gRPC's retry policy config (
initialBackoff,maxBackoff,backoffMultiplier-- the same shape asbase/cap/the2**attemptgrowth here): documented in gRPC's service config spec, implemented acrossgrpc/grpc's per-language retry implementations.
"Full Jitter" specifically comes from an AWS Architecture Blog post on backoff strategies that's become a de facto standard -- most production retry libraries (including gRPC's) implement some variant of it.