4. Resilience

Retry with Exponential Backoff + Jitter

midnetworkingretry-backoff

Retrying a failing call immediately just hammers a struggling dependency harder. Plain exponential backoff (wait base * 2^attempt, capped) fixes that, but if every client backs off on the exact same schedule, they all retry in lockstep and cause a synchronized burst anyway -- a "thundering herd". The fix, used widely in production systems, is "Full Jitter": pick a RANDOM delay between 0 and the capped exponential value, so retries spread out instead of syncing up.

rand is injected as a zero-argument callable returning a float in [0, 1), and sleep as a one-argument callable, so tests can supply fixed/fake values for exact, reproducible results (real code lets them default to random.random/time.sleep, which the starter already wires up).

Implement:

  • RetryPolicy.next_delay(attempt): if attempt >= max_retries, return None (give up). Otherwise compute exp_backoff = min(cap, base * 2**attempt), then return rand() * exp_backoff.
  • RetryPolicy.execute(fn): call fn(). If it raises, use next_delay() to decide the retry delay -- sleep (via self._sleep) and call fn() again. Keep retrying until either fn() succeeds (return its value) or next_delay runs out of budget, at which point the original exception from the most recent failed call must propagate to the caller -- retrying is invisible on success, and never invisible on final failure.

Example

policy = RetryPolicy(base=1.0, cap=100.0, max_retries=5, rand=lambda: 1.0)
policy.next_delay(0)  # -> 1.0   (base * 2**0 = 1.0, capped at 100, rand=1.0)
policy.next_delay(1)  # -> 2.0   (base * 2**1 = 2.0)
policy.next_delay(2)  # -> 4.0   (base * 2**2 = 4.0)

calls = []
def flaky():
    calls.append(None)
    if len(calls) < 3:
        raise ValueError("not yet")
    return "ok"

policy.execute(flaky)  # -> "ok", after 2 retries (3 total calls)

Constraints

  • min(cap, ...) must be applied BEFORE multiplying by the jitter factor -- jitter scales the already-capped delay, it isn't itself capped separately.
  • execute must not swallow or replace the exception type -- if fn finally fails for good, the caller sees exactly what fn raised, not a generic "gave up" error.
  • execute must call self._sleep, not a real clock -- tests inject a fake that just records delays, so a real time.sleep call would make the test suite itself slow (and Judge0 runs under a tight CPU budget).

Real-world grounding

  • gRPC's retry policy config (initialBackoff, maxBackoff, backoffMultiplier -- the same shape as base/cap/the 2**attempt growth here): documented in gRPC's service config spec, implemented across grpc/grpc's per-language retry implementations.

"Full Jitter" specifically comes from an AWS Architecture Blog post on backoff strategies that's become a de facto standard -- most production retry libraries (including gRPC's) implement some variant of it.

hints
retry_policy.pysign in to run
Loading...
test result

This is a preview. Sign in to run this exercise against the real tests.