10. Runtime I/O

Stream Backpressure

midconcurrencydata-structures

Node's writable streams (files, sockets, HTTP responses) don't let a fast producer overwhelm a slow destination -- every writable stream tracks how much data is sitting in its internal buffer, and once that crosses a configured highWaterMark, write() starts returning false as a signal to the producer: "you can keep calling me, but you should really pause." Once the destination catches up and the buffer drops back under the threshold, the stream emits a 'drain' event telling the producer it's safe to resume.

This exercise models that mechanism with no real I/O at all -- write() and consume() are explicit calls standing in for "more data arrived" and "the destination finished flushing the oldest queued chunk."

Implement:

  • WritableStream.write(chunk_size) -- append chunk_size to the internal queue and add it to self.buffered. Return False once self.buffered has reached self.high_water_mark (and remember, via self._needs_drain, that a drain is now owed); otherwise return True.
  • WritableStream.consume() -- pop the OLDEST queued chunk (FIFO -- data finishes flushing in the order it was written) and subtract its size from self.buffered. If a drain was owed (self._needs_drain) and self.buffered has now dropped below self.high_water_mark, clear _needs_drain, append the new self.buffered value to self.drain_events, and return True. Otherwise return False.

Example

stream = WritableStream(high_water_mark=100)
stream.write(60)   # -> True   (buffered=60, still under 100)
stream.write(50)   # -> False  (buffered=110, at/over 100 -- pause!)
stream.consume()   # buffered=50, dropped below 100 -- 'drain' fires
stream.drain_events  # -> [50]

Constraints

  • consume() only has something to do when the queue is non-empty -- calling it on an empty stream should just return False.
  • A drain event fires at most once per "crossing" -- consuming further chunks while already under high_water_mark must NOT append additional entries to drain_events until another write() pushes the buffer back over the threshold.

Real-world grounding

The two tests at the end of this exercise are the actual point: a producer that ignores write()'s False return and keeps writing anyway drives the buffer to unbounded size (nothing is holding it back), while a producer that pauses on False and waits for a 'drain'-equivalent signal before resuming keeps the buffer bounded near high_water_mark no matter how much data is waiting to go out. That's the entire reason backpressure exists.

hints
writable_stream.pysign in to run
Loading...
test result

This is a preview. Sign in to run this exercise against the real tests.