Skip to main content
>_ supraj.dev

Module 6: Evaluation & Production Ops · 2h

01 · UNDERSTAND

Day 30 Theory — Production Streaming API: Backpressure, Disconnects and Progress Events

Why production streaming is harder than Day 08

Day 08 introduced streaming as a user-experience and protocol concept. A production agent stream may now carry model tokens, retrieval status, tool calls, approval pauses, warnings and terminal outcomes.

The system must remain correct even when the client is slow or disappears.

Define an event contract

A production stream should use explicit event types and stable payloads.

run.started
step.started
tool.started
tool.completed
message.delta
warning
run.failed
run.completed

The client should not need to infer workflow state from arbitrary text.

Backpressure

Backpressure happens when the producer creates data faster than the consumer can receive or process it.

If we ignore this, buffers can grow until memory is exhausted.

fast producer -> queue -> slow client
                    ↑ grows without bound if unmanaged

Use bounded queues, flow control, batching or producer throttling appropriate to the stack.

Disconnect handling

A disconnected client may mean the work is no longer valuable, but not always.

For a simple generated answer, cancellation may save cost. For a long-running ticket action, the workflow may need to continue and persist results even after the browser disconnects.

The product must define whether disconnect means:

  • cancel the run,
  • detach the client but continue,
  • continue only durable steps.

Progress events versus private reasoning

Expose useful operational progress, not hidden chain-of-thought.

Good:

“Searching knowledge base”

“Waiting for manager approval”

“Ticket updated”

Do not stream private model reasoning traces as a debugging feature.

Resumability

For long workflows, clients may reconnect. A robust design can use run IDs and persisted event/state so the UI can recover current status rather than assuming the original TCP connection lives forever.

Service Desk connection

Today the Service Desk's streaming layer becomes a durable product interface rather than a token animation.

The principle is:

Model streaming as an event protocol with bounded buffering, explicit disconnect semantics and safe progress—not as an endless text socket.

02 · APPLY

Lesson Overview

This is the applied companion for Day 30. Read DAY_30_THEORY.md first for the beginner-first teaching of Production Streaming API: Backpressure, Disconnects and Progress Events. Then use the real service-desk-day-30/ project to trace, run, debug, and explain the concept.

Service Desk Alignment

Day 30 adds Production Streaming API: Backpressure, Disconnects and Progress Events to the running Service Desk. Start with the day project, then follow imports and tests to identify the actual runtime path.

Why This Topic Matters

The theory chapter explains why Production Streaming API: Backpressure, Disconnects and Progress Events is needed. Here the goal is evidence: identify where the capability is implemented, what observable behavior changes, and how the repository proves both success and failure behavior.

System Architecture

The implementation boundary for Production Streaming API: Backpressure, Disconnects and Progress Events is the service-desk-day-30/ project. Use its README and tests as the executable architecture map.

Repository Implementation Map

Use the real Day 30 repository, not a fabricated sample, to connect theory to implementation.

Theory concepts to locate:

  • Why production streaming is harder than Day 08
  • Define an event contract
  • Backpressure
  • Disconnect handling

Most relevant implementation modules first:

  • Follow the project README.

Follow imports/calls from the relevant module and confirm behavior in tests. Record input → mechanism → observable output/state → failure evidence.

Code Walkthrough & Mechanics

Read the project README with these theory sections beside you:

  • Why production streaming is harder than Day 08 — locate its implementation and evidence.
  • Define an event contract — locate its implementation and evidence.
  • Backpressure — locate its implementation and evidence.
  • Disconnect handling — locate its implementation and evidence.
  • Progress events versus private reasoning — locate its implementation and evidence.

For each concept identify the real function/class/protocol boundary, its input/state, its observable result, and the assertion in tests/test_production_streaming.py that proves the behavior. If a concept has no implementation or evidence, record that as a gap rather than inventing one.

Common Mistakes & Debugging Guidance

Use the theory—not generic timeout or .env advice—to decide what can fail today.

Failure lens: revisit Disconnect handling, Progress events versus private reasoning.

  • Reproduce the smallest case that violates one of those expectations.
  • Trace the real Day 30 modules until you find the first incorrect state/output/decision.
  • Use tests/test_production_streaming.py as executable evidence.
  • Add a regression test if the failure is not already represented.
  • Fix the smallest responsible boundary and rerun the relevant test before the full suite.

Your debugging explanation must name the topic-specific invariant that failed, not merely say “an exception occurred.”

Practical Lab Instructions

  1. Summarize these theory ideas before opening the implementation:
    • Why production streaming is harder than Day 08
    • Define an event contract
    • Backpressure
    • Disconnect handling
  2. Inspect the most relevant real Day 30 modules first:
    • Follow README.md.
  3. Inspect the automated evidence:
    • tests/test_production_streaming.py
  4. Establish the baseline:
    cd service-desk-day-30
PYTHONPATH=. pytest tests/test_production_streaming.py -q
  5. Trace one theory concept through the actual nested modules and tests.
  6. Run one success case and record input → mechanism → observable result.
  7. Exercise one topic-specific failure/boundary case and name the invariant that protects the system.
  8. Re-run the relevant tests and explain theory → implementation → evidence.

Done when: another student can reproduce your trace without relying on an invented sample.

Key Takeaways

  1. Be able to explain Why production streaming is harder than Day 08 and point to its implementation/evidence in Day 30.
  2. Be able to explain Define an event contract and point to its implementation/evidence in Day 30.
  3. Be able to explain Backpressure and point to its implementation/evidence in Day 30.

Knowledge Check & Scenario Questions

  1. Concept: Using Why production streaming is harder than Day 08, explain the engineering problem Day 30 is solving without naming a framework as the answer.
  2. Mechanism: How does Define an event contract appear in the real project? Start from the project entry point and name the observable state/output/event that changes.
  3. Failure: For Backpressure, describe one incorrect implementation or boundary condition and the evidence you would expect in tests/test_production_streaming.py.
  4. Design review: Which assumption in today's design would you verify before reusing this implementation in a different production system?

Official References

03 · EXPLAIN

Interview checkpoint

Explain one design decision from this lesson, the alternative you rejected, and the failure mode or evidence that justified your choice.