Day 21 Theory — Persistence, Checkpoints and Fault Recovery

Lesson Overview

This is the applied companion for Day 21. Read DAY_21_THEORY.md first for the beginner-first teaching of Persistence, Checkpoints and Fault Recovery. Then use the real service-desk-day-21/ project to trace, run, debug, and explain the concept.

Service Desk Alignment

Day 21 adds Persistence, Checkpoints and Fault Recovery to the running Service Desk. Start with checkpointer.py, graph.py, agent.py, workflow.py, debugger.py, serializer.py, then follow imports and tests to identify the actual runtime path.

Why This Topic Matters

The theory chapter explains why Persistence, Checkpoints and Fault Recovery is needed. Here the goal is evidence: identify where the capability is implemented, what observable behavior changes, and how the repository proves both success and failure behavior.

System Architecture

This is a repository surface map, not a claim that modules call each other in the displayed order. The modules are ranked by relevance to today's theory.

graph LR
    T[Day 21: Persistence, Checkpoints and Fault Recovery]
    T --> M1[checkpointer.py]
    T --> M2[graph.py]
    T --> M3[agent.py]
    T --> M4[workflow.py]
    T --> M5[debugger.py]
    T --> M6[serializer.py]
    T --> M7[models.py]

Follow imports and tests to discover the actual runtime flow.

Repository Implementation Map

Use the real Day 21 repository, not a fabricated sample, to connect theory to implementation.

Theory concepts to locate:

• What a checkpoint captures
• Checkpointing versus business persistence
• Thread/session identity
• Recovery after failure

Most relevant implementation modules first:

• service_desk/checkpointer.py
• service_desk/graph.py
• service_desk/agent.py
• service_desk/workflow.py
• service_desk/debugger.py
• service_desk/serializer.py
• service_desk/models.py

Follow imports/calls from the relevant module and confirm behavior in tests. Record input → mechanism → observable output/state → failure evidence.

Code Walkthrough & Mechanics

Read checkpointer.py, graph.py, agent.py, workflow.py, debugger.py, serializer.py, models.py with these theory sections beside you:

• What a checkpoint captures — locate its implementation and evidence.
• Checkpointing versus business persistence — locate its implementation and evidence.
• Thread/session identity — locate its implementation and evidence.
• Recovery after failure — locate its implementation and evidence.
• Exactly-once is difficult — locate its implementation and evidence.

For each concept identify the real function/class/protocol boundary, its input/state, its observable result, and the assertion in tests/test_checkpoints.py that proves the behavior. If a concept has no implementation or evidence, record that as a gap rather than inventing one.

Common Mistakes & Debugging Guidance

Use the theory—not generic timeout or .env advice—to decide what can fail today.

Failure lens: revisit Recovery after failure, Exactly-once is difficult.

• Reproduce the smallest case that violates one of those expectations.
• Trace the real Day 21 modules until you find the first incorrect state/output/decision.
• Use tests/test_checkpoints.py as executable evidence.
• Add a regression test if the failure is not already represented.
• Fix the smallest responsible boundary and rerun the relevant test before the full suite.

Your debugging explanation must name the topic-specific invariant that failed, not merely say “an exception occurred.”

Practical Lab Instructions

1. Summarize these theory ideas before opening the implementation:
   • What a checkpoint captures
   • Checkpointing versus business persistence
   • Thread/session identity
   • Recovery after failure
2. Inspect the most relevant real Day 21 modules first:
   • service_desk/checkpointer.py
   • service_desk/graph.py
   • service_desk/agent.py
   • service_desk/workflow.py
   • service_desk/debugger.py
   • service_desk/serializer.py
   • service_desk/models.py
3. Inspect the automated evidence:
   • tests/test_checkpoints.py
4. Establish the baseline:

   cd service-desk-day-21
   PYTHONPATH=. pytest tests/test_checkpoints.py -q
   

5. Trace one theory concept through the actual nested modules and tests.
6. Run one success case and record input → mechanism → observable result.
7. Exercise one topic-specific failure/boundary case and name the invariant that protects the system.
8. Re-run the relevant tests and explain theory → implementation → evidence.

Done when: another student can reproduce your trace without relying on an invented sample.

Key Takeaways

1. Be able to explain What a checkpoint captures and point to its implementation/evidence in Day 21.
2. Be able to explain Checkpointing versus business persistence and point to its implementation/evidence in Day 21.
3. Be able to explain Thread/session identity and point to its implementation/evidence in Day 21.

Knowledge Check & Scenario Questions

1. Concept: Using What a checkpoint captures, explain the engineering problem Day 21 is solving without naming a framework as the answer.
2. Mechanism: How does Checkpointing versus business persistence appear in the real project? Start from service_desk/checkpointer.py and name the observable state/output/event that changes.
3. Failure: For Thread/session identity, describe one incorrect implementation or boundary condition and the evidence you would expect in tests/test_checkpoints.py.
4. Design review: Which assumption in today's design would you verify before reusing this implementation in a different production system?

Official References

• LangGraph Checkpoints & SqliteSaver: https://langchain-ai.github.io/langgraph/reference/checkpoints/
• SQLite Transaction Control: https://www.sqlite.org/lang_transaction.html