Day 23 Theory — Orchestration Patterns: Router, Planner, Evaluator and Agents-as-Tools

Lesson Overview

This is the applied companion for Day 23. Read DAY_23_THEORY.md first for the beginner-first teaching of Orchestration Patterns: Router, Planner, Evaluator and Agents-as-Tools. Then use the real service-desk-day-23/ project to trace, run, debug, and explain the concept.

Service Desk Alignment

Day 23 adds Orchestration Patterns: Router, Planner, Evaluator and Agents-as-Tools to the running Service Desk. Start with router.py, planner.py, evaluator_optimizer.py, benchmark.py, orchestrator.py, specialist_agents.py, then follow imports and tests to identify the actual runtime path.

Why This Topic Matters

The theory chapter explains why Orchestration Patterns: Router, Planner, Evaluator and Agents-as-Tools is needed. Here the goal is evidence: identify where the capability is implemented, what observable behavior changes, and how the repository proves both success and failure behavior.

System Architecture

This is a repository surface map, not a claim that modules call each other in the displayed order. The modules are ranked by relevance to today's theory.

graph LR
    T[Day 23: Orchestration Patterns - Router, Planner, Evaluator and Agents-as-Tools]
    T --> M1[router.py]
    T --> M2[planner.py]
    T --> M3[evaluator_optimizer.py]
    T --> M4[benchmark.py]
    T --> M5[orchestrator.py]
    T --> M6[specialist_agents.py]
    T --> M7[models.py]

Follow imports and tests to discover the actual runtime flow.

Repository Implementation Map

Use the real Day 23 repository, not a fabricated sample, to connect theory to implementation.

Theory concepts to locate:

• Not every workflow needs the same orchestration pattern
• Router
• Planner
• Evaluator / critic

Most relevant implementation modules first:

• service_desk/router.py
• service_desk/planner.py
• service_desk/evaluator_optimizer.py
• service_desk/benchmark.py
• service_desk/orchestrator.py
• service_desk/specialist_agents.py
• service_desk/models.py

Follow imports/calls from the relevant module and confirm behavior in tests. Record input → mechanism → observable output/state → failure evidence.

Code Walkthrough & Mechanics

Read router.py, planner.py, evaluator_optimizer.py, benchmark.py, orchestrator.py, specialist_agents.py, models.py with these theory sections beside you:

• Not every workflow needs the same orchestration pattern — locate its implementation and evidence.
• Router — locate its implementation and evidence.
• Planner — locate its implementation and evidence.
• Evaluator / critic — locate its implementation and evidence.
• Agents as tools — locate its implementation and evidence.

For each concept identify the real function/class/protocol boundary, its input/state, its observable result, and the assertion in tests/test_orchestration_patterns.py that proves the behavior. If a concept has no implementation or evidence, record that as a gap rather than inventing one.

Common Mistakes & Debugging Guidance

Use the theory—not generic timeout or .env advice—to decide what can fail today.

Failure lens: revisit Evaluator / critic, Agents as tools.

• Reproduce the smallest case that violates one of those expectations.
• Trace the real Day 23 modules until you find the first incorrect state/output/decision.
• Use tests/test_orchestration_patterns.py as executable evidence.
• Add a regression test if the failure is not already represented.
• Fix the smallest responsible boundary and rerun the relevant test before the full suite.

Your debugging explanation must name the topic-specific invariant that failed, not merely say “an exception occurred.”

Practical Lab Instructions

1. Summarize these theory ideas before opening the implementation:
   • Not every workflow needs the same orchestration pattern
   • Router
   • Planner
   • Evaluator / critic
2. Inspect the most relevant real Day 23 modules first:
   • service_desk/router.py
   • service_desk/planner.py
   • service_desk/evaluator_optimizer.py
   • service_desk/benchmark.py
   • service_desk/orchestrator.py
   • service_desk/specialist_agents.py
   • service_desk/models.py
3. Inspect the automated evidence:
   • tests/test_orchestration_patterns.py
4. Establish the baseline:

   cd service-desk-day-23
   PYTHONPATH=. pytest tests/test_orchestration_patterns.py -q
   

5. Trace one theory concept through the actual nested modules and tests.
6. Run one success case and record input → mechanism → observable result.
7. Exercise one topic-specific failure/boundary case and name the invariant that protects the system.
8. Re-run the relevant tests and explain theory → implementation → evidence.

Done when: another student can reproduce your trace without relying on an invented sample.

Key Takeaways

1. Be able to explain Not every workflow needs the same orchestration pattern and point to its implementation/evidence in Day 23.
2. Be able to explain Router and point to its implementation/evidence in Day 23.
3. Be able to explain Planner and point to its implementation/evidence in Day 23.

Knowledge Check & Scenario Questions

1. Concept: Using Not every workflow needs the same orchestration pattern, explain the engineering problem Day 23 is solving without naming a framework as the answer.
2. Mechanism: How does Router appear in the real project? Start from service_desk/router.py and name the observable state/output/event that changes.
3. Failure: For Planner, describe one incorrect implementation or boundary condition and the evidence you would expect in tests/test_orchestration_patterns.py.
4. Design review: Which assumption in today's design would you verify before reusing this implementation in a different production system?

Official References

• Building Effective Agents (Anthropic Research): https://www.anthropic.com/research/building-effective-agents
• LangGraph Multi-Agent Architecture: https://langchain-ai.github.io/langgraph/concepts/multi_agent/