Module 1: Engineering Foundations · 4.25h

01 · UNDERSTAND

Day 02 Theory — Async Python, Event Loops and Concurrent I/O

Why we are learning this today

Yesterday the Service Desk application was deliberately simple: receive data, validate it, and return a result. Real AI applications quickly become dominated by waiting. They wait for an LLM provider, a vector database, an HTTP API, a ticketing system, a secrets service, or a human approval. If we handle every wait one after another, a program can feel slow even when the CPU is mostly idle.

Async Python gives us a way to use that waiting time productively.

The goal is not to make every function async. The goal is to understand when work is CPU-bound, when it is I/O-bound, and how an event loop coordinates many waiting operations safely.

Foundation refresher: synchronous execution

In ordinary synchronous Python, one function runs until it returns before the next line can continue. That is easy to reason about and is often exactly what we want.

profile = fetch_user_profile()
tickets = fetch_open_tickets()

If each network call takes one second, the total wait is roughly two seconds because the second call does not begin until the first one finishes.

The CPU is not necessarily busy during those two seconds. Most of the time it is waiting for the operating system to receive bytes from the network.

The important distinction: concurrency is not parallelism

Concurrency means several tasks can make progress over the same period of time. One task can pause while another runs.

Parallelism means multiple pieces of work are literally executing at the same instant, usually on multiple CPU cores or processors.

asyncio is mainly a concurrency tool. It is especially useful for I/O-heavy workloads.

What is a coroutine?

Calling a normal function starts executing it immediately. Calling an async def function creates a coroutine object. The coroutine only makes progress when the event loop runs it.

async def fetch_ticket(ticket_id: str):
    ...

The keyword await means:

“This operation cannot make useful progress right now. Give control back to the event loop, and resume me when the awaited operation is ready.”

That is the heart of cooperative asynchronous programming.

What the event loop actually does

A useful mental model is a receptionist coordinating several support calls.

Start Task A.
Task A reaches a network wait.
Instead of blocking the whole program, the event loop runs Task B.
Task B may also reach a wait.
When Task A's network response arrives, the event loop schedules Task A to continue.

Task A ── work ── wait for API ───────── resume ── done
Task B ────────── work ── wait ── resume ───────── done
Task C ───────────────── work ─────────────── done
          one event loop coordinating progress

There is no magic background thread implied by await. The tasks cooperate by yielding control at await points.

Why `await` does not automatically make code non-blocking

This is a common beginner mistake.

async def bad_example():
    time.sleep(5)

Even though the function is declared async, time.sleep() blocks the thread. The event loop cannot run other tasks during that sleep.

For asynchronous code, use an async-aware operation such as:

await asyncio.sleep(5)

The same rule applies to libraries. An async application should use clients designed for asynchronous I/O, such as httpx.AsyncClient, when concurrency is required.

Tasks and `asyncio.gather`

When two operations are independent, we can schedule them together.

profile, tickets = await asyncio.gather(
    fetch_user_profile(user_id),
    fetch_open_tickets(user_id),
)

This does not mean “run everything concurrently because concurrent is always faster.” Dependencies still matter. If operation B needs the output of A, then B cannot safely start first.

Think in terms of a dependency graph:

Independent:       Dependent:
A ──┐              A ──> B
    ├─> continue
B ──┘

Only the left side is naturally concurrent.

Cancellation is part of correctness

A production request can disappear because a browser disconnects, a timeout expires, a deployment shuts down, or a parent task is cancelled. Async code must be prepared for cancellation.

Cancellation is not just an error to hide. It is a control signal telling the program that the result is no longer needed.

Cleanup still matters: close files, release database connections, and avoid leaving half-finished side effects.

Timeouts define a boundary, not a universal number

A timeout answers: how long is this caller willing to wait?

The correct value depends on the operation, the user experience, and the upstream service contract. There is no universal “best timeout.”

Use timeouts deliberately and handle the timeout path explicitly.

When async is the wrong tool

Async does not make CPU-heavy work faster. If a function spends several seconds compressing data, calculating embeddings locally on CPU, or performing image processing, the event loop can still be blocked.

For CPU-bound work, consider process-based parallelism, worker queues, native libraries, or an external compute service.

Service Desk connection

Today our Service Desk begins behaving like a real networked application. A single ticket may eventually require several independent reads: user identity, ticket history, knowledge-base context, policy data, or provider metadata.

The design principle is:

Keep sequential work sequential. Make independent I/O concurrent only when doing so preserves correctness.

By the end of the day, you should be able to look at a function and explain exactly where it can pause, what else can run while it waits, and what happens if the operation is cancelled or times out.

02 · APPLY

Lesson Overview

This is the applied companion for Day 02. Read DAY_02_THEORY.md first for the beginner-first teaching of Async Python, Event Loops and Concurrent I/O. Then use the real service-desk-day-02/ project to trace, run, debug, and explain the concept.

Service Desk Alignment

Day 02 adds Async Python, Event Loops and Concurrent I/O to the running Service Desk. Start with services.py, models.py, then follow imports and tests to identify the actual runtime path.

Why This Topic Matters

The theory chapter explains why Async Python, Event Loops and Concurrent I/O is needed. Here the goal is evidence: identify where the capability is implemented, what observable behavior changes, and how the repository proves both success and failure behavior.

System Architecture

graph TD
    A[Event Loop] -->|Schedule| B[Coroutine 1: Fetch API]
    A -->|Schedule| C[Coroutine 2: Fetch DB]
    B -->|Yield on await| A
    C -->|Yield on await| A
    A -->|Resume| D[asyncio.gather Results]

Worked Code Example

import asyncio
import httpx
import time

async def fetch_endpoint(endpoint_id: int) -> dict:
    """Simulates async non-blocking HTTP fetch."""
    await asyncio.sleep(0.05)  # Yield control to event loop
    return {"endpoint_id": endpoint_id, "status": "online"}

async def run_concurrent_fetches():
    start = time.perf_counter()
    # Schedule 5 coroutines concurrently onto the single-threaded event loop
    tasks = [fetch_endpoint(i) for i in range(5)]
    results = await asyncio.gather(*tasks)
    elapsed = time.perf_counter() - start
    print(f"Fetched {len(results)} endpoints concurrently in {elapsed:.3f}s")
    return results

if __name__ == "__main__":
    asyncio.run(run_concurrent_fetches())

Detailed Code Explanation

Read services.py, models.py with these theory sections beside you:

Why we are learning this today — locate its implementation and evidence.
Foundation refresher: synchronous execution — locate its implementation and evidence.
The important distinction: concurrency is not parallelism — locate its implementation and evidence.
What is a coroutine? — locate its implementation and evidence.
What the event loop actually does — locate its implementation and evidence.

For each concept identify the real function/class/protocol boundary, its input/state, its observable result, and the assertion in tests/test_async_services.py that proves the behavior. If a concept has no implementation or evidence, record that as a gap rather than inventing one.

Common Mistakes & Debugging Guidance

Mistake 1: Mixing blocking requests.get() inside async def functions, freezing the single-threaded event loop.
Mistake 2: Calling asyncio.run() inside an already running event loop (e.g. inside FastAPI or Jupyter).
Mistake 3: Forgetting to await a coroutine function, resulting in unexecuted <coroutine object> references.

Practical Lab Instructions

Summarize these theory ideas before opening the implementation:
- Why we are learning this today
- Foundation refresher: synchronous execution
- The important distinction: concurrency is not parallelism
- What is a coroutine?
Inspect the most relevant real Day 02 modules first:
- service_desk/services.py
- service_desk/models.py
Inspect the automated evidence:
- tests/test_async_services.py

Establish the baseline:

cd service-desk-day-02
PYTHONPATH=. pytest tests/test_async_services.py -q

Trace one theory concept through the actual nested modules and tests.
Run one success case and record input → mechanism → observable result.
Exercise one topic-specific failure/boundary case and name the invariant that protects the system.
Re-run the relevant tests and explain theory → implementation → evidence.

Done when: another student can reproduce your trace without relying on an invented sample.

Key Takeaways

Async Python achieves concurrency on a single thread by yielding CPU execution during I/O wait states.
Use asyncio.gather() for concurrent I/O fan-out. Timeout boundaries such as asyncio.wait_for() can bound how long the caller waits; cancellation and cleanup behavior still need explicit design.
Never execute blocking synchronous CPU or disk/network calls inside async coroutine handlers.

Knowledge Check & Scenario Questions

Knowledge Check: What occurs when a blocking time.sleep(5) is executed inside an async def route?
- Answer: It blocks the single-threaded event loop, freezing all concurrent incoming requests for 5 seconds.
Scenario Question: How does asyncio.gather() handle an exception raised by one of its child tasks by default?
- Answer: By default, the first raised exception is propagated to the caller. Other submitted awaitables are not automatically cancelled solely because that exception was propagated, so their lifecycle still needs to be understood and managed.

Official References

Python asyncio Official Documentation: https://docs.python.org/3/library/asyncio.html
HTTPX Async Client Guide: https://www.python-httpx.org/async/

03 · EXPLAIN

Interview checkpoint

Explain one design decision from this lesson, the alternative you rejected, and the failure mode or evidence that justified your choice.