Day 01 — Student Lesson

AI Agent Engineering Starts with Software Engineering

Today you will build the first version of the AI Service Desk Agent. It is intentionally simple. There is no advanced agent framework and no complex RAG system yet.

That is deliberate.

Before we let a language model make decisions, call tools, retrieve documents, or take actions, we need a reliable software foundation around it.

---

1. What is Generative AI?

Generative AI refers to AI systems that can create new outputs such as text, code, images, audio, or structured data from the input and context they receive.

A useful beginner distinction is:

• traditional software primarily follows rules written directly by developers,
• generative models learn statistical patterns from training data and generate outputs from those learned patterns plus the context they receive at inference time.

This does not mean traditional software disappears. Production AI applications still require ordinary software engineering.

---

2. What is a Large Language Model?

A Large Language Model, or LLM, is a neural-network model trained on large amounts of language-like data.

At a practical level, you can think of an LLM as a system that:

1. receives text represented as tokens,
2. processes the current context,
3. predicts probabilities for possible next tokens,
4. selects a next token according to its decoding configuration,
5. repeats the process until the response is complete or a stopping condition is reached.

You do not need to understand transformer mathematics on Day 1. For now, remember this:

An LLM generates likely continuations. It is not automatically verifying every sentence against a trusted database.

---

3. Training vs Inference

Training

Training is the expensive process in which model parameters are adjusted using large datasets and optimization algorithms.

Training changes the model.

Inference

Inference happens when an already-trained model receives a new request and produces an output.

Inference normally does not retrain the model for that single request.

A simple analogy:

• Training = studying for an exam.
• Inference = answering a question during the exam using what you learned plus the question in front of you.

The analogy is not mathematically exact, but it is useful for building intuition.

---

4. What is a Token?

Models do not necessarily process text one full English word at a time.

A tokenizer converts text into smaller units called tokens. Depending on the tokenizer, a token may represent:

• a complete word,
• part of a word,
• punctuation,
• whitespace patterns,
• or another text fragment.

Tokens matter because model context limits and usage are typically measured in tokens rather than ordinary word counts.

We will study context budgets much more deeply later in the course.

---

5. Why Can LLMs Hallucinate?

An LLM can produce an answer that looks confident and fluent but is unsupported or incorrect.

This is commonly called a hallucination.

Example:

A model may invent:

• a product policy,
• a URL,
• an API method,
• a legal clause,
• an employee name,
• or a technical fact.

Why?

Because the generation process is optimized for producing likely continuations, not for automatically proving that every claim is true.

This leads to one of the most important rules of production AI engineering:

Fluent output is not evidence of correctness.

---

6. LLM vs AI Application vs AI Agent

These terms are related but not identical.

LLM

The model itself.

Input context → Model → Generated output

AI Application

Software that uses an LLM together with normal application components.

User
  ↓
API / Application
  ├─ validation
  ├─ authentication / authorization
  ├─ business rules
  ├─ LLM
  ├─ databases / tools
  └─ logging / evaluation

AI Agent

An application where a model participates in a controlled decision-action-observation loop.

Goal
 ↓
Decide
 ↓
Request action/tool
 ↓
Application validates and executes
 ↓
Observe result
 ↓
Decide what to do next

An AI agent is therefore more than “a chatbot with a long prompt.”

---

7. Deterministic Software vs Probabilistic Model Behavior

A deterministic rule follows explicit application logic.

if ticket.priority == Priority.CRITICAL:
    return RouteTarget.SECURITY

If the relevant input and state are the same, deterministic code follows the same rule.

An LLM is useful when the input is ambiguous or unstructured, but its output is model-driven and can vary with context, model, and decoding settings.

This suggests a practical engineering principle:

Use deterministic application controls for rules that must be enforced exactly. Use model capability where flexible language understanding or generation adds measurable value.

Examples of deterministic controls:

• schema validation,
• authorization,
• allowed enum values,
• rate limits,
• spend caps,
• side-effect restrictions,
• tool allowlists.

Examples where models may add value:

• summarization,
• intent understanding,
• semantic classification,
• extraction from messy text,
• reasoning over unstructured context,
• planning within bounded controls.

---

8. Typed Data Contracts

External data should be treated as untrusted until validated.

Suppose the Service Desk API receives this JSON:

{
  "id": "INC-1001",
  "email": "student@example.com",
  "priority": "high",
  "body": "VPN is not connecting"
}

Application code should not simply assume every field is present and valid.

Pydantic lets us declare an expected contract.

Example:

from enum import Enum
from pydantic import BaseModel, EmailStr, Field

class Priority(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"
    CRITICAL = "critical"

class Ticket(BaseModel):
    id: str = Field(min_length=3, max_length=50)
    email: EmailStr
    priority: Priority
    body: str = Field(min_length=1, max_length=4000)

Now malformed values can be rejected at the application boundary before downstream logic relies on them.

---

9. What Validation Does Not Mean

This distinction is extremely important.

Validation

Does the data match the declared shape and constraints?

Authorization

Is this user allowed to perform this operation?

Factual correctness

Is the information actually true?

These are different questions.

A syntactically valid email address does not prove the user owns it.

A valid priority value does not prove the user is authorized to assign it.

A valid JSON response from an LLM does not prove the contents are factually correct.

Remember:

Valid structure ≠ authorized action ≠ true information.

---

10. Why Start with a Deterministic Router?

Our first Service Desk router uses explicit rules.

For example:

security-related issue → security queue
billing-related issue  → billing queue
technical issue        → technical queue
otherwise              → general queue

Why not immediately ask an LLM?

Because the deterministic implementation gives us a baseline.

Later we can compare a model-driven classifier against that baseline using evidence such as:

• quality on a test dataset,
• latency,
• cost,
• failure modes,
• operational complexity.

Without a baseline, “AI is better” is only an assumption.

---

11. Secret Management Basics

Never commit real API keys, passwords, or credentials to Git.

A typical local development pattern is:

.env            ← actual local values, ignored by Git
.env.example    ← variable names/placeholders, safe to commit

Example .env.example:

MODEL_API_KEY=
MODEL_NAME=

The actual .env might contain a real secret locally, but it should be excluded from version control.

In production, secrets are normally supplied through secure environment/configuration systems rather than committed source files.

---

12. Day 1 Lab

Goal

Build or review the typed Service Desk baseline and make the Day 1 tests pass.

Setup

cd service-desk-day-01
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pytest -v

On Windows PowerShell, virtual-environment activation will use the corresponding Windows path.

Tasks

Task 1 — Inspect the data model

Find:

• Ticket
• Priority
• RouteTarget

For each field, explain why its type or constraint exists.

Task 2 — Inspect the router

Trace the routing order.

Ask yourself:

• Which rules have higher priority?
• Could two keywords match?
• Is precedence explicit?

Task 3 — Run tests

pytest -v

Do not stop at “green tests.” Read the test names and understand what behavior each one checks.

Task 4 — Break the input intentionally

Try an invalid payload such as:

{
    "id": "X",
    "email": "not-an-email",
    "priority": "urgent",
    "body": ""
}

Observe the validation errors.

Task 5 — Reason beyond validation

Answer:

If the payload is structurally valid, what important problems might still remain?

Possible answers:

• the user may not be authorized,
• the email might not belong to the user,
• the issue description may be false,
• the routing rules may be incomplete.

---

13. Failure Scenario

Imagine this code:

text = ticket.body.lower()

If invalid input allows body=None to reach this point, the application may fail with an error such as:

AttributeError: 'NoneType' object has no attribute 'lower'

A stronger boundary validates required input before downstream code assumes it exists.

But remember: schema validation improves input safety; it does not solve authorization, business correctness, or every runtime failure.

---

14. Knowledge Check

Question 1

What is the practical difference between training and inference?

Question 2

Why can an LLM produce a fluent but false answer?

Question 3

What is the difference between an LLM and an AI application?

Question 4

Why should authorization not be enforced only through prompt instructions?

Question 5

What does Pydantic validation give us?

Question 6

Does valid structured output prove factual correctness? Why or why not?

Question 7

Why do we build a deterministic baseline before adding an LLM classifier?

---

15. Scenario Questions

Scenario A — Password reset

A model decides that a user requesting a password reset “sounds legitimate.” Should that be enough to reset the password?

Expected reasoning: No. Identity verification and authorization must be enforced by the application/security system.

Scenario B — Valid JSON, wrong value

An LLM returns:

{
  "priority": "critical"
}

The JSON is valid and critical is a permitted enum value. Does that prove the ticket is truly critical?

Expected reasoning: No. Schema conformance and semantic correctness are different.

Scenario C — Simple exact routing rule

Every ticket with a verified security_incident=true must go to the security queue. Should we ask an LLM to decide this?

Expected reasoning: Usually no. The deterministic rule is simpler and enforces the requirement exactly.

---

16. Homework / Extension

Complete the following without adding an LLM:

1. Add one new RouteTarget, for example ACCESS.
2. Add an explicit deterministic routing rule for access/login-related tickets.
3. Add at least two tests:
   • one positive routing case,
   • one precedence or boundary case.
4. Write three sentences explaining when a future LLM classifier might outperform these rules.
5. Write three sentences explaining which rules should remain deterministic even after an LLM is introduced.

Optional challenge:

Create a small table of 10 example tickets and manually label the expected route. This will become the beginning of an evaluation dataset later in the course.

---

17. Day 1 Summary

You should leave Day 1 with five ideas:

1. An LLM is a probabilistic model component, not the complete production application.
2. Production AI still requires validation, authorization, tests, observability, and failure handling.
3. Typed contracts catch many malformed inputs early and make assumptions explicit.
4. Deterministic rules remain the right tool for many exact requirements.
5. The Service Desk deterministic router gives us a baseline that future AI capability must justify through measurable improvement.

Next: Day 2 — Async Python, Event Loops & Concurrent I/O.