Agent Evals & Safety: Trust but Verify

By the end

What you'll build

• Classify agent failures with a structured taxonomy, including the silent 'corrupt success' mode.
• Read and debug nested agent traces — thoughts, tool calls, observations, spans — to locate the exact point a run went wrong.
• Bisect a run to separate root cause from downstream symptom, and recognize healthy recovery that only looks like failure.
• Write deterministic assertion checks over trace JSON that grade final state, not prose.
• Choose between deterministic checks and LLM-as-judge, accounting for judge biases like position and self-enhancement.
• Compute and interpret agent eval metrics: pass@1, pass@k, pass^k reliability, and flakiness.
• Model the prompt-injection threat (direct, indirect, tool-output) per OWASP LLM01:2025 and design layered defenses.
• Apply defensive patterns: data/instruction separation, least-privilege tools, allow-lists, human-in-the-loop, and read-back verification.
• Practice defensive red-team thinking against sandboxed agents — graded on the fix, never the attack.
• Build a reusable eval suite and run it in a real open-source harness on your own machine.

Curriculum

What's inside

6 modules · 32 lessons

1. 01

   Why Agents Fail — A Failure Taxonomy

   6 lessons
   • Trust but Verify: Why Evaluation Is the Job
   • The Agent Failure Taxonomy
   • Flashcards: The Failure Taxonomyflashcard sr
   • Corrupt Success: When the Agent Says “Done” and Isn't
   • Trace Lab: Your First Trace — Classify the Failuretrace lab
   • + 1 more lesson
2. 02

   Reading Traces & Spans

   6 lessons
   • Anatomy of a Trace: Spans, Nesting, and Timings
   • Trace Lab: Navigate a Nested Runtrace lab
   • Span Debugging: Bisecting the Run
   • Python Lab: Assert No Forbidden Tool Callpython lab
   • Trace Lab: Detecting Corrupt Successtrace lab
   • + 1 more lesson
3. 03

   Deterministic Checks vs LLM-as-Judge

   5 lessons
   • Two Families of Checks: Assertions vs LLM-as-Judge
   • Python Lab: Assertions Over Trace JSONpython lab
   • Flashcards: Eval Metrics & Termsflashcard sr
   • Python Lab: pass@1, pass@k, pass^k, and Flakinesspython lab
   • Quiz: Checks & Metricsquiz
4. 04

   Prompt Injection & Defensive Design

   6 lessons
   • The Injection Threat Model
   • Trace Lab: Spot the Injectiontrace lab
   • Defensive Patterns: Building Agents That Resist Injection
   • Agent Lab: Fix the Injection (Graded on the Fix)agent lab
   • Error Analysis: Anatomy of an Indirect-Injection Incident
   • + 1 more lesson
5. 05

   Red-Team Thinking (Defensive)

   4 lessons
   • Think Like an Adversary, Build Like a Defender
   • Flashcards: Attack Classes & Their Defensesflashcard sr
   • Trace Lab: Red-Team Scenario — Find the Missing Defensetrace lab
   • Quiz: Red-Team Thinking (Defensive)quiz
6. 06

   Capstone — Build an Eval Suite

   5 lessons
   • Designing an Eval Suite: Coverage, Regression, and CI
   • Python Lab: Capstone — A Mini Eval Harnesspython lab
   • Trace Lab: Capstone — Full Run Inspectiontrace lab
   • Ship It For Real: From Sandbox to a Live Eval Harness
   • Evidence Quiz: You Ran a Real Harness (Course Wrap)quiz

The shape of it

How this course works