Building AI Agents — A Course for Software Engineers & Architects
About This Course
Who This Is For
Software engineers and architects who can already code and design systems, and want to add agentic AI systems to their toolkit — not AI beginners.
How to Use This
Read sequentially — each chapter assumes the architectural vocabulary built in the ones before it. Key terms are called out for quick reference.
Estimated Time
Approximately 60–80 hours of self-paced study, plus hands-on time prototyping agents alongside each chapter.
What You’ll Build
The architectural judgment to design, build, secure, and ship reliable single- and multi-agent systems in production.
Chapter Road Map
Ch 1–2
Foundations
What an agent actually is, and the LLM mechanics every agent builder must know.
Ch 3–4
Core Capabilities
Giving agents hands (tool use) and a mind that persists (memory).
Ch 5–6
Reasoning & Knowledge
How agents plan multi-step work and ground themselves in real, current data.
Ch 7–8
Architecture & Tooling
Composing agents into systems, and the frameworks engineers actually use to build them.
Ch 9–10
Production Readiness
Testing, observability, cost control, security, and how to ship agents that don’t fall over.
01
Before writing a single line of agent code, you need a precise mental model of what an “agent” is, how it differs from the chatbots and scripts you’ve already built, and the architectural vocabulary the rest of this course relies on.
1.1What Actually Makes Something an “Agent”
The defining trait isn’t using an LLM — it’s a system that autonomously decides its own next action toward a goal, in a loop, without a human scripting every step. Distinguishing this from a chatbot, a workflow engine, and a single LLM call.
1.2The Agent Loop: Perceive → Reason → Act → Observe
The core control loop every agent implements in some form: gathering context, deciding an action, executing it, and feeding the result back in. Why this loop, not the LLM itself, is the architectural unit you design.
1.3Agents vs. Workflows vs. Pipelines — Choosing the Right Pattern
A deterministic pipeline is cheaper, faster, and easier to test than an agent. Agents earn their complexity only when the path to the goal can’t be fully specified in advance. A decision framework for when each fits.
1.4Levels of Agent Autonomy
A practical scale from fixed scripts, to LLM-augmented steps, to single-step tool callers, to fully autonomous multi-step agents — and why most production systems should sit lower on this scale than the hype suggests.
1.5Core Components of an Agent System
Mapping the model, the system prompt/instructions, the tool registry, the memory store, the orchestrator, and the execution environment to concrete pieces of code you will own and maintain.
1.6The Software Engineering Mindset Shift
Traditional software is deterministic; agents are probabilistic and non-reproducible by default. Why this forces new habits around testing, versioning, and defining “correct” behaviour.
1.7Where Agents Fit in a Larger System Architecture
Treating an agent as a bounded service with a contract — inputs, outputs, side effects, and failure modes — so it can be integrated into existing microservice or monolith architectures cleanly.
1.8Synchronous vs. Asynchronous Agent Execution Models
Request-response agents for low-latency interactive use cases versus long-running, checkpointed background agents for multi-hour tasks — and the very different infrastructure each demands.
1.9Common Agent Archetypes You’ll Build
A field guide to recurring patterns: coding agents, research/retrieval agents, customer-support agents, data-pipeline agents, and DevOps/SRE agents — and the architectural emphasis each one demands.
1.10Reading and Evaluating Agent Architecture Diagrams
A standard visual vocabulary — model nodes, tool edges, memory stores, control-flow loops — so you can read, critique, and produce agent architecture diagrams the way you would a sequence diagram.
1.11Setting Up Your First Engineering Sandbox
A minimal local environment — API access, a single tool, a logging layer — built deliberately bare so every later chapter’s additions are visible and attributable to a specific concept.
02
An agent is only as reliable as your command of the model underneath it. This chapter covers the LLM mechanics that directly determine whether your agent behaves predictably in production — not theory for its own sake.
2.1Context Windows as a Finite, Shared Resource
Every system prompt, tool definition, memory entry, and conversation turn competes for the same fixed budget. Why context window management is one of the first real engineering constraints you hit.
2.2System Prompts as an Architectural Contract
Treating the system prompt like an API specification — defining role, constraints, output format, and escalation behaviour — rather than a loose set of instructions tweaked by trial and error.
2.3Structured Output & Schema Enforcement
Forcing the model into JSON Schema or typed outputs so the rest of your codebase can consume agent responses like any other API payload, including handling and retrying malformed responses.
2.4Function/Tool Calling — The Mechanism Underneath
How the model actually decides to emit a tool call versus text, what the request/response cycle looks like at the API level, and why this is the single most important capability for agents.
2.5Temperature, Determinism & Reproducibility Tradeoffs
Why lower temperature doesn’t make an agent deterministic, what actually causes run-to-run variance, and where reproducibility matters enough to engineer around it.
2.6Prompt Engineering Patterns Specific to Agents
Few-shot tool-use examples, explicit reasoning scaffolds, and self-correction instructions — prompting techniques that measurably change agent reliability, not generic “better prompting” advice.
2.7Model Selection for Agentic Workloads
Comparing models on tool-calling accuracy, instruction-following under long contexts, and latency/cost — and why the “smartest” model is often the wrong default for high-volume agent steps.
2.8Streaming & Partial Output Handling
Architecting agents to act on streamed tokens for responsiveness, while safely deferring tool execution until a complete, validated tool call has arrived.
2.9Token Economics & Cost Modelling
Building a per-request cost model across input tokens, output tokens, and tool round-trips, so cost is a design input rather than a surprise on the first production invoice.
2.10Failure Modes at the Model Layer
Hallucinated tool arguments, refusal loops, repetition, and silent truncation — recognising these signatures early so you can attribute bugs correctly instead of blaming your orchestration code.
2.11Working With the Anthropic / OpenAI SDKs Directly
Reading raw request/response payloads for messages, tool_use blocks, and stop reasons before relying on any framework abstraction — so you can debug what frameworks hide.
03
Tools are what turn a text generator into an agent that affects the real world. This chapter is about designing tools the way you’d design any API surface — with discipline, validation, and failure handling.
3.1Designing Tool Schemas Like API Contracts
Writing tool names, parameter descriptions, and types so precisely that the model’s tool-selection accuracy improves measurably — schema quality is a primary lever, not documentation boilerplate.
3.2Granularity: One Big Tool vs. Many Small Tools
The tradeoff between a few flexible tools that increase model decision-making burden, and many narrow tools that increase context size and selection ambiguity — and how to land in between.
3.3Building a Tool Registry
A central, versioned catalogue mapping tool names to schemas and executable handlers, so tools can be added, deprecated, and permissioned without touching agent logic.
3.4Input Validation & Sanitisation Before Execution
Never trusting model-generated tool arguments directly — validating types, ranges, and allow-lists before a tool call touches a database, filesystem, or external API.
3.5Designing Useful Tool Error Responses
Returning errors the model can actually reason about and recover from — structured, specific, actionable messages — instead of raw stack traces that send the agent into a retry loop.
3.6Idempotency & Side-Effect Safety
Designing tools so retries from agent confusion or network failure don’t double-charge a card or send a duplicate email — the same idempotency discipline you’d apply to any distributed system.
3.7Sandboxing Code-Execution Tools
Running model-generated code in isolated, resource-limited, network-restricted environments — containers, microVMs, or WASM — so a code-execution tool can’t become an attack surface.
3.8Parallel vs. Sequential Tool Calls
When models can request multiple tool calls in one turn, deciding which can safely run concurrently and which must be serialised due to shared state or ordering dependencies.
3.9Tool Permissions & Scoped Access Control
Binding each tool invocation to the calling user’s actual permissions rather than a privileged service account, so an agent never has broader access than the human it acts on behalf of.
3.10Rate Limiting & Circuit Breakers for Tool Calls
Protecting downstream systems from runaway agent loops with per-tool rate limits, timeouts, and circuit breakers — defensive patterns borrowed directly from distributed systems engineering.
3.11The Model Context Protocol (MCP) for Tool Standardisation
How MCP standardises tool and data-source exposure across agents and vendors, why it matters for avoiding N×M integration sprawl, and how to expose your own services as MCP servers.
3.12Testing Tools in Isolation From the Model
Unit-testing tool handlers against their schema independently of any LLM call, so tool bugs and model-reasoning bugs are never conflated during debugging.
04
An agent without memory re-derives everything from scratch every turn. This chapter covers how to give agents state — within a session, across sessions, and across users — without blowing the context budget or leaking data between users.
4.1Short-Term vs. Long-Term Memory — An Architectural Distinction
Short-term memory lives in the active context window for the current task; long-term memory persists in external storage and must be deliberately retrieved. Conflating the two is a common design mistake.
4.2Conversation History Management & Truncation Strategies
Sliding windows, importance-based pruning, and turn-budget allocation — concrete strategies for keeping a growing conversation within the context window without losing critical state.
4.3Summarisation-Based Memory Compression
Periodically compressing older turns into a running summary the model can read instead of the raw transcript, and the information-loss tradeoffs this introduces.
4.4Vector Stores as Long-Term Episodic Memory
Embedding past interactions or facts and retrieving the most relevant ones by similarity search, rather than replaying full history on every turn.
4.5Structured Memory: Key-Value, Graph & Relational Approaches
When unstructured vector recall isn’t precise enough — using explicit fact stores, knowledge graphs, or relational schemas for memory that needs to be queried exactly, not just “similarly.”
4.6Working Memory / Scratchpad Patterns
Giving an agent an explicit space to write intermediate reasoning, plans, or partial results during a multi-step task — distinct from both conversation history and persistent memory.
4.7Memory Write Policies — What Gets Remembered & When
Deciding whether memory is written automatically after every turn, explicitly by tool call, or via an asynchronous background process — each with different latency and consistency implications.
4.8Memory Scoping & Multi-Tenant Isolation
Strictly partitioning memory by user, session, or organisation so retrieval can never leak one user’s data into another’s context — a security requirement, not a nice-to-have.
4.9Memory Staleness, Conflict Resolution & Forgetting
Handling contradictory facts learned over time, expiring outdated memory, and deliberately implementing “forgetting” for both correctness and privacy compliance.
4.10Evaluating Memory System Quality
Testing recall precision, retrieval latency, and whether irrelevant memories get pulled into context and degrade response quality — memory needs its own evaluation suite, separate from the agent’s.
4.11Choosing a Memory Backend: Redis, Postgres, Vector DBs & Hybrids
Practical tradeoffs between latency, query flexibility, and operational familiarity when picking infrastructure most engineering teams already run versus a dedicated vector database.
05
The orchestration pattern you choose determines how an agent breaks a goal into steps, recovers from mistakes, and knows when it’s done. These are the load-bearing design patterns of agentic systems.
5.1The ReAct Pattern — Reasoning Interleaved With Acting
Alternating explicit “thought” steps with tool actions and observations, and why making reasoning explicit measurably improves tool-use accuracy over direct action.
5.2Plan-and-Execute Architectures
Generating a full multi-step plan upfront, then executing it step by step with a lighter-weight model — separating expensive planning from cheaper execution.
5.3Task Decomposition Strategies
Breaking an ambiguous high-level goal into well-scoped, independently verifiable subtasks, and recognising when a task resists clean decomposition altogether.
5.4Reflection & Self-Critique Loops
Having an agent (or a second model call) critique its own output against the original goal before finalising it, and the cost/latency tradeoff this adds to every task.
5.5Tree-of-Thought & Multi-Path Exploration
Exploring several candidate reasoning paths in parallel and selecting the best, for problems where the first plausible approach often isn’t the correct one.
5.6Defining Termination Conditions
Explicit, checkable conditions for “the task is done” — not just “the model stopped calling tools” — to prevent both premature stopping and infinite looping.
5.7Loop Detection & Runaway Agent Prevention
Detecting repeated identical actions, oscillating states, or step-count explosions, and forcing a hard stop, escalation, or fallback before cost and risk spiral.
5.8State Machines as an Alternative to Free-Form Looping
Constraining an agent’s behaviour to an explicit finite state machine with defined transitions — trading some flexibility for dramatically more predictable, debuggable behaviour.
5.9Graph-Based Orchestration (Nodes, Edges & Conditional Routing)
Modelling an agent’s workflow as a directed graph with conditional edges, enabling branching, cycles, and parallel paths that are explicit and inspectable in code.
5.10Human-in-the-Loop Checkpoints in a Plan
Designing explicit pause points where a plan is surfaced for human approval before high-stakes or irreversible steps execute.
5.11Comparing Orchestration Patterns by Task Shape
A practical mapping from task characteristics — open-ended vs. well-defined, reversible vs. irreversible, single-shot vs. long-running — to the orchestration pattern that fits best.
06
Agents that need current, private, or large volumes of knowledge can’t rely on what’s baked into model weights. This chapter covers retrieval architecture — and specifically how it changes when retrieval is driven by an autonomous agent rather than a fixed pipeline.
6.1Why Agents Need Retrieval — Limits of Parametric Knowledge
Model weights encode statistical patterns, not a queryable, citable, updatable knowledge base — the architectural reason retrieval exists at all.
6.2Chunking Strategies for Real-World Documents
Fixed-size, semantic, and structure-aware chunking, and why a poorly chosen chunking strategy is the most common root cause of bad retrieval results.
6.3Embeddings & Vector Similarity Search in Practice
How embedding models map text to vectors, what cosine similarity is actually measuring, and the practical limits of similarity search for precise factual queries.
6.4Hybrid Search — Combining Lexical & Semantic Retrieval
Why BM25-style keyword search still beats embeddings on exact terms (codes, IDs, names), and how to fuse both signals into a single ranked result set.
6.5Re-ranking for Precision
Using a second, more expensive cross-encoder model to re-score an initial candidate set, trading a small latency cost for a meaningful jump in answer relevance.
6.6Agentic RAG — Retrieval as a Tool, Not a Fixed Pipeline
Letting the agent decide when to retrieve, reformulate its own queries, and issue multiple retrieval rounds — versus the traditional fixed retrieve-then-generate pipeline.
6.7Query Rewriting & Decomposition for Better Retrieval
Having the agent rewrite a vague user question into one or more precise retrieval queries before search, substantially improving hit quality on ambiguous inputs.
6.8Grounding & Citation Enforcement
Architecting prompts and post-processing so every factual claim in an agent’s answer traces back to a specific retrieved source, reducing hallucination and enabling verification.
6.9Freshness, Indexing Pipelines & Incremental Updates
Keeping a retrieval index in sync with a constantly changing source-of-truth via incremental re-indexing, rather than expensive full rebuilds.
6.10Connecting Agents to Structured Data Sources
Beyond document retrieval — giving agents safe, scoped query access to SQL databases, APIs, and internal services as additional retrieval surfaces.
6.11Evaluating Retrieval Quality Independently of Generation
Measuring retrieval precision and recall against a labelled set before ever looking at the generated answer, so retrieval bugs and generation bugs are diagnosed separately.
07
A single agent with too many responsibilities becomes unreliable. This chapter covers decomposing work across multiple specialised agents — the distributed-systems design problem hiding inside “multi-agent AI.”
7.1Why Decompose Into Multiple Agents at All
Single agents degrade with too many tools and too much context. Splitting by responsibility narrows each agent’s tool set and prompt, often improving accuracy more than a bigger model would.
7.2Orchestrator–Worker (Supervisor) Pattern
A central orchestrator agent decomposes a task and delegates subtasks to specialised worker agents, then assembles their results — the most common production multi-agent topology.
7.3Peer-to-Peer & Debate-Style Agent Collaboration
Agents that critique or negotiate with each other as equals rather than through a central authority, and where this earns its added complexity versus a supervisor pattern.
7.4Pipeline & Sequential Hand-off Architectures
Agents arranged in a fixed sequence, each consuming the previous agent’s output — simpler to reason about and debug than dynamic delegation, at the cost of flexibility.
7.5Agent Communication Protocols & Message Formats
Defining structured, typed message contracts between agents instead of passing raw natural language, so inter-agent communication can be validated, logged, and replayed like any service call.
7.6Emerging Standards: A2A and Cross-Vendor Agent Interop
How protocols like Agent-to-Agent (A2A) aim to let agents built on different frameworks or vendors discover and call each other, and what to standardise on today versus wait on.
7.7Shared State vs. Message Passing Between Agents
The classic distributed-systems tradeoff reappears here — a shared blackboard/state store versus strict message passing — with the same consistency and coupling consequences.
7.8Conflict Resolution When Agents Disagree
Voting, a designated tie-breaker agent, or escalation to a human — explicit strategies for when two agents reach contradictory conclusions about the same task.
7.9Cost & Latency Multiplication in Multi-Agent Systems
Every additional agent hop multiplies tokens, round-trips, and failure surface — why multi-agent systems must be justified by accuracy or capability gains, not added by default.
7.10Debugging Multi-Agent Systems
Tracing a single user request across several agent boundaries, correlating logs by a shared trace ID, and isolating which agent in the chain introduced an error.
7.11When Multi-Agent Is the Wrong Answer
Recognising when the “multi-agent” label is being applied to what should really be a single well-designed agent with better tools — a common over-engineering trap.
08
Frameworks trade control for velocity. This chapter gives you the engineering criteria to evaluate them honestly, rather than adopting whichever one trends this month.
8.1Build vs. Framework — An Engineering Decision, Not a Trend
Weighing development speed against debuggability, vendor lock-in, and the risk of fighting a framework’s abstractions once requirements get unusual.
8.2Graph-Based Frameworks (e.g., LangGraph)
Modelling agent workflows as explicit state graphs with nodes and conditional edges — strong for production systems that need predictable, inspectable control flow.
8.3Conversation-Based Multi-Agent Frameworks (e.g., AutoGen)
Frameworks that model multi-agent collaboration as a structured conversation between agent roles, and the kinds of tasks this conversational metaphor suits best.
8.4Role-Based Crew Frameworks (e.g., CrewAI)
Assigning agents fixed roles, goals, and backstories within a defined crew structure — fast to prototype, with tradeoffs once you need fine-grained control over execution order.
8.5The Claude Agent SDK & Provider-Native Agent Tooling
Building agents close to the model provider’s own primitives — tool use, computer use, and context management — minimising abstraction layers between your code and the model.
8.6Code-First / “No Framework” Agent Architectures
Implementing the agent loop directly in plain code with the provider SDK — maximum control and minimum magic, at the cost of building your own orchestration primitives.
8.7Evaluating a Framework’s Production Readiness
Checking for first-class support of streaming, retries, checkpointing, observability hooks, and human-in-the-loop interrupts before betting a production system on any framework.
8.8Avoiding Framework Lock-In
Isolating framework-specific code behind your own interfaces so a future migration doesn’t require rewriting your entire agent logic — basic dependency-inversion discipline applied to AI tooling.
8.9Local Development & Testing Tooling for Agents
Mocking model responses and tool calls for fast, deterministic local iteration without burning API credits or waiting on network latency for every test.
8.10Visual / Low-Code Agent Builders — Where They Fit
Appropriate for prototyping and non-engineering stakeholders; recognising their ceiling once an agent needs custom logic, complex error handling, or tight performance budgets.
8.11Keeping Up With a Fast-Moving Ecosystem
A practical approach to evaluating new frameworks and protocols as they emerge — what to pilot, what to ignore, and how to avoid chasing every new release.
09
A demo agent and a production agent are different engineering problems. This chapter covers the testing, observability, and operational discipline that makes the difference.
“The agent worked in the demo” and “the agent works in production” are two completely different engineering claims — only one of them has been tested against the failure modes that actually occur at scale.
— Production Engineering Principle
9.1Why Traditional Unit Tests Aren’t Enough
Non-deterministic outputs break exact-match assertions. Why agent testing needs behavioural and property-based checks instead of, or alongside, classic unit tests.
9.2Building an Evaluation (Eval) Suite
Curated test cases with defined success criteria — task completion, tool-call correctness, output quality — scored automatically or via LLM-as-judge on every change.
9.3LLM-as-Judge — Strengths, Biases & Pitfalls
Using a model to grade another model’s output at scale, while accounting for known biases like favouring longer or more confident-sounding answers.
9.4Regression Testing Across Model & Prompt Versions
Re-running your eval suite whenever the underlying model, prompt, or tool schema changes, since a “silent” model update can quietly degrade behaviour your tests would have caught.
9.5Observability: Tracing Every Step of an Agent Run
Structured logging of every model call, tool call, and intermediate state with correlated trace IDs, so a failed run can be replayed and inspected step by step.
9.6Production Metrics That Actually Matter
Task success rate, average steps-to-completion, tool error rate, cost per resolved task, and latency percentiles — the dashboard an on-call engineer actually needs.
9.7Cost Control & Budget Guardrails
Hard per-request and per-user token/cost ceilings, model routing to cheaper models for simple steps, and alerting before a runaway agent loop becomes a budget incident.
9.8Output Guardrails & Validation Layers
Checking agent outputs against policy, format, and safety constraints before they reach a user or trigger a side-effecting tool — a deterministic safety net around a probabilistic core.
9.9Graceful Degradation & Fallback Strategies
Defined fallback behaviour for tool failures, low-confidence outputs, or budget exhaustion — returning a safe partial result or escalating to a human, never failing silently.
9.10Versioning Prompts, Tools & Agent Configurations
Treating prompts and tool schemas as versioned artefacts with change history and rollback capability, exactly like application code — not as strings edited in place.
9.11Canary Releases & A/B Testing for Agent Changes
Rolling out a new prompt, model, or tool to a small traffic slice first, with the eval metrics to detect a regression before it reaches every user.
9.12On-Call Runbooks for Agent Incidents
Documented response steps for the specific incident classes agents introduce — cost spikes, loop storms, and silently degraded output quality — distinct from traditional service outages.
10
Agents that take autonomous action introduce a threat model that traditional application security doesn’t fully cover. This final chapter is about deploying agents you can trust with real authority.
10.1Prompt Injection — The Core New Threat Model
How untrusted content (a webpage, a document, a tool result) can contain instructions that hijack an agent’s behaviour, and why this is fundamentally different from classic injection attacks.
10.2Defending Against Direct & Indirect Prompt Injection
Input/output filtering, privilege separation between trusted instructions and untrusted data, and treating injection defence as layered, not solvable by a single filter.
10.3The Principle of Least Privilege for Agents
Granting an agent only the specific tools and data access its current task requires, scoped per-request rather than per-deployment, to bound the damage of any single compromised run.
10.4Sandboxing & Blast-Radius Containment
Isolating execution environments and irreversible actions so that even a successfully manipulated agent can’t cause damage beyond a tightly bounded, recoverable scope.
10.5Human-in-the-Loop Approval for High-Stakes Actions
Designing explicit, unavoidable approval gates before financial transactions, data deletion, external communications, or other irreversible actions — regardless of agent confidence.
10.6Audit Logging for Compliance & Forensics
Immutable, complete records of every decision and action an agent takes on a user’s behalf — required both for debugging and for regulatory and legal accountability.
10.7Data Privacy & PII Handling in Agent Pipelines
Tracking what personal data flows through prompts, memory, and logs, and applying redaction, retention limits, and access controls consistently across every stage.
10.8Deployment Patterns: Serverless, Containers & Long-Running Workers
Matching infrastructure to the agent’s execution model — short request-response agents fit serverless well, while long-running or stateful agents need persistent, checkpointed workers.
10.9Scaling Agent Systems Under Load
Horizontal scaling of stateless orchestration, connection pooling to model providers, and queueing strategies for traffic spikes that exceed provider rate limits.
10.10Multi-Provider & Failover Strategies
Architecting an abstraction layer that can fail over between model providers or regions during an outage, without that abstraction degrading day-to-day reliability.
10.11Compliance Considerations for Regulated Industries
Translating sector-specific regulatory requirements — finance, healthcare, government — into concrete constraints on agent autonomy, logging, and data handling.
10.12A Pre-Launch Architecture Review Checklist
A consolidated checklist spanning every chapter in this course — tools, memory, orchestration, evals, observability, and security — to run before any agent goes live.