A Zero to Prod book
Keeping AI Agents Honest
What AI Platform Engineers and Architects Actually Do, and How to Do It. By Bill McCoy.
The harness, not the model.
Free sample: the first 30 pages, through chapter 1, about 50 minutes to read. The whole book is 589 pages.
Written by AI agents. Kept honest by the harness Bill McCoy built, the one this book teaches.
“Chicken or egg? The harness came first. Then the agents wrote the book inside it.”
From the book
About this book
What this book is
This is a hands-on guide for engineers and architects who have to make AI agents safe, reliable, measurable and affordable inside an organization. It covers two kinds of agent:
- Coding agents, which work in your codebase: Claude Code, Codex CLI, GitHub Copilot, Cursor, Devin Desktop (formerly Windsurf) and others.
- Product agents, which work inside your application: an assistant that reads records, calls tools and takes actions for your users.
Its subject is everything around the model: what the agent is told, what it's allowed to touch, the checks its work has to pass, how you know whether it's any good, and what it costs. The book calls that the harness. The model usually comes from someone else; the harness is yours to build. Most of the job postings behind chapter 1 ask for work on agents, and two-thirds or more ask for observability, evaluation, guardrails and security. All four are harness work.
Every guardrail in the book is built and run in a companion repository: a small helpdesk application with an agent inside it. You don't take a check on trust. You run it, break it on purpose, and watch it fail with a message that says what to do.
By the end, you should be able to:
- Read an "AI platform engineer" or "AI architect" job posting and say which of five different jobs it describes.
- Explain what an agent is, what a model call costs and why its output changes from run to run, well enough to predict how it fails.
- Give an agent the context it needs without drowning it, and keep what it reads true with checks.
- Design tools and MCP (Model Context Protocol) servers that act with the permissions of the person the agent works for.
- Turn rules your team keeps in prose into checks that fail the build with a message an agent can act on, and put a person's approval in front of the actions that matter.
- Tell whether an agent is any good when its output changes every run, and hold back a new prompt, model or tool until it passes.
- Run it all for an organization, with one gateway for model calls, budgets, and numbers that say whether the harness works.
- Take a request to production along one path, with a person deciding at each gate and a page at the end that says which claims were measured to hold.
- Walk into a harness someone else built and choose the first thing to fix.
Who it's for
Engineers and architects who already know software delivery: git, the command line, APIs, CI and containers. You're new to AI agents, or you've used them yourself and now have to make them safe for other people. Most often that means a platform, DevOps (development and operations), infrastructure, site reliability or backend engineer moving into AI platform work; an architect who sets the patterns and gates those teams build on; or a tech lead who has to hire for either. The introduction describes both roles, and the jobs around them.
The book assumes nothing about AI. Every AI idea is explained from first principles, starting with the smallest real example, and every term is defined the first time it appears.
It's probably not the book for you if:
- You want to train, fine-tune or serve models, or run GPU (graphics processing unit) clusters. That's the infrastructure kind of AI platform job. Chapter 1 shows how to spot it in a posting and points to books that cover it.
- You're new to software delivery itself. The first chapter's lab already assumes a terminal, git and Node.js.
- You want a reference for specialists, or a tour of every product. This is plain English for engineers and architects new to the field. It goes deep on a few mechanisms, and dates what it says about any product, because product names and features change.
- You want a lab built on a coding tool other than Claude Code, or on a model API other than Anthropic's or OpenAI's. The companion repository's hooks and settings are Claude Code's, and its clients for a real model call Anthropic's API and OpenAI's; everything else runs on a mock. Appendix A maps the same harness across five coding tools, and the chapters teach each general idea with more than one vendor's version.
What it covers, and what it doesn't
The book opens with an introduction, then has nine parts and seven appendices.
| Part | Chapters | What it covers |
|---|---|---|
| Introduction | Before chapter 1 | The roles, where AI stands today, who makes the models and how they reach you, and what changes fast |
| I. Getting Started | 1 to 5 | The job family, from a dated sample of real postings; AI models, just enough to reason about them; what makes an agent; the eight parts of a harness; setting up the companion repository |
| II. Context: What the Agent Knows | 6 to 10 | Instruction files that stay small; the repository as the source of truth; documentation checked against the code; retrieval without the hype; work that spans many sessions |
| III. Tools: What the Agent Can Do | 11 to 14 | Designing tools for agents; building an MCP server; governing MCP servers across an organization; multi-agent patterns |
| IV. Guardrails: What the Agent Can't Break | 15 to 20 | Fitness functions and structural tests; boundary rules; custom linters that teach; policy engines; permissions, sandboxes and human approval; the agent attack surface |
| V. Evaluation: Knowing Whether It Works | 21 to 23 | Evaluating output that changes every run; model judges; evaluations as a gate |
| VI. Feedback Loops | 24 and 25 | What belongs in CI when agents write the code; sending a failed check back to the agent |
| VII. Running It | 26 to 31 | Measuring a harness; gateways, cost and model routing; the central AI team; rolling it out to teams; Zero to Prod, one path from a request to production; walking into a harness someone else built |
| VIII. Your Career | 32 and 33 | Skills by kind of job, a path through the book and a portfolio; interviewing for AI platform roles |
| IX. Quick Lists | 34 to 36 | Ten failure modes and how to catch them; ten guardrails worth adding first; ten questions to ask in your first week |
The appendices:
- A. The Same Harness in Five Tools. Where Claude Code, Codex CLI, GitHub Copilot, Cursor and Devin Desktop read their instructions, where their hooks live, how they connect MCP servers, and how each runs without a person at the keyboard. Every entry is dated.
- B. The Zero to Prod Kit. Every file chapter 30's path produces, from the intake brief to the proof page, 40 files in the companion repository's
templates/folder, in the order its stages produce them. Each comes as a skeleton to copy, a filled example and the check that proves it, and all but the week-one assessment checklist, which is for a person, are proved by a test in the companion repository. - C. A 90-Day Plan for a New AI Platform Role, in four phases, with a 60-day cut for a short engagement.
- D. Dogfooding: How This Book Was Built. The book's own harness, from its records, and what it never measured.
- E. Glossary. The terms the book defines, in plain words, each with the chapter that explains it.
- F. Sources. Every work and document the book cites, what each is good for, the chapters that cite it and the date it was read.
- G. Quick-Reference Card. One page: the eight parts of a harness, the lab's main commands, the first ten guardrails, week one and its ten questions, and the ten failure modes, each pointing to its chapter.
How to use it
Read the introduction, then Part I, because the rest of the book uses their vocabulary. After that, every chapter stands on its own, in any order.
Every chapter opens with three bullets that say what the chapter gives you. If none of the three is something you need today, skip it. Chapters 1 to 33 then take the same steps: the problem, how it goes wrong, how it works, a Try it, a field note where there is one, how to tell it's working, what to check before you rely on it, how to talk about it in an interview, and a short Keep this list. The three quick lists in Part IX, chapters 34 to 36, keep the same frame, with ten items in place of the problem, how it goes wrong and how it works.
Two more labels keep the book honest. A made-up example is marked "illustrative scenario," and anything real comes from a cited source or a field note. Each "Check before you rely on it" list names the parts of its chapter most likely to go stale, with dates.
Paths through the book
| If you're | Read, after Part I |
|---|---|
| Heading for an enablement job: a central team whose AI platform other teams build on | Part VII, then Parts IV and V |
| Building the harness around coding agents | Parts II, IV and VI |
| Building agents into a product | Parts III and V, then chapter 19 |
| Running one AI system for one purpose (what chapter 1 calls applied) | Parts III and V |
| Setting up how requests become production systems | Chapter 30, then Appendix B |
| Starting a new role, or inheriting a harness | Chapters 31 and 36 in your first week, then chapter 35 and Appendix C |
| Leading a team, or hiring for one | Chapter 1 (it also says how to write a posting), Part IX, then chapters 26, 28 and 29 |
| Preparing for interviews | Chapters 32 and 33, and the interview section at the end of each chapter you read |
| Short on time | Part IX: three lists of ten, each pointing to the chapter that goes deeper |
The first four rows follow chapter 1's kinds of job. Chapter 32 turns each into an order through the whole book, and a script in the companion repository, postings/skills.mjs, prints that order for any kind of job. The last five rows are my suggestions.
Try it, and what it costs
Every chapter, and Appendices B and C, has a Try it section: steps to run in the companion repository, with the exact commands and the output to expect. Most include a step that breaks something on purpose, so you see a guard fail before you rely on it. Each chapter has a tag in the repository, and every Try it starts by checking out its tag, which gives you the repository at the commit that chapter's Try it was checked against, so your output matches the book's. A tag can hold more than its chapter builds: chapter 9's tag already contains the code chapters 15 to 18 and 24 explain, and several tags share a commit (9 and 10, 14 to 18, 24 and 25). Chapter 5 explains the tags and the setup that goes with them.
Everything runs against a mock model by default, at no cost. The mock plays the model's part from a fixed script, so it needs no network connection or account and gives the same output every time. It checks the machinery, not a model's judgment; chapters 21, 22, 23, 26 and 27 each say what only a real model can answer. Spending money is something you ask for, one command at a time. A command calls a real model only when you add --real and the credential for its model's provider, Anthropic or OpenAI, is present, and every such run is bounded: the evaluation commands refuse --real without a dollar cap, the triage assistant stops at its turn limit, and every real call passes the lab's budget check first. Treat --real itself as the switch that spends money.
The callouts
- Field note: something that happened in my own production systems, anonymized, with real numbers, failures included, and checked against those systems' records. They are one engineer's small portfolio of apps, so they show how a mechanism behaves, not how it scales.
- Try it: steps to run in the companion repository, with the output to expect.
- Watch out: a mistake that costs you.
- Under the hood: detail you can skip on a first read.
- Keep this: the lines to remember. Every chapter ends with a few.
The companion repository
The companion repository, gracezero-agent-harness-lab on GitHub, holds everything the labs run: a small helpdesk application with a triage assistant inside it, a deterministic mock model with clients for Anthropic's API and OpenAI's behind the same interface, the book's guardrails with tests that plant a violation to prove each one catches it, chapter 1's job-postings sample, and Appendix B's templates. It was written from scratch for this book and contains none of my private code or data. Chapter 5 walks through all of it.
What you need: Git 2.45 or newer (git --version shows yours), Node.js 24 or newer, and Python 3.12 or newer, plus an internet connection the first time setup runs. The repository's CI runs it with Python 3.14 on Linux, Windows and macOS, and with Python 3.12 on Linux. Chapter 16's Go, Java and .NET rules run only in CI, so you don't need those languages.
The repository's address: https://github.com/BillM9000/gracezero-agent-harness-lab
The book's page: https://gracezero.com/keeping-ai-agents-honest.
Getting started. On macOS or Linux:
git clone https://github.com/BillM9000/gracezero-agent-harness-lab ~/src/agent-harness-lab
cd ~/src/agent-harness-lab
node setup.mjs
node check.mjs
On Windows, clone into a folder with a short path, such as C:\src (chapter 5 explains why):
git clone https://github.com/BillM9000/gracezero-agent-harness-lab C:\src\agent-harness-lab
cd C:\src\agent-harness-lab
node setup.mjs
node check.mjs
node setup.mjs installs every package at the exact version the lock files list, and is safe to run again. node check.mjs runs every check the repository has, one line each, apart from chapter 16's Go, Java and .NET rules, which CI runs in jobs of its own, and chapter 24's mutation run, which takes minutes and runs nightly. It's the command CI runs.
How this book was built
This book was built inside the harness it teaches: a spec, a short instruction file, a checker that fails when a citation doesn't resolve, a fresh copy of the repository for every Try it, and a person approving whatever can't be undone. The line under the epigraph says who did what. Appendix D, "Dogfooding: How This Book Was Built," is the long version, told from the records.
Copyright 2026 Bill McCoy. All rights reserved.