Harness engineering: the system around the model
A model is not an agent. The harness around it — loop, tools, permissions, guardrails and evaluation — decides whether it works in production.
By Diego Bianqui · Updated
Agent = model + harness
A large language model on its own only predicts text. It becomes an agent when something around it lets it act: a loop that calls the model, tools it can use, memory it can read and write, rules about what it may do, and checks on what it did. That surrounding system is the harness, and designing it is harness engineering.
The term is new — it gained traction in early 2026 — but the idea will be familiar to any software engineer. The model is a powerful, probabilistic component. The harness is the deterministic engineering that makes that component safe and useful: interfaces, permissions, error handling, observability and tests.
This matters because most production failures of agents are harness failures, not model failures: a tool with more permissions than the task needs, a loop with no budget, a missing approval step, a retry that repeats a payment, no trace to explain what happened. Swapping in a stronger model rarely fixes those.
What a harness contains
| Part | What it decides |
|---|---|
| Control loop | How the agent plans, acts, observes and decides when to stop: step limits, time and cost budgets, termination conditions. |
| Tools | Which actions exist, their names, schemas and descriptions — the agent's whole action space. Often exposed through the Model Context Protocol. |
| Permissions | Which identity each tool call runs as, with what scopes, against which environments. Least privilege, applied to agency. |
| Context | What the model sees at each step: instructions, retrieved data, memory and tool results. See context engineering. |
| Guardrails | Checks on inputs, outputs and tool arguments: validation, policy rules, content filters, allow-lists. |
| Human approval | Which actions pause for a person: payments, deletions, external messages, anything irreversible. |
| State and recovery | Checkpoints, idempotent retries, timeouts and a way to resume or roll back long-running work. |
| Observability | Traces of every step, prompt, tool call and decision, so behaviour can be explained and audited. See AgentOps. |
| Evaluation | Tests that prove the agent does the job — and keeps doing it after a model, prompt or tool change. |
Design principles that hold up in production
- Shrink the action space first. If the agent doesn't need a tool, remove it before you try to secure it. Fewer, well-described tools also make the model more accurate.
- Make permissions explicit and narrow. Each tool runs with its own credentials, scoped to the task, never the user's or an administrator's full access.
- Put a budget on everything: steps, tokens, wall-clock time and money. A loop without limits is an incident waiting for a trigger.
- Make actions idempotent or reversible. Retries are inevitable; duplicate side effects should not be.
- Keep deterministic work deterministic. Validation, routing, calculations and policy checks belong in code, not in a prompt.
- Gate the irreversible. Anything that moves money, deletes data or speaks for the organization waits for a human or a strict policy check.
- Build a kill switch and test it. Stopping an agent — and revoking its credentials — should take seconds, not a deployment.
- Trace by default. If you can't replay what the agent saw and did, you can't debug it, evaluate it or defend it in an audit.
The harness is model-agnostic — keep it that way
Models change every few months. A good harness lets you swap the model behind it and run the same evaluation suite to decide whether the new one is better for your task. That only works if prompts, tools, permissions and evals live in the harness — versioned, tested and owned by your team — rather than being scattered through vendor-specific code.
Frameworks and SDKs (LangGraph, the OpenAI Agents SDK, the Claude Agent SDK and others) give you a starting harness: a loop, tool calling, memory hooks. They are useful. But the decisions that make an agent safe for your organization — which tools, which permissions, which approvals, which tests — are yours to engineer, whatever framework you choose.
How harness engineering connects to security and governance
Most of the risks in the OWASP Top 10 for Agentic Applications are harness problems: tool misuse, identity and privilege abuse, unexpected code execution, cascading failures, rogue agents. The controls live in the harness too — scoped tools, sandboxes, approval gates, budgets, monitoring.
The same is true for governance. An AI management system under ISO/IEC 42001 or the NIST AI RMF needs evidence that controls work. A well-engineered harness produces that evidence as a by-product: traces, eval results, approval logs and permission configurations.
A short checklist
- Every tool has a written purpose, a schema, an owner and the narrowest credentials that work.
- The loop has step, token, time and cost limits, and a defined stop condition.
- Irreversible actions require approval or a policy check that is enforced in code.
- Every run is traced end to end, and traces are kept long enough to investigate incidents.
- An evaluation suite runs on every change to the model, prompts or tools.
- Someone can stop the agent and revoke its access in under a minute — and has practised it.
Related guides
- Context engineering: what the model sees, and when
- AgentOps: running AI agents in production
- The OWASP Top 10 for Agentic Applications, explained
Sources
- OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026 (external site)
- Model Context Protocol, Model Context Protocol specification (external site)
Working on agents in production? Get in touch.