Context engineering: what the model sees, and when
Most agent failures are context failures. Context engineering is the discipline of deciding what the model sees at every step.
By Diego Bianqui · Updated
From prompt engineering to context engineering
Prompt engineering is about writing a good instruction. That was enough when a model answered one question at a time. An agent is different: it runs for many steps, calls tools, reads documents and remembers earlier turns. At every step, something has to decide what goes into the model's context window — and what stays out. That decision is context engineering.
The context window is the model's entire working memory. Whatever isn't in it doesn't exist for the model; whatever is in it competes for attention. Long contexts cost more, respond slower, and degrade accuracy when they fill up with stale or irrelevant material.
What goes into an agent's context
| Source | Examples | Typical risk |
|---|---|---|
| Instructions | System prompt, role, policies, output format | Too long, contradictory, or buried under later content |
| Task input | The user's request, the ticket, the case file | Missing details the agent then guesses |
| Conversation history | Earlier turns and decisions | Grows without bound; old turns distract |
| Retrieved knowledge | Search and RAG results, documents | Irrelevant or outdated passages |
| Tool definitions | Names, schemas, descriptions of available tools | Too many similar tools confuse selection |
| Tool results | API responses, file contents, query results | Huge payloads, or untrusted text that carries instructions |
| Memory | Notes, preferences, facts saved across sessions | Stale or poisoned memories persist |
| State | Plans, scratchpads, progress markers | Lost on long tasks, causing loops or repeated work |
How context goes wrong
- Overload: so much material that the important instruction or fact gets less attention than it needs.
- Distraction: old turns or long tool outputs pull the model toward the wrong part of the task.
- Confusion: dozens of overlapping tools, so the model picks the wrong one or invents arguments.
- Conflict: instructions, retrieved documents and memory that disagree, with no rule for which wins.
- Poisoning: untrusted content — a web page, an email, a file, a memory entry — that contains instructions the model then follows. This is a security issue, listed by OWASP as memory and context poisoning (ASI06).
Core techniques
| Technique | What it means in practice |
|---|---|
| Select | Retrieve just in time and only what the current step needs. Rank and filter before inserting. Expose only the tools relevant to the task. |
| Compress | Summarize or compact long histories; truncate large tool results and keep a pointer to the full data instead. |
| Structure | Stable instructions first, dynamic content after, clear delimiters and labels for each source so the model knows what is data and what is instruction. |
| Persist | Write durable notes, plans and decisions to external memory, and read them back deliberately instead of keeping everything in the window. |
| Isolate | Give sub-tasks to sub-agents with their own clean context, and pass back only the result. |
| Protect | Treat retrieved and tool-provided text as untrusted data, never as instructions; validate before it can trigger actions. |
Keeping the stable part of the context identical between calls also makes prompt caching effective, which can cut latency and cost substantially on long-running agents.
Measure it — don't guess
Context changes are code changes. A new retrieval strategy, a shorter system prompt or a summarization step can improve one case and break three others. Treat them like any other change: run them against an evaluation suite of representative tasks, compare success rate, cost and latency, and keep the traces. That is where context engineering meets AgentOps.
Context engineering is one half of building an agent; the other half is the harness — the loop, tools, permissions and guardrails around the model. Good context with a weak harness gives you an agent that knows what to do but can't be trusted to do it; a strong harness with poor context gives you a safe agent that does the wrong thing.
A short checklist
- You can list every source that feeds the context, and who controls each one.
- Untrusted content is labelled as data and cannot trigger actions without validation.
- Long histories and large tool results are summarized or truncated by design, not by accident.
- Each agent or sub-agent sees only the tools its task needs.
- Memory has an owner, an expiry policy and a way to correct or delete entries.
- Every context change is evaluated before release.
Related guides
- Harness engineering: the system around the model
- AgentOps: running AI agents in production
- The OWASP Top 10 for Agentic Applications, explained
Sources
- OWASP GenAI Security Project, OWASP Top 10 for Agentic Applications for 2026 (ASI06 Memory & Context Poisoning) (external site)
Working on agents in production? Get in touch.