What an Agent Knows — and Why It Sometimes Seems to Forget
It can look like forgetting. Usually, it is something more ordinary: the information was kept in the wrong place — or it was never kept at all.
At 9:00 a.m., you ask an operations agent to do a familiar job: look at yesterday’s store performance and recommend what to do today.
It begins well. It pulls GMV. It checks refunds. Then it asks for Store B’s settlement data and gets nothing back. The report cannot be finished, so it records the work as pending.
An hour later, the agent is called again. It checks GMV again. It checks refunds again. It acts as though Store B had never existed.
This can look like forgetting. Usually, it is something more ordinary: the information was kept in the wrong place—or it was never kept at all.
An agent does not have one big, undifferentiated memory. It works with several kinds of information, each with its own job and lifetime. Once those lifetimes are clear, “remembering” and “forgetting” become much easier to explain.
Six places information can live
← scrolls horizontally · left stripe = the lifetime that governs the row
| Concept | Lifetime | What it means in the store-report example | Where it lives |
|---|---|---|---|
| Context | One model call | What the model sees right now: the system prompt, part of the conversation, instructions from an active skill, GMV results, file excerpts, and retrieved memories | The model’s input window. When the call ends, this view disappears. Atlan |
| Session | One task | The whole morning’s run: the user’s request, the order of tool calls, and working drafts such as “GMV first, refunds second” | Runtime session state. It can be discarded after the task finishes or times out. OpenAI Developers |
| State | Across calls, but temporary | Store B: settlement pending. It tells the next call where this unfinished report stopped | An external database or queue, rather than the context window. Mem0 |
| Memory | Across tasks, long term | The store’s timezone, finance’s required table, and the rule that missing values must read N/A | A vector store or structured key-value store. It enters Context only when retrieved. Machine Learning Mastery |
| Artifact | Long-lived | The finished daily report | A file system or object store |
| Event log | Long-lived and append-only | Which tools ran, what they returned, and who approved the report | An audit log for review and compliance, not normally model input |
The useful distinction is not technical. It is practical.
Context and Session are both alive while work is happening. But Context is brief: it belongs to one exchange with the model. A Session is broader: it covers the entire task.
State and Memory both outlast an exchange. But State is unfinished business with an expiry date. Memory is a durable fact that may matter next week, next month, or in another task entirely.
That is why Store B: settlement pending belongs in State. It will stop being true as soon as the settlement arrives. The store’s timezone belongs in Memory. It should not need to be rediscovered every morning. Confuse the two, and the agent may either preserve stale information or redo work it should have retained. OpenAI Developers Mem0
What happens when the report resumes?
When Store B fails to respond, the agent should not rely on the chat transcript to carry the fact forward. It should write Store B: pending to State storage.
Later, when the system calls the model again, the runtime—sometimes called the harness—builds a new Context. It brings together the system prompt, the conversation that still matters, the full instructions for any active skill, recent tool results, relevant file excerpts, retrieved Memory, and the saved State.
The agent may now see two very different facts side by side: the long-term rule that missing figures must be marked N/A, and the short-term fact that Store B is still pending. The first came from Memory; the second came from State. Together, they allow the report to continue without pretending the number is available.
This assembled Context is a working desk, not a filing cabinet. It is rebuilt before each model call and cleared when that call ends. Atlan
The desk is also finite. A runtime may include around 5,000 characters from a file in a small 8K window, and around 40,000 in a 200K-plus window. The numbers vary. The point does not: the model sees a chosen slice of the history, not the whole archive.
A skill is not just another tool
Tools do something. A GMV tool runs a query and returns a number. A refund tool returns a rate. A Store B tool tells you whether settlement data exists.
A skill tells the agent how to do the work. It may say which checks come first, how to handle a missing value, or how to lay out the final report.
With Claude Skills, for example, choosing a skill can place the full SKILL.md into the conversation and change the execution environment, including available tools or model selection. It is better understood as prompt expansion plus context adjustment than as a one-shot action that returns a result. Claude Skills deep dive
| Type | In plain language | In this report |
|---|---|---|
| Tool | “Go get this.” | Fetch GMV, refunds, or Store B settlement data. |
| Skill | “Here is how to proceed.” | Explain how to analyze the numbers, mark N/A, and structure the report. |
This matters because the skill’s instructions use space too. Once active, they compete with tool output, file excerpts, and retrieved Memory for room in Context.
When the window gets crowded
The model does not usually choose what to remove. The runtime does, following a policy.
One common policy is a sliding window. The runtime drops the oldest messages until the input fits again. Some systems remove about 30 percent first, then trim in further 10 percent steps.
Another policy looks at dependencies. It can remove a completed tool-call record that nothing later relies on. It can then strip away exploratory material in layers: reasoning traces first, then large tool outputs and file-reading details, and finally a full section replaced with a short summary. Arize
The first approach is simple, but it has a weakness. The original request may be crucial and still be old. If the runtime removes it only because it came first, the agent can lose the thread of the job. Reddit discussion
The safer answer is not to keep every important fact in Context forever. Put unfinished work in State. Put stable facts in Memory. Save the report as an Artifact. Keep actions and approvals in the Event log. Then Context can stay small and useful. When a fact falls out of view, the system can retrieve it instead of guessing.
Frequently asked questions
Is Context the agent’s memory?
No. Context is what the model can see during one call. Memory is held outside the model and retrieved when it is relevant. Until it is retrieved, the model cannot use it in that call.
Does the report disappear when the Session ends?
It should not. The Session can end, while the completed report remains as an Artifact. The Event log can preserve the trail of actions and approvals.
Why not save Store B: pending as long-term Memory?
Because it is a temporary condition, not a lasting fact. Once the settlement data arrives, the statement is wrong. Keep it in State with a clear completion or expiry condition; otherwise a later task may mistake an old delay for a current one.
Why does an agent sometimes run the same query twice?
The first result may have existed only in Context, which has already disappeared. Or State may not have been saved, or not read back. In many cases, repeated work is a problem of information handling around the model, not a failure of the model itself.
How do you prevent real information loss when Context is trimmed?
Do not make Context the only home for important information. Keep temporary work in State, durable rules in Memory, finished files as Artifacts, and a record of actions in the Event log. Context should hold only what the current call needs to think clearly.
This is Week 3 of a 26-week series on agent systems from first principles: capability maps, skills vs. tools, memory and context, loops, harnesses, and the patterns that separate a demo from production.
Happy learning, happy building.
Tao