GuidesAI and agents

AI agent memory: what it remembers, and how

By the Banzai Labs teamUpdated on 2 min read

Short-term memoryThe current conversationLong-term memoryFacts kept between sessionswriterecallRemember little, remember right, and be able to forget
The agent keeps the conversation in short-term memory and writes useful facts to long-term memory.

Two memories

What the agent sees, and what it can look up
Short-term memory
  • The current conversation
  • Limited by the context window
  • Lost at the end of the session
Long-term memory
  • Facts saved in a store
  • Read and written through tools
  • Kept from one session to the next

How it works

  1. 1WriteThe agent saves a useful fact.
  2. 2OrganizeBy user, by topic, with a date.
  3. 3RecallAt the start of a session, or on demand.
  4. 4ForgetWhat is outdated or what the user asks to remove.

The pitfalls

Pitfall Remedy
A remembered mistake keeps coming back Review and correction possible
Memory that grows forever Summaries, expiry, merging duplicates
One customer’s data with another Memory partitioned per user
A malicious instruction saved Memory is data, not an instruction

The concepts: the context window and the AI agent loop. The framework: AI, GDPR and the AI Act.

Frequently asked questions

Why not keep everything in the context?
The context has a limited size and every token reread costs money. After a few exchanges, you summarize and store lasting facts in long-term memory.
What should be remembered?
Stable preferences, decisions, information useful later. Not needless sensitive data, nor model mistakes that could repeat themselves.
What about the GDPR?
A memory about a person is personal data: notice, retention period, and the ability to view and erase what is kept.

Read next

All guides