Tools & agents
Memory (in AI systems)
Also known as: agent memory
How an AI application remembers things beyond a single request, since the model itself forgets everything once the conversation ends.
Memory is how an AI assistant remembers you and past conversations, because the model on its own does not. Each time you send a message, the underlying model starts fresh; it only "knows" what is put in front of it right now. So applications add memory: they store useful things (your preferences, earlier chats) and feed the relevant bits back in when needed, creating the feeling of continuity.
By default a language model is stateless: it remembers nothing between requests. Any sense of memory is built by the application around it. Two kinds matter: - Short-term memory: the current conversation, kept by re-sending recent messages inside the context window. - Long-term memory: facts, preferences or past interactions stored outside the model (often in a vector database) and retrieved when relevant.
How long-term memory works
Store information, and when a new request comes in, fetch the pieces most relevant to it, using retrieval, and add them to the prompt. This is how assistants "remember" you across sessions and how agents keep track of long tasks.
The limits
Memory is only as good as what is stored and retrieved; too much irrelevant memory clutters the context window and can confuse the model.
Because LLM inference is stateless across calls, "memory" is an application-level construct layered over the model. Short-term memory is the managed context window (conversation history, possibly summarised or truncated); long-term memory externalises state, typically embedding items into a vector store and retrieving relevant entries via RAG to re-inject at inference. Designs vary in what is stored (raw turns, summaries, extracted facts, episodic vs semantic memory) and in read/write policies (when to persist, update, forget).
Key challenges: relevance and recall of retrieval, staleness and conflict resolution, context-budget management (memory competes with task content for tokens), privacy and consent for stored user data, and the "lost in the middle" degradation of long contexts. In agentic systems, memory also spans intermediate results and plans across a trajectory. Emerging approaches add structured/graph memory and learned memory controllers, but robust, reliable long-term memory remains an active engineering and research problem.