Lesson · 40 min · Free
Short-Term Memory: Making Agents Remember
Short-Term Memory: Making Agents Remember body { font-family: sans-serif; line-height: 1.6; margin: 20px; } h1, h2 { color: #2c3e50; } pre, code { background-color: #ecf0f1; padding: 10px; border-radius: 5px; overflow-x:
Short-Term Memory: Making Agents Remember
Welcome to our exploration of short-term memory in AI agents, a crucial component for developing intelligent systems that can engage in meaningful, multi-turn interactions. In the context of AI, short-term memory, often referred to as "context window" or "working memory," allows an agent to retain information from recent interactions within a single session. Unlike long-term memory, which stores knowledge for extended periods and across sessions, short-term memory is ephemeral. It's about remembering what was just said or done to maintain coherence and relevance in an ongoing conversation or task. For AI agents, especially those built on Large Language Models (LLMs), short-term memory is primarily managed by feeding previous turns of a conversation back into the model as part of the current prompt. This creates a conversational history that the LLM can refer to. Without this mechanism, an LLM is stateless; each new prompt is treated as an independent request, leading to agents that forget prior statements, preferences, or context, making complex interactions impossible. Consider our Nutrition Agent. If a user asks, "What are the benefits of Vitamin D?", and then immediately asks, "What about Vitamin K?", without short-term memory, the agent wouldn't understand that "What about Vitamin K?" is implicitly asking about the benefits of Vitamin K. The agent would likely respond with a generic description of Vitamin K or ask for clarification. By including the previous turns in the prompt, the agent can infer the user's intent and provide a more helpful, context-aware response.
Implementing Short-Term Memory with Prompt History
The most common and straightforward way to implement short-term memory for LLM-based agents is by appending the history of previous user inputs and agent responses to the current prompt. This effectively extends the context window for the LLM, allowing it to "see" the conversation's trajectory. Let's look at a simplified conceptual example using a Python-like pseudo-code. Imagine a function that interacts with an LLM: # Pseudo-code for an LLM interaction without memory def interact_stateless(user_input): prompt = f"User: {user_input}\nAgent:" response = call_llm_api(prompt) return response # Example interaction: # User: "What are the benefits of Omega-3 fatty acids?" # Agent: "Omega-3s support heart health, brain function, and reduce inflammation." # User: "What about Vitamin C?" # Agent: "Vitamin C is an essential nutrient found in citrus fruits..." (Forgets the "benefits" context) Now, let's introduce a simple short-term memory mechanism: # Pseudo-code for an LLM interaction with short-term memory conversation_history = [] # Stores (role, message) tuples def interact_with_memory(user_input): global conversation_history # Add current user input to history conversation_history.append({"role": "user", "content": user_input}) # Construct the full prompt including history # For simplicity, we'll join messages. Real implementations use structured messages. full_prompt_messages = [] for turn in conversation_history: full_prompt_messages.append(f"{turn['role'].capitalize()}: {turn['content']}") # Add the agent's turn to complete the prompt structure for the LLM full_prompt_messages.append("Agent:") prompt_for_llm = "\n".join(full_prompt_messages) response_content = call_llm_api(prompt_for_llm) # Add agent's response to history conversation_history.append({"role": "agent", "content": response_content}) return response_content # Example interaction with memory: # User: "What are the benefits of Omega-3 fatty acids?" # Agent: "Omega-3s support heart health, brain function, and reduce inflammation." # User: "What about Vitamin C?" # Agent: "Vitamin C offers benefits such as immune system support, antioxidant properties..." (Understands "benefits" context) This approach has a direct limitation: the context window size of the LLM. Each token (word or sub-word unit) in the prompt consumes part of this window. As the conversation grows, the prompt becomes longer, eventually hitting the LLM's maximum token limit. When this happens, older parts of the conversation must be truncated or summarized to make space for new turns. Advanced memory management techniques involve strategies like summarizing past turns, using embedding-based retrieval for relevant historical snippets (a precursor to long-term memory), or employing specialized memory modules.
Key Takeaways
Short-term memory in AI agents refers to the ability to retain context from recent interactions within a single session. It's crucial for maintaining conversational coherence and allowing agents to understand follow-up questions. For LLM-based agents, short-term memory is typically implemented by including previous turns of the conversation within the current prompt. The primary limitation is the LLM's context window size, which dictates how much history can be passed at once. Without short-term memory, an LLM is stateless, treating each prompt as an isolated request. Practice Exercise: Imagine our Nutrition Agent is in a conversation with a user. The user first asks, "What nutrients are important for bone health?" The agent responds with a list including Calcium and Vitamin D. The user then asks, "And what foods are rich in the first one?" Describe how the agent, equipped with the simple short-term memory mechanism shown in the code example, would formulate its prompt to the LLM for this second question. Specifically, what would the conversation_history look like before the second prompt is constructed, and what would be the crucial contextual information the LLM would infer from it?
Watch the full lesson — free
This topic is part of AI Agents Crash Course: From Zero to Nutrition Agent, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →