12. How to add a conversation history summary

Keeping a conversation record persistence One of the most common use cases. This has the advantage of making conversation easier to sustain.

But the longer the conversation, the more the conversation record accumulates context window Will take more. This is LLM Calls are more expensive and longer, and may potentially be undesirable as errors may occur. One way to solve this is to generate a summary of the conversation to date, and recently N It is used with dog messages.

In this guide, we will look at examples of how to implement this.

The following steps are required.

• Make sure the conversation is too long (can be checked by number of messages or message length)

• Create a summary if it's too long (requires prompt for it)

• last N Delete the rest of the messages except the dog messages

An important part of this process is deleting old messages ( DeleteMessage ) Is doing.

Preferences

Copy

# Configuration file for managing API keys as environment variables
from dotenv import load_dotenv

# Load API key information
load_dotenv()

Copy

 True 

Copy

# Set up LangSmith tracking. https://smith.langchain.com
# !pip install -qU langchain-teddynote
from langchain_teddynote import logging

# Enter a project name.
logging.langsmith("CH17-LangGraph-Modules")

Copy

 Start tracking LangSmith. 
[Project name] 
CH17-LangGraph-Modules 

Summarize long conversations and save them as conversations

After creating a summary for a long conversation, delete the existing conversation and save the summary as a conversation.

Condition

• Generate a summary if the length of the conversation exceeds 6

Copy

from typing import Literal, Annotated
from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, RemoveMessage, HumanMessage
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import MessagesState, StateGraph, START
from langgraph.graph.message import add_messages

# Setting up memory storage
memory = MemorySaver()


# A status class that contains message status and summary information.
class State(MessagesState):
    messages: Annotated[list, add_messages]
    summary: str


# Initializing models for conversation and summarization
model = ChatOpenAI(model_name="gpt-4o-mini", temperature=0)

ask_llm Node messages Inject to llm to get an answer.

if, Summary of previous conversations If this exists, add it as a system message and include it in the conversation.

However, if the previous conversation summary does not exist, only the previous conversation content is used.

Copy

def ask_llm(state: State):
    # Check previous summary information
    summary = state.get("summary", "")

    # If there is previous summary information, add it as a system message.
    if summary:
        # Generate system messages
        system_message = f"Summary of conversation earlier: {summary}"
        # Combine system messages with previous messages
        messages = [SystemMessage(content=system_message)] + state["messages"]
    else:
        # Use only previous messages
        messages = state["messages"]

    # Calling the model
    response = model.invoke(messages)

    # Return response
    return {"messages": [response]}

should_continue Nodes go to the summary node if the length of the conversation exceeds 6.

If not, it returns an immediate answer. ( END Go to node)

Copy

from langgraph.graph import END


# Conversation End or Summary Decision Logic
def should_continue(state: State) -> Literal["summarize_conversation", END]:
    # Check message list
    messages = state["messages"]

    # If the number of messages exceeds 6, move to the summary node.
    if len(messages) > 6:
        return "summarize_conversation"
    return END

summarize_conversation The node summarizes the conversation and deletes the old message.

Copy

# Conversation summary and message organization logic
def summarize_conversation(state: State):
    # Check previous summary information
    summary = state.get("summary", "")

    # Generate summary message if previous summary information exists
    if summary:
        summary_message = (
            f"This is summary of the conversation to date: {summary}\n\n"
            "Extend the summary by taking into account the new messages above in Korean:"
        )
    else:
        # Generate a summary message
        summary_message = "Create a summary of the conversation above in Korean:"

    # Combine summary message with previous message
    messages = state["messages"] + [HumanMessage(content=summary_message)]
    # Calling the model
    response = model.invoke(messages)
    # Delete old messages
    delete_messages = [RemoveMessage(id=m.id) for m in state["messages"][:-2]]
    # Return summary information
    return {"summary": response.content, "messages": delete_messages}

Copy

# Initialize Workflow Graph
workflow = StateGraph(State)

# Adding Dialogue and Summary Nodes
workflow.add_node("conversation", ask_llm)
workflow.add_node(summarize_conversation)

# Set the starting point to a conversation node
workflow.add_edge(START, "conversation")

# Add conditional edge
workflow.add_conditional_edges(
    "conversation",
    should_continue,
)

# Add an edge from the summary node to the end node
workflow.add_edge("summarize_conversation", END)

# Compile workflow and set memory checkpoints
app = workflow.compile(checkpointer=memory)

Visualize the graph.

Copy

from langchain_teddynote.graphs import visualize_graph

visualize_graph(app)

Graph execution

Copy

# Function to output update information
def print_update(update):
    # Update dictionary iteration
    for k, v in update.items():
        # Print message list
        for m in v["messages"]:
            m.pretty_print()
        # Output when summary information exists
        if "summary" in v:
            print(v["summary"])

Copy

# Import HumanMessage class for message handling
from langchain_core.messages import HumanMessage

# Initialize a settings object containing a thread ID.
config = {"configurable": {"thread_id": "1"}}

# Create and print the first user message
input_message = HumanMessage(content="안녕하세요? 반갑습니다. 제 이름은 테디입니다.")
input_message.pretty_print()

# Processing the first message and outputting updates in stream mode
for event in app.stream({"messages": [input_message]}, config, stream_mode="updates"):
    print_update(event)

# Create and print a second user message
input_message = HumanMessage(content="Remember what my name is?")
input_message.pretty_print()

# Processing the second message in stream mode and outputting updates
for event in app.stream({"messages": [input_message]}, config, stream_mode="updates"):
    print_update(event)

# Create and print a third user message
input_message = HumanMessage(content="My job is an AI researcher")
input_message.pretty_print()

# Third message processing and update output in stream mode
for event in app.stream({"messages": [input_message]}, config, stream_mode="updates"):
    print_update(event)

Copy

 ================================ Human Message ================================= 

Hello? nice to meet you. My name is Teddy. 
================================== Ai Message ================================== 

Hello, Teddy! nice to meet you. How can I help you? 
================================ Human Message ================================= 

Do you remember what my name is? 
================================== Ai Message ================================== 

Yes, you said Teddy! How can I help you? 
================================ Human Message ================================= 

My job is an AI researcher. 
================================== Ai Message ================================== 

That's cool, Teddy! As an AI researcher, what are you primarily interested in? Or do you have a project that is currently underway? 

So far, you can see that the summary hasn't been done at all-this is because there are only 6 messages in the list.

Copy

# Retrieving state configuration values
values = app.get_state(config).values
values

Copy

 {'messages': [HumanMessage (content='Hello? nice to meet you. My name is Teddy.', additional_kwargs={}, response_metadata={}, id='358bb01e-98c5-41ea-a0ee-74aea38d9c98'), AIMessage (content='Hello, nice to meet you. '4p,'None-TAG1>'fusal': None={, response_metadata}'token_usage': ={'completion_tokens': 18,'prompt_tokens': 21,'total_  usage_metadata={'input_tokens': 21,'output_tokens': 18,'total_tokens': 39,'input_token_details': {'cache_read': 0},'output_token '1oks', response_metadata={'*pokens':'TAG1>'token_usage': }' completion_tokens': 17,'prompt_tokens': 56,'total_t  system_fingerprint':'fp_0ba0d124f1','finish_reason':'stop','logprobs': None}, id=-48d7-a869-3c44086042764ec84aee-aade-499e-91af-f025dec4de70'), AIMessage (content=' Nice, Teddy! As an AI researcher, what are you primarily interested in? Or do you have a project that is currently underway?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 38,'prompt_tokens': 90,'t  '1'G,'2fens': 0},'prompt_tokens_details': None,'cached_tokens':'  fp_0ba0d124f1','finish_reason':'stop','logprobs': None}, id='run-9ccc2a65-fec2-4a29-a613-70a10fa3a7a9-0', usage_metad  fp_0ba0d124f1','finish_reason':'stop','logprobs': None}, id='run-9ccc2a65-fec2-4a29-a613-70a10fa3a7a9-0', usage_metad 

Now I'll send another message

Copy

# Creating a user input message object
input_message = HumanMessage(
    content="I've been researching more about LLM lately. I'm reading recent papers on LLM."
)

# Print message content
input_message.pretty_print()

# Real-time processing and update output of stream events
for event in app.stream({"messages": [input_message]}, config, stream_mode="updates"):
    print_update(event)

Copy

 ================================ Human Message ================================= 

I'm learning more about LLM recently. I am reading a recent paper on LLM. 
================================== Ai Message ================================== 

Research on LLM (large language model) is a really interesting field! There have been many developments in recent years, and various papers are being published. Have any specific topics or questions? Or are you looking for a recommended paper or material? 
================================ Remove Message ================================ 


================================ Remove Message ================================ 


================================ Remove Message ================================ 


================================ Remove Message ================================ 


================================ Remove Message ================================ 


================================ Remove Message ================================ 


Dialogue summary: 

The user said, "Hello? nice to meet you. My name is Teddy."Say hello and introduce yourself. Subsequently, the user said that his job was an AI researcher, and that he is currently working on a LLM (large language model) and is reading a recent paper. Questions followed about AI interest and ongoing research. 

Check the current status and you will see the last two messages with a summary of the conversation.

Copy

# Retrieving state configuration values
values = app.get_state(config).values
values

Copy

 {'messages': [HumanMessage (content=' I'm learning more about LLM lately. I am reading a recent paper on LLM.', additional_kwargs={}, response_metadata={}, id='202b9b14-5a4d-41cc-a36c-d8f76ef73a25'), AIMessage (content= There have been many developments in recent years, and various papers are being published. Have any specific topics or questions? Or are you looking for a recommended paper or material?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 70,'prom_tokens':1  finish_reason':'stop','logprobs': None}, id='run-71abf2c7-4cad-44e9-8c00-589452f720b6-0', usage_metadata={'input_tokens' nice to meet you. My name is Teddy."Say hello and introduce yourself. Subsequently, the user said that his job was an AI researcher, and that he is currently working on a LLM (large language model) and is reading a recent paper. Questions followed about AI interest and ongoing research.'}  output_token_details': {'reasoning': 0}})],'summary':'Conversation summary:\n\n User said, "Hello? nice to meet you. My name is Teddy."Say hello and introduce yourself. Subsequently, the user said that his job was an AI researcher, and that he is currently working on a LLM (large language model) and is reading a recent paper. Questions followed about AI interest and ongoing research.'}  output_token_details': {'reasoning': 0}})],'summary':'Conversation summary:\n\n User said, "Hello? nice to meet you. My name is Teddy."Say hello and introduce yourself. Subsequently, the user said that his job was an AI researcher, and that he is currently working on a LLM (large language model) and is reading a recent paper. Questions followed about AI interest and ongoing research.'} 

Copy

messages = values["messages"]
messages

Copy

 [HumanMessage (content=' I'm learning more about LLM recently. I am reading a recent paper on LLM.', additional_kwargs={}, response_metadata={}, id='202b9b14-5a4d-41cc-a36c-d8f76ef73a25'), AIMessage (content= There have been many developments in recent years, and various papers are being published. Have any specific topics or questions? Or are you looking for a recommended paper or material?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 70,'prom_tokens':1  stop','logprobs': None}, id='run-71abf2c7-4cad-44e9-8c00-589452f720b6-0', usage_metadata={'input_tokens': 160,'output 

Now you can resume the conversation.

Even with the last two messages, you can ask about the previous conversation (because the previous one is summarized).

Copy

# Creating a user message object
input_message = HumanMessage(content="Remember what my name is?")

# Print message content
input_message.pretty_print()

# Real-time processing and updating of stream events
for event in app.stream({"messages": [input_message]}, config, stream_mode="updates"):
    print_update(event)

Copy

 ================================ Human Message ================================= 
Do you remember what my name is? 
================================== Ai Message ================================== 
Yes, your name is Teddy! 

Copy

# Creating a user message object
input_message = HumanMessage(content="Do you even remember my job?")

# Print message content
input_message.pretty_print()

# Real-time processing and update output of stream events
for event in app.stream({"messages": [input_message]}, config, stream_mode="updates"):
    print_update(event)

Copy

 ================================ Human Message ================================= 
Do you remember my job? 
================================== Ai Message ================================== 
Yes, you said you are an AI researcher. You said you were studying LLM.