10. Multi-agent for research introducing the STORM concept

The purpose of this tutorial is to cover how to build research automation systems using LangGraph.

Research is a labor-intensive task that is often delegated to analysts. AI has considerable potential to support this research process. This tutorial covers how to build custom AI-based research and report generation workflows.

In this tutorial, we aim to customize the research process by building a lightweight multi-agent system. Users provide research topics, and the system creates a team of AI analysts focusing on each subtopic.

In this process Human-in-the-loop Use to subdivide sub-topics before research begins.

STORM thesis According to, Similar topic lookup Wow Simulation of conversations from various perspectives This allows you to increase the frequency of use of reference sources and the density of information.

Mainly covered - LangGraph's main theme : Memory, Human-in-the-loop, Controllability - Goals of research automation : Build custom research processes - Source selection : Select input source for study - plan : Providing topics and creating a team of AI analysts - LLM utilization : In-depth interview with expert AI - Research course : Collect information and conduct interviews in parallel - Output format : Insight integrated into the final report - Settings : Preferences and API key settings - Analyst creation : Analyst creation and review through Human-In-The-Loop - Conduct an interview : Create questions and collect answers - Parallel interview : Parallelization of interviews through Map-Reduce - Final report preparation : Introduction and conclusion of the report This tutorial covers three themes:

• Memory

• Human-in-the-loop

• Controllability

Now we will combine these concepts to cover research automation, one of AI's most popular applications.

Research is a labor-intensive task that is often delegated to analysts. AI has considerable potential to support this research process. However, research needs customization. Raw LLM outputs are often not suitable for real decision-making walkflows.

Custom AI base Research and report generation Workflows are a promising way to solve this.

Copy

 True 

Preferences

Copy

# Configuration file for managing API keys as environment variables
from dotenv import load_dotenv

# Load API key information
load_dotenv()

Copy

 True 

Copy

# Set up LangSmith tracking. https://smith.langchain.com
# !pip install -qU langchain-teddynote
from langchain_teddynote import logging

# Enter a project name.
logging.langsmith("CH17-LangGraph-Use-Cases")

Copy

 Start tracking LangSmith. 
[Project name] 
CH17-LangGraph-Use-Cases 

Copy

from langchain_teddynote.models import get_model_name, LLMs
from langchain_openai import ChatOpenAI

# Get the latest models
GPT4o = get_model_name(LLMs.GPT4o)

# Initialize model
llm = ChatOpenAI(model=GPT4o)

Analyst Generation: Human-In-The-Loop

• Analyst creation : Human-In-The-Loop Utilize to create and review analysts.

Copy

from typing import List
from typing_extensions import TypedDict
from pydantic import BaseModel, Field
from langchain_teddynote.graphs import visualize_graph


# A class that defines the analyst's properties and metadata.
class Analyst(BaseModel):
    # Key Affiliation Information
    affiliation: str = Field(
        description="Primary affiliation of the analyst.",
    )
    # name
    name: str = Field(description="Name of the analyst.")

    # role
    role: str = Field(
        description="Role of the analyst in the context of the topic.",
    )
    # Description of focus, concerns and motivations
    description: str = Field(
        description="Description of the analyst focus, concerns, and motives.",
    )

    # A property that returns the analyst's personal information as a string.
    @property
    def persona(self) -> str:
        return f"Name: {self.name}\nRole: {self.role}\nAffiliation: {self.affiliation}\nDescription: {self.description}\n"


# A collection of analysts
class Perspectives(BaseModel):
    # List of Analysts
    analysts: List[Analyst] = Field(
        description="Comprehensive list of analysts with their roles and affiliations.",
    )

The following defines the status of tracking a set of analysts created through the Analyst class.

Copy

Copy# state definition
class GenerateAnalystsState(TypedDict):
    # research topic
    topic: str
    # Maximum number of analysts to create
    max_analysts: int
    # people feedback
    human_analyst_feedback: str
    # List of Analysts
    analysts: List[Analyst]

Analyst creation node definition

Next, we will define the Analyst creation node.

The code below implements a logic that creates various analysts for a given research topic. Each analyst has a unique role and affiliation, and provides a professional perspective on the subject.

Copy

from langgraph.graph import END
from langchain_core.messages import HumanMessage, SystemMessage

# Analyst Creation Prompt
analyst_instructions = """You are tasked with creating a set of AI analyst personas. 

Follow these instructions carefully:
1. First, review the research topic:

{topic}

2. Examine any editorial feedback that has been optionally provided to guide creation of the analysts: 

{human_analyst_feedback}

3. Determine the most interesting themes based upon documents and / or feedback above.

4. Pick the top {max_analysts} themes.

5. Assign one analyst to each theme."""


# Analyst Creation Node
def create_analysts(state: GenerateAnalystsState):
    """Function to create analyst personas"""

    topic = state["topic"]
    max_analysts = state["max_analysts"]
    human_analyst_feedback = state.get("human_analyst_feedback", "")

    # Apply structured output format to LLM
    structured_llm = llm.with_structured_output(Perspectives)

    # Configuring System Prompts for Analyst Creation
    system_message = analyst_instructions.format(
        topic=topic,
        human_analyst_feedback=human_analyst_feedback,
        max_analysts=max_analysts,
    )

    # Create an analyst persona by calling LLM
    analysts = structured_llm.invoke(
        [SystemMessage(content=system_message)]
        + [HumanMessage(content="Generate the set of analysts.")]
    )

    # Save the generated analyst list to status
    return {"analysts": analysts.analysts}


# User Feedback Node (You can leave the content blank as we will be doing status updates)
def human_feedback(state: GenerateAnalystsState):
    """Breakpoint nodes to get user feedback"""
    pass


# A function that determines the next step in a workflow based on whether or not there is human feedback.
def should_continue(state: GenerateAnalystsState):
    """A function that determines the next step in the workflow."""

    human_analyst_feedback = state.get("human_analyst_feedback", None)
    if human_analyst_feedback:
        return "create_analysts"

    return END

Graph generation

Now create an analyst creation graph.

Copy

from langgraph.graph import START, END, StateGraph
from langgraph.checkpoint.memory import MemorySaver
from langchain_core.messages import SystemMessage

# create a graph
builder = StateGraph(GenerateAnalystsState)

# add note
builder.add_node("create_analysts", create_analysts)
builder.add_node("human_feedback", human_feedback)

# edge connection
builder.add_edge(START, "create_analysts")
builder.add_edge("create_analysts", "human_feedback")

# Add conditional edge: If there is human feedback, it will return back to the analyst creation node.
builder.add_conditional_edges(
    "human_feedback", should_continue, ["create_analysts", END]
)

# create memory
memory = MemorySaver()

# Compile graph (set breakpoints)
graph = builder.compile(interrupt_before=["human_feedback"], checkpointer=memory)

# Graph visualization
visualize_graph(graph)

Graph execution for analyst creation

Copy

from langchain_core.runnables import RunnableConfig
from langchain_teddynote.messages import random_uuid, invoke_graph

config = RunnableConfig(
    recursion_limit=10,
    configurable={"thread_id": random_uuid()},
)

# Set the number of analysts
max_analysts = 3

# Set input data
topic = "What is the difference between Modular RAG and the existing Naive RAG? production level 에서 사용하는 이점"

# Setting input data
inputs = {
    "topic": topic,
    "max_analysts": max_analysts,
}

# running the graph
invoke_graph(graph, inputs, config)

Copy

================================================== 
🔄 Node: create_analysts 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
affiliation='AI Research Institute' name='Dr. Emily Chen' role='Research Scientist' description='Dr. Chen focuses on the technical distinctions between Modular RAG and Naive RAG, analyzing how modularity in retrieval-augmented generation systems enhances flexibility and efficiency. Her primary concern is to understand the architectural advancements and their implications on computational performance.' 
affiliation='Tech Industry Solutions' name='Michael Johnson' roule='Industry Analyst' description='Michael evaluates the practical benefits of Modular RAG in production environments. He is particularly interested in identifying how modular approaches can lead to improved scalability and adaptability in real-world applications, compared to traditional methods.' 
affiliation='Data Security and Privacy Association' name='Fatima Al-Mansoori' role='Data Privacy Consultant' description='Fatima examines the implications of using Modular RAG with respect to data privacy and security. Her focus is on understanding how modular systems impact data handling practices and what advantages they night offer in protecting sensitive information in production settings.' 
================================================== 

================================================== 
🔄 Node: __interrupt__ 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
================================================== 

__interrupt__ When outputted, it is ready to receive human feedback.

Now bring the status below to provide human feedback.

Copy

# get the current state of the graph
state = graph.get_state(config)

# check the next node to run
print(state.next)

Copy

('human_feedback',)

update_state() Inject human feedback through. At this time human_analyst_feedback Save feedback on the key.

Also as_node Specifies the node to receive feedback through the factor.

Copy

# Perform the role of the human_feedback node by updating the graph state
graph.update_state(
    config,
    {
        "human_analyst_feedback": "Add in someone named Teddy Lee from a startup to add an entrepreneur perspective"
    },
    as_node="human_feedback",
)

Copy

{'configurable': {'thread_id': '05e30b80-5231-443d-945e-1733fdca46be','checkpoint_ns':'','checkpoint_id': '1efc0827-6ac3- 

None When the value is given as input, then the graph proceeds.

Reference

• When you want to resume, None Resume the graph by assigning values.

Copy

Copy# continue to proceed
invoke_graph(graph, None, config)

Copy

================================================== 
🔄 Node: create_analysts 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
affiliation='Tech Research Institute' name='Dr. Emily Chang' role='RAG Technology Specialist' description='Dr. Chang focuses on the technological differences and advancements between Modular RAG and Naive RAG. She is particularly interested in the architectural improvements and how these impact performance and scalability.' 
affiliation='Global Tech Consultancy' name='Raj Patel' role='Production Implementation Expert' description='Raj Patel specializes in the deployment and operational integration of advanced AI systems in production environments. His focus is on the practical benefits and challenges of understanding Modular RAG in real-world applications.' 
affiliation='Innovative AI Startup' name='Teddy Lee' role='Entrepreneur & Innovator' description='Teddy Lee brings an entrepreneurial perspective, evaluating the business advantages and market He focuses on how startups can range these technologies for competitive assistantage and innovation.' 
================================================== 

================================================== 
🔄 Node: __interrupt__ 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
================================================== 

again __interrupt__ When outputted, it is ready to receive human feedback.

It is also possible to reconcile the analyst's persona created by providing human feedback again, the same as the previous method.

But if there is no additional feedback None You can end the analyst creation task by assigning values.

Copy

# Update status by assigning None value if no additional feedback is available.
human_feedback_input = None

# Perform the role of the human_feedback node by updating the graph state
graph.update_state(
    config, {"human_analyst_feedback": human_feedback_input}, as_node="human_feedback"
)

Copy

 {'configurable': {'thread_id': '05e30b80-5231-443d-945e-1733fdca46be','checkpoint_ns':'','checkpoint_id': '1efc0827-8158 

Copy

# continue to proceed
invoke_graph(graph, None, config)

Outputs the final result.

Copy

Copy# get the final state of the graph
final_state = graph.get_state(config)

# get the list of analysts generated in the final state
analysts = final_state.values.get("analysts")

# output the number of generated analysts
print(f"number of analysts created: {len(analysts)}", end="\n================================\n")

# output persona for each analysts
for analyst in analysts:
    print(analyst.persona)
    print("- " * 30)

Copy

Number of analysts generated: 3 
================================ 
Name: Dr. Emily Chang 
Role: RAG Technology Specialist 
Affiliation: Tech Research Institute 
Description: Dr. Chang focuses on the technological differences and advancements between Modular RAG and Naive RAG. She is particularly interested in the architectural improvements and how theye impact performance and scalability. 

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
Name: Raj Patel 
Role: Production Implementation Expert 
Affiliation: Global Tech Consultancy 
Description: Raj Patel specializes in the deployment and operational integration of advanced AI systems in production environments. His focus is on the practical benefits and challenges of understanding Modular RAG in real-world applications. 

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  
Name: Teddy Lee 
Role: Entrepreneur & Innovator 
Affiliation: Innovative AI Startup 
Description: Teddy Lee brings an entrepreneurial perspective, evaluating the business advantages and market potential of Modular RAG over Naive RAG. He focuses on how startups can range these technologies for competitive assistantage and innovation. 

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  

final_state.next Indicates the node to run next in the graph. Here's all the work done, so empty tuple This output.

Copy

# Get the state of the next node to be executed in the graph
print(final_state.next)

Copy

 ()

Conduct an interview

Create question

• Analysts ask questions to experts.

Copy

import operator
from typing import Annotated
from langgraph.graph import MessagesState


# defining intervie status
class InterviewState(MessagesState):
    # number of conversation turns
    max_num_turns: int
    # a list of contects containing source documents
    context: Annotated[list, operator.add]
    # designated analysts
    analyst: Analyst
    # a string that stores the interview content
    interview: str
    # list of report sections
    sections: list


# defining a search query data class
class SearchQuery(BaseModel):
    search_query: str = Field(None, description="Search query for retrieval.")

The following defines the node that creates the interview question.

Copy

Copyquestion_instructions = """You are an analyst tasked with interviewing an expert to learn about a specific topic. 

Your goal is boil down to interesting and specific insights related to your topic.

1. Interesting: Insights that people will find surprising or non-obvious.

2. Specific: Insights that avoid generalities and include specific examples from the expert.

Here is your topic of focus and set of goals: {goals}

Begin by introducing yourself using a name that fits your persona, and then ask your question.

Continue to ask questions to drill down and refine your understanding of the topic.

When you are satisfied with your understanding, complete the interview with: "Thank you so much for your help!"

Remember to stay in character throughout your response, reflecting the persona and goals provided to you."""


# defining a node that generates a question
def generate_question(state: InterviewState):
    # get messages from analysts in status
    analyst = state["analyst"]
    messages = state["messages"]

    # create a question
    # generate system messages based on analysts goals
    system_message = question_instructions.format(goals=analyst.persona)
    # generating question using LLM
    question = llm.invoke([SystemMessage(content=system_message)] + messages)

    # Log message to status
    return {"messages": [question]}

Tool definition

Experts answer questions by collecting information in parallel from multiple sources.

Various tools are available, including web document scraping, VectorDB, web browsing, and Wikipedia search.

This tutorial uses Arxiv, Tavily search.

Copy

# reset web search toolw
from langchain_teddynote.tools.tavily import TavilySearch

# creating a tavily search instance for eb search
tavily_search = TavilySearch(max_results=3)

Copy

from langchain_community.retrievers import ArxivRetriever

# Creating an ArxivRetriever instance for Arxiv search
arxiv_retriever = ArxivRetriever(
    load_max_docs=3,
    load_all_available_meta=True,
    get_full_documents=True,
)

# output search results
arxiv_search_results = arxiv_retriever.invoke("Modular RAG vs Naive RAG")
print(arxiv_search_results)

Copy

 [Document (metadata={'Published': '2024-07-26','Title':'Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks','Authors':'Yunfan Gao, Yun Xiong, Meng Wangen, Haof The\nincreasing demands of application scenarios have driven the evolution of RAG,\nleading to the  

(...sway...) 

existing work, employ enabled\nRAG components to construct their own RAG processes, or simply use organized datasets and corpora\nto accelerate their own RAG workflow. Compared to existing RAG toolkits, FlashRAG is more suited\nfor researchers. To summarize, the key features and capabilities of our FlashRAG library can be\noutlined in the following four aspects:\nExtensive and Customizable Modular RAG Framework.\nTo facilitate an easily expandable\nRAG process, we simple At the component level, we offer\ncomprehensive RAG compon')] 

Copy

# AArxiv metadata output
arxiv_search_results[0].metadata

Copy

 {'Published': '2024-07-26','Title':'Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks','Authors':'Yunfan Gao, Yun Xiong, Meng Wang, Haofen Wang','Summ The\nincreasing demands of application  

(...sway...) 

'4:'Clin','Critical','CIR','Clinone','Published_first_time' 

Copy

# Output Arxiv content
print(arxiv_search_results[0].page_content)

Copy

 One 
Modular RAG: Transforming RAG Systems into 
LEGO-like Reconfigurable Frameworks 
Yunfan Gao, Yun Xiong, Meng Wang, Haofen Wang 
Abstract—Retrieval-augmented 
Generation 
(RAG) 
has 

(...sway...) 

dundancy and Noise. Feeding all retrieved chunks directly 
into LLMs is not always negative. Research indicates that 
an experience of redundant and noisy information may interfere 
with the LLM's identification of key information, thereby 
increasing the risk of generating erroneous and hallucinated 
responses. [9] 
To overcome the aforementioned limitations,

Format and output document search results.

Copy

# formatting document search results
formatted_search_docs = "\n\n---\n\n".join(
    [
        f'<Document source="{doc.metadata["entry_id"]}" date="{doc.metadata.get("Published", "")}" authors="{doc.metadata.get("Authors", "")}"/>\n<Title>\n{doc.metadata["Title"]}\n</Title>\n\n<Summary>\n{doc.metadata["Summary"]}\n</Summary>\n\n<Content>\n{doc.page_content}\n</Content>\n</Document>'
        for doc in arxiv_search_results
    ]
)

Copy

print(formatted_search_docs)

Copy

Retrieval-augmented Generation (RAG) has markedly enhanced the capabilities 
of Large Language Models (LLMs) in tackling knowledge-intensive tasks. The 
increasing demands of application scenarios have driven the evolution of RAG, 
reading to the integration of advanced retrievers, LLMs and other complementary 
technologies, whatch in turn has amplified the intricacy of RAG systems. 
However, the rapid advancements are outpacing the foundational RAG paradigm, 
with many methods struggling to be unified under the process of 
"retrieve-then-generate". In this context, this paper examines the limitations 
of the existing RAG paradigm and introduces the modern RAG framework. By 
decomposing complex RAG systems into independent modules and specialized 
operators, it facilitates a highly reconfigurable framework. Modular RAG 
transcends the traditional linear architecture, embracing a more advanced 
design that integrates routing, scheduling, and fusion mechanics. Drawing on 
extensive research, this paper further identifies prevalent RAG 
patterns-linear, conditional, branching, and looping-and offers a comprehensive 
analysis of their respective implementation nuances. Modular RAG presidents 
innovative opportunities for the conceptualization and deployment of RAG 
systems. Finally, the paper explores the potential emergence of new operators 
and paradigms, establishing a solid theoretical foundation and a practical 
roadmap for the continued evolution and practical deployment of RAG 
technologies. 


(...Hushi...) 

Node generation

Copy

from langchain_core.messages import get_buffer_string
from langchain_core.messages import SystemMessage, HumanMessage, AIMessage

# Writing a search query
search_instructions = SystemMessage(
    content=f"""You will be given a conversation between an analyst and an expert. 

Your goal is to generate a well-structured query for use in retrieval and / or web-search related to the conversation.

First, analyze the full conversation.

Pay particular attention to the final question posed by the analyst.

Convert this final question into a well-structured web search query"""
)


# Defining a function to perform a web search
def search_web(state: InterviewState):
    """Finding documents through web search"""

    # Create a search query
    structured_llm = llm.with_structured_output(SearchQuery)
    search_query = structured_llm.invoke([search_instructions] + state["messages"])

    # Perform a search
    search_docs = tavily_search.invoke(search_query.search_query)

    # Format your search results
    formatted_search_docs = "\n\n---\n\n".join(
        [
            f'<Document href="{doc["url"]}"/>\n{doc["content"]}\n</Document>'
            for doc in search_docs
        ]
    )

    return {"context": [formatted_search_docs]}


# Creating an Arxiv search node
def search_arxiv(state: InterviewState):
    """Arxiv 검색 노드"""

    # Create a search query
    structured_llm = llm.with_structured_output(SearchQuery)
    search_query = structured_llm.invoke([search_instructions] + state["messages"])

    try:
        # perform a search
        arxiv_search_results = arxiv_retriever.invoke(
            search_query.search_query,
            load_max_docs=2,
            load_all_available_meta=True,
            get_full_documents=True,
        )

        # format your search results
        formatted_search_docs = "\n\n---\n\n".join(
            [
                f'<Document source="{doc.metadata["entry_id"]}" date="{doc.metadata.get("Published", "")}" authors="{doc.metadata.get("Authors", "")}"/>\n<Title>\n{doc.metadata["Title"]}\n</Title>\n\n<Summary>\n{doc.metadata["Summary"]}\n</Summary>\n\n<Content>\n{doc.page_content}\n</Content>\n</Document>'
                for doc in arxiv_search_results
            ]
        )

        return {"context": [formatted_search_docs]}
    except Exception as e:
        print(f"Error occurred while searching Arxiv: {str(e)}")
        return {
            "context": ["<Error>Failed to retrieve Arxiv search results.</Error>"]
        }


answer_instructions = """You are an expert being interviewed by an analyst.

Here is analyst area of focus: {goals}. 

You goal is to answer a question posed by the interviewer.

To answer question, use this context:

{context}

When answering questions, follow these guidelines:

1. Use only the information provided in the context. 

2. Do not introduce external information or make assumptions beyond what is explicitly stated in the context.

3. The context contain sources at the topic of each individual document.

4. Include these sources your answer next to any relevant statements. For example, for source # 1 use [1]. 

5. List your sources in order at the bottom of your answer. [1] Source 1, [2] Source 2, etc

6. If the source is: <Document source="assistant/docs/llama3_1.pdf" page="7"/>' then just list: 

[1] assistant/docs/llama3_1.pdf, page 7 

And skip the addition of the brackets as well as the Document source preamble in your citation."""


# Define a function to generate answers to questions
def generate_answer(state: InterviewState):
    """Generate answer to question node"""

    # Get messages from analysts in status
    analyst = state["analyst"]
    messages = state["messages"]
    context = state["context"]

    # Generate answers to questions
    system_message = answer_instructions.format(goals=analyst.persona, context=context)
    answer = llm.invoke([SystemMessage(content=system_message)] + messages)

    # Name the message as an expert response
    answer.name = "expert"

    # Add a message to the status
    return {"messages": [answer]}


# Defining the interview save function
def save_interview(state: InterviewState):
    """save interview"""

    # get message
    messages = state["messages"]

    # Convert interview to string
    interview = get_buffer_string(messages)

    # Save to interview key
    return {"interview": interview}


# Defining a message routing function
def route_messages(state: InterviewState, name: str = "expert"):
    """Routing between questions and answers"""

    # get message
    messages = state["messages"]
    max_num_turns = state.get("max_num_turns", 2)

    # check the number of expert answers
    num_responses = len(
        [m for m in messages if isinstance(m, AIMessage) and m.name == name]
    )

    # Ends when the expert answers more than the maximum number of turns
    if num_responses >= max_num_turns:
        return "save_interview"

    # This router runs after each question-answer pair.
    # Bring the last question to signal the end of discussion
    last_question = messages[-2]

    if "Thank you so much for your help" in last_question.content:
        return "save_interview"
    return "ask_question"


# Session Creation Instructions
section_writer_instructions = """You are an expert technical writer. 

Your task is to create a detailed and comprehensive section of a report, thoroughly analyzing a set of source documents.
This involves extracting key insights, elaborating on relevant points, and providing in-depth explanations to ensure clarity and understanding. Your writing should include necessary context, supporting evidence, and examples to enhance the reader's comprehension. Maintain a logical and well-organized structure, ensuring that all critical aspects are covered in detail and presented in a professional tone.

Please follow these instructions:
1. Analyze the content of the source documents: 
- The name of each source document is at the start of the document, with the <Document tag.

2. Create a report structure using markdown formatting:
- Use ## for the section title
- Use ### for sub-section headers

3. Write the report following this structure:
a. Title (## header)
b. Summary (### header)
c. Comprehensive analysis (### header)
d. Sources (### header)

4. Make your title engaging based upon the focus area of the analyst: 
{focus}

5. For the summary section:
- Set up summary with general background / context related to the focus area of the analyst
- Emphasize what is novel, interesting, or surprising about insights gathered from the interview
- Create a numbered list of source documents, as you use them
- Do not mention the names of interviewers or experts
- Aim for approximately 400 words maximum
- Use numbered sources in your report (e.g., [1], [2]) based on information from source documents

6. For the Comprehensive analysis section:
- Provide a detailed examination of the information from the source documents.
- Break down complex ideas into digestible segments, ensuring a logical flow of ideas.
- Use sub-sections where necessary to cover multiple perspectives or dimensions of the analysis.
- Support your analysis with data, direct quotes, and examples from the source documents.
- Clearly explain the relevance of each point to the overall focus of the report.
- Use bullet points or numbered lists for clarity when presenting multiple related ideas.
- Ensure the tone remains professional and objective, avoiding bias or unsupported opinions.
- Aim for at least 800 words to ensure the analysis is thorough.

7. In the Sources section:
- Include all sources used in your report
- Provide full links to relevant websites or specific document paths
- Separate each source by a newline. Use two spaces at the end of each line to create a newline in Markdown.
- It will look like:

### Sources
[1] Link or Document name
[2] Link or Document name

8. Be sure to combine sources. For example this is not correct:

[3] https://ai.meta.com/blog/meta-llama-3-1/
[4] https://ai.meta.com/blog/meta-llama-3-1/

There should be no redundant sources. It should simply be:

[3] https://ai.meta.com/blog/meta-llama-3-1/

9. Final review:
- Ensure the report follows the required structure
- Include no preamble before the title of the report
- Check that all guidelines have been followed"""


# Defining a section creation function
def write_section(state: InterviewState):
    """Generate answer to question node"""

    # Get context, analyst from state
    context = state["context"]
    analyst = state["analyst"]

    # Defining system prompts for section creation
    system_message = section_writer_instructions.format(focus=analyst.description)
    section = llm.invoke(
        [SystemMessage(content=system_message)]
        + [HumanMessage(content=f"Use this source to write your section: {context}")]
    )

    # add section to status
    return {"sections": [section.content]}

Create an interview graph

Define and run the graphs that conduct the interview.

Copy

from langgraph.graph import StateGraph
from langgraph.checkpoint.memory import MemorySaver

# Add nodes and edges
interview_builder = StateGraph(InterviewState)
interview_builder.add_node("ask_question", generate_question)
interview_builder.add_node("search_web", search_web)
interview_builder.add_node("search_arxiv", search_arxiv)
interview_builder.add_node("answer_question", generate_answer)
interview_builder.add_node("save_interview", save_interview)
interview_builder.add_node("write_section", write_section)

# Flow settings
interview_builder.add_edge(START, "ask_question")
interview_builder.add_edge("ask_question", "search_web")
interview_builder.add_edge("ask_question", "search_arxiv")
interview_builder.add_edge("search_web", "answer_question")
interview_builder.add_edge("search_arxiv", "answer_question")
interview_builder.add_conditional_edges(
    "answer_question", route_messages, ["ask_question", "save_interview"]
)
interview_builder.add_edge("save_interview", "write_section")
interview_builder.add_edge("write_section", END)

# Create an interview graph
memory = MemorySaver()
interview_graph = interview_builder.compile(checkpointer=memory).with_config(
    run_name="Conduct Interviews"
)

# graph visualization
visualize_graph(interview_graph)

Copy

# Select the first analyst from the list of analysts
analysts[0]

Copy

 Analyst (affiliation='Tech Research Institute', name='Dr. Emily Chang', role='RAG Technology Specialist', description='Dr. Chang focuses on the technological differences and advancements between Modular RAG and Naive RAG. She is particularly interested in the architectural improvements and how these impact performance and scalability.') 

Graph execution

Now run the graph and output the results.

Copy

from IPython.display import Markdown

# setting the topic
topic = "What is the difference between Modular RAG and the existing Naive RAG? production level 에서 사용하는 이점"

# 인터뷰 시작 메시지 생성
messages = [HumanMessage(f"So you said you were writing an article on {topic}?")]

# 스레드 ID 설정
config = RunnableConfig(
    recursion_limit=100,
    configurable={"thread_id": random_uuid()},
)

# 그래프 실행
invoke_graph(
    interview_graph,
    {"analyst": analysts[0], "messages": messages, "max_num_turns": 5},
    config,
)

Copy

================================================== 
🔄 Node: ask_question 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  
================================== Ai Message ================================== 

Hello, I'm Alex Thompson, a technology analyst. I'm very eager to learn more about the differences between Modular RAG and Naive RAG, particularly in terms of their architectural advancements and how theyse influence performance and scalability.  

Dr. Chang, could you start by explaining what fundamentally distinguishes Modular RAG from Naive RAG? 
================================================== 

================================================== 
🔄 Node: search_web 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  

The RAG research paradigm is continuously evolving, and RAG is categorized into three stages: Naive RAG, Advanced RAG, and Modular RAG. Naive RAG has several limits, including Retrieval Challenges and Generation Difficulties. The latter RAG architectures were proposed to address these problems: Advanced RAG and Modular RAG. Due to the 


--- --- 


RAG is a strategic enhancement designed to elevate the performance of LLMs. By incorporating a step that retrieves information during text generation, RAG ensures that the model's responses are accurate and up-to-date. Naive RAG: This is the most basic version, where the system simple retrieves the relevant information from a knowledge base and direct grants it to the LLM to generale the response. By undertanding the flow of data from the user's query to the final response, we can appreciate how each component of the Naive RAG system plays a pivotal roule in ensuring that the user receives accurate and relevant information. Unlike Naive RAG, which direct incorporates retrieved information, Advanced RAG involves additional processing steps that option the response, and overall quality of the response. 


--- --- 


See the comparison between all three paradigms of RAG — Naive RAG, Advanced RAG and Modular RAG below. Comparison between the three paradigms of RAG (Gao et al. 2024) 1️ ⁇  Naive RAG 

================================================== 

================================================== 
🔄 Node: search_arxiv 🔄 
- - - - - - - - - - - - - - - - - - - - - - - - - - - -  




Retrieval-augmented generation (RAG) utilizes retrieved texts to enhance 
large language models (LLMs). Studies show that while RAG provides valuable 
external information (benefit), it may also mislead LLMs (detriment) with noisy 
or incorrect retrieved texts. Although many existing methods attempt to 
preserve benefit and avoid detriment, they lac a theoretical exploitation for 
RAG. The benefit and detriment in the next token prediction of RAG remain a 
black box that cannot be quantified or compared in an explainable manner, so 
existing methods are data-driven, need additional utility evaluators or 
post-hoc. This paper takes the first step towards provisioning a theory to explain 
and trade off the benefit and detriment in RAG. First, we model RAG as the 
fusion between distribution of LLMs knowledge and distribution of retrieved 
texts. Then, we formalize the trade-off between the value of external knowledge 
(benefit) and its potential risk of misleading LLMs (detriment) in next token 
prediction of RAG by distribution difference in this fusion. Finally, we probe 
that the actual effect of RAG on the token, which is the comparison between 
benefit and detriment, can be predicted without any training or accessing the 
utility of retrieval. Based on our theory, we propos a professional novel method, 
Tok-RAG, whatch achieves collaborative generation between the pure LLM and RAG 
at token level to preserve benefit and avoid detriment. Experiments in 
real-world tasks using LLMs such as OPT, LLaMA-2, and Mistral show the 
effectiveness of our method and support our theoretical findings. 



A THEORY FOR TOKEN-LEVEL HARMONIZATION IN 
RETRIEVAL-AUGMENTED GENERATION 
Shicheng Xu 
Liang Pang∗Huawei Shen 
Xueqi Cheng 
CAS Key Laboratory of AI Safety, Institute of Computing Technology, CAS 
{xushicheng21s,pangliang,shenhuawei,cxq}@ict.ac.cn 
ABSTRACT 
Retrieval-augmented generation (RAG) utilizes retrieved texts to enhance large 
language models (LLMs). Studies show that while RAG provides valuable external 

(...sway...) 

-**Enhanced Accuracy**: By reducing retrieval noise and focusing on relevant information through sophisticated retrieval mechanisms, Modular RAG enhances the accuracy of generalated responses. This is crucial in applications where precision and relative area paramount [3]. 
-**Increased Robustness**: The modular framework's ability to dynamically adjust to varying conditions and integrate new data sources contributes to its robustness, making it it suite for a wide range of applications [2]. 
-**Future Directions**: The modern approx opens avenues for future research and development, particularly in the creation of new operators and paradigms that can further enhance the capabilities of RAG systems [1][2]. 

### Sources 

[1] http://arxiv.org/abs/2406.00944v2  
[2] http://arxiv.org/abs/2407.21059v1  
[3] http://arxiv.org/abs/2401.12599v1  
[4] http://arxiv.org/abs/2410.07176v1  
================================================== 

Copy

Copyimport operator

• Interview with langgraph Send() Parallelized using functions, which map Corresponds to the step.

• Interview results reduce It is incorporated into the body of the report at the stage.

• Add an introduction and a final step in writing an introduction to the final report.

Interview in parallel (map-reduce)

Copy

## Bridging the Gap: Advancements from Naive RAG to Modular RAG

### Summary

Retrieval-Augmented Generation (RAG) systems have significantly enhanced the capabilities of Large Language Models (LLMs), enabling them to efficiently tackle knowledge-intensive tasks by accessing external knowledge bases. This has led to a substantial improvement in performance across various applications, including knowledge question answering, recommendation systems, and customer service [1]. However, as the field evolves, the foundational Naive RAG paradigm, which relies on a basic retrieve-then-generate process, is being outpaced by the increasing complexity of application scenarios and the need for more sophisticated systems [1].

The introduction of Modular RAG frameworks marks a novel advancement in the RAG landscape. Unlike Naive RAG, which suffers from limitations such as shallow understanding of queries, retrieval redundancy, and noise, Modular RAG decomposes complex systems into independent modules and specialized operators. This transformation allows for a highly reconfigurable framework that transcends traditional linear architectures by integrating routing, scheduling, and fusion mechanisms, thus facilitating better performance and scalability [1].

The benefits of Modular RAG are further supported by theoretical models that explain the interplay between the benefits and detriments of RAG, highlighting the importance of balancing the external knowledge integrated into LLMs [2]. Additionally, Modular RAG's architecture allows it to outperform Naive RAG across several key metrics, making it an effective solution for enterprise applications, including customer support chatbots [3]. This report draws insights from multiple sources to paint a comprehensive picture of these advancements:

1. [1] http://arxiv.org/abs/2407.21059v1
2. [2] http://arxiv.org/abs/2406.00944v2
3. [3] https://adasci.org/how-does-modular-rag-improve-upon-naive-rag/

### Comprehensive Analysis

#### Modular RAG: A Reconfigurable Framework

The Modular RAG framework represents a significant shift in how RAG systems are conceptualized and deployed. By breaking down the system into modular components, each with specialized operators, Modular RAG allows for a flexible and adaptable architecture. This approach not only addresses the limitations of the Naive RAG model but also introduces advanced mechanisms such as routing, scheduling, and fusion. These features enable the system to handle complex queries more effectively, improving the overall performance and scalability of RAG applications [1].

- **Modular Design**: The decomposition of RAG systems into modules allows each component to be optimized individually. This modularity supports the integration of cutting-edge technologies and methodologies, which can be tailored to specific tasks or application scenarios.
- **Enhanced Query Understanding**: By moving beyond simple semantic similarity calculations, Modular RAG facilitates a deeper exploration of the relationships between queries and documents, reducing the noise and redundancy issues prevalent in Naive RAG [1].

#### Theoretical Insights and Practical Implications

The evolution from Naive to Modular RAG is underpinned by theoretical advancements that provide a framework for understanding the trade-offs involved in RAG systems. The theory of token-level harmonization in RAG, for instance, models the fusion of LLM knowledge with retrieved texts to balance the benefits and potential detriments of integrating external information [2].

- **Benefit vs. Detriment**: The theory formalizes the trade-off between the value of external knowledge (benefit) and the risk of misleading LLMs (detriment). This framework helps predict the impact of RAG on token-level predictions, aiding in the development of methods like Tok-RAG that enhance LLM performance by preserving benefits while avoiding detriments [2].
- **Practical Applications**: In real-world tasks, experiments with methods such as Tok-RAG have demonstrated improvements in performance, supporting the theoretical findings and showcasing the practical applicability of these advancements [2].

#### Performance and Scalability

Modular RAG's architecture offers substantial improvements over Naive RAG, particularly in terms of performance, accuracy, and flexibility. This makes it an ideal solution for a wide range of applications, from knowledge-intensive customer support systems to enterprise-scale deployments [3].

- **Key Metrics**: Modular RAG outperforms Naive RAG across various metrics, including relevance, response time, scalability, and customer satisfaction. The modular approach allows organizations to achieve better outcomes by tailoring the system to specific needs and continuously optimizing its components [3].
- **Scalability Challenges**: As RAG systems expand, challenges related to scalability, such as increased delay and communication overhead, become significant. Modular RAG addresses these by leveraging advanced architectures and distribution strategies to maintain high performance and resource efficiency [1].

In conclusion, the transition from Naive to Modular RAG represents a pivotal development in the field of AI model architectures. By addressing the limitations of earlier models and introducing innovative design principles, Modular RAG sets a new standard for performance and scalability in RAG systems. This analysis underscores the importance of continued research and development in this area to fully realize the potential of these technologies in practical applications.

### Sources

[1] http://arxiv.org/abs/2407.21059v1  
[2] http://arxiv.org/abs/2406.00944v2  
[3] https://adasci.org/how-does-modular-rag-improve-upon-naive-rag/  

Copy

# Print completed interview section
Markdown(interview_graph.get_state(config).values["sections"][0])

Output results in markdown format.