# 01. Cross Encoder Reranker

Cross encoder reranker is a technique used to improve the performance of search enhancement generation (RAG) systems. This document describes how to implement reranker in retriever using the crossing encoder model of the Hugging Face.

#### Key features and how it works <a href="#id-2" id="id-2"></a>

**purpose**

* Re-rank the searched documents to top the documents most relevant to the question

**structure**

* Receive and process questions and documents simultaneously

**How it works**

* Directly output similarities using questions and documents as one input
* Simultaneously analyze questions and documents through the Self-attention mechanism

**Advantages**

* More accurate similarities can be measured
* Explore semantic similarities between questions and documents

**Limit point**

* High operating costs and long time
* Difficulty applying directly to large sets of documents

#### Actual use <a href="#id-3" id="id-3"></a>

* In general, reranking only for the top k documents in the initial search
* After quickly extracting candidates with Bi-encoder, use them in a way that increases accuracy with Cross encoder

#### implementation <a href="#id-4" id="id-4"></a>

* Using a cross encoder model from Hugging Face or a model like BAAI/bge-reranker
* Easily integrated through CrossEncoderReranker components in frames such as LangChain

#### The main advantages of Reranker <a href="#reranker" id="reranker"></a>

1. Measure more accurate similarities
2. Exploring in-depth semantic similarities
3. Search results improvement
4. RAG system performance improvement
5. Flexible integration
6. Various pre-learning models available

#### Set the number of documents when using Reranker <a href="#reranker_1" id="reranker_1"></a>

* Generally running for the top 5\~10 documents
* The optimal number of documents needs to be determined through experimentation and evaluation

#### Trade-offs when using Reranker <a href="#reranker-trade-offs" id="reranker-trade-offs"></a>

1. Accuracy vs processing time
2. Performance improvement vs calculation cost
3. Search speed vs relevance accuracy
4. Meet system requirements
5. Data set characteristics consideration A simple example will explain how to implement Cross Encoder Reranker.

Copy

```
# Document output helper functions
def pretty_print_docs(docs):
    print(
        f"\n{'-' * 100}\n".join(
            [f"Document {i+1}:\n\n" + d.page_content for i, d in enumerate(docs)]
        )
    )
```

Copy

```
from langchain_community.document_loaders import TextLoader
from langchain_community.vectorstores import FAISS
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter

# load document
documents = TextLoader("./data/appendix-keywords.txt").load()

# Setting up the text splitter
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=100)

# split document
texts = text_splitter.split_documents(documents)

# Setting up the embedding model
embeddingsModel = HuggingFaceEmbeddings(
    model_name="sentence-transformers/msmarco-distilbert-dot-v5"
)

# Creating FAISS index from document and setting up search engine
retriever = FAISS.from_documents(texts, embeddingsModel).as_retriever(
    search_kwargs={"k": 10}
)

# Query settings
query = "Word2Vec Let me tell you about it?"

# Execute queries and return result documents
docs = retriever.invoke(query)

# output the results document
pretty_print_docs(docs)
```

Copy

```
 Document 1: 

Open Source 

Definition: Open source means software that allows source code to be released and freely used, modified, and distributed by anyone. This plays an important role in promoting collaboration and innovation. 
Example: The Linux operating system is a representative open source project. 
Associates: software development, community, technology collaboration 

Structured Data 

Definition: Structured data is data organized according to a defined format or schema. It can be easily retrieved and analyzed from databases, spreadsheets, etc. 
Example: A customer information table stored in a relational database is an example of structured data. 
Associations: database, data analysis, data modeling 

Parser 
---------------------------------------------------------------------------------------------------- 
Document 2: 

Definition: LLM refers to a large language model trained with large-scale text data. These models are used for various natural language understanding and creation tasks. 
Example: OpenAI's GPT series is a representative large language model. 
Associated Keywords: natural language processing, deep learning, text generation 

FAISS (Facebook AI Similarity Search) 

Definition: FAISS is a high-speed similarity search library developed by Facebook, specifically designed to effectively search for similar vectors in large vector sets. 
Example: FAISS can be used to quickly find similar images among millions of image vectors. 
Associated Keywords: vector search, machine learning, database optimization 

Open Source 
---------------------------------------------------------------------------------------------------- 
Document 3: 

InstructGPT 

Definition: InstructGPT is a GPT model optimized to perform specific tasks according to user's instructions. This model is designed to produce more accurate and relevant results. 
Example: If a user provides specific instructions such as "draft email", InstructGPT creates an email based on the relevant content. 
Associated Keyword: Artificial Intelligence, Natural Language Understanding, Command-Based Processing 

Keyword Search 

Definition: Keyword search is the process of finding information based on keywords entered by the user. It is used as a basic search method in most search engines and database systems. 
Example: When a user searches for "Coffee Shop Seoul", it returns a list of related coffee shops. 
Associates: search engine, data search, information search 

Page Rank 
---------------------------------------------------------------------------------------------------- 
Document 4: 

Token 

Definition: Token means splitting text into smaller units. This can usually be a word, sentence, or verse. 
Example: Split the sentence "I go to school" into "I", "to school", and "go". 
Associated Keywords: tokenization, natural language processing, parsing 

Tokenizer 

Definition: A talkizer is a tool for splitting text data into tokens. It is used to preprocess data in natural language processing. 
Example: "I love programming."The sentences ["I", "love", "programming", "."]Split to. 
Associated Keywords: tokenization, natural language processing, parsing 

VectorStore 

Definition: Vector Store is a system that stores data converted to vector format. It is used for search, classification and other data analysis tasks. 
Example: Save the word embedding vectors to the database for quick access. 
Associated Keyword: Embedding, Database, Vectorization 

SQL 
---------------------------------------------------------------------------------------------------- 
Document 5: 

Page Rank 

Definition: A page rank is an algorithm that evaluates the importance of a web page, mainly used to rank search engine results. This is evaluated by analyzing the link structure between web pages. 
Example: Google search engines use the page rank algorithm to rank search results. 
Associates: Search engine optimization, web analytics, link analysis 

Data mining 

Definition: Data mining is the process of discovering useful information from large amounts of data. It utilizes technologies such as statistics, machine learning, and pattern recognition. 
Example: It is an example of data mining that retailers analyze customer purchase data to develop a sales strategy. 
Associated Keyword: Big Data, Pattern Recognition, Predictive Analysis 

Multimodal 
---------------------------------------------------------------------------------------------------- 
Document 6: 

Semantic Search 

Definition: A semantic search is a search method that goes beyond a simple keyword match for a user's query and grasps its meaning and returns related results. 
Example: When a user searches for a "solar planet", it returns information about the related planet, such as "Vegetic", "Mars", etc. 
Associates: natural language processing, search algorithms, data mining 

Embedding 

Definition: Embedding is the process of converting text data, such as words or sentences, into a low-dimensional, continuous vector. This allows the computer to understand and process the text. 
Example: The word "apple" is expressed in vectors such as [0.65, -0.23, 0.17]. 
Associated Keywords: natural language processing, vectorization, deep learning 

Token 
---------------------------------------------------------------------------------------------------- 
Document 7: 

Pandas 

Definition: Pandas is a library that provides data analysis and manipulation tools for the Python programming language. This allows you to perform data analysis tasks efficiently. 
Example: You can use pandas to read CSV files, refine data, and perform various analyzes. 
Associates: Data Analysis, Python, Data Processing 

GPT (Generative Pretrained Transformer) 

Definition: GPT is a proactive language model pre-trained with a large dataset, utilized for a variety of text-based tasks. It can generate natural language based on the text entered. 
Example: A chatbot that generates detailed answers to questions provided by the user can use the GPT model. 
Associates: Natural language processing, text generation, deep learning 

InstructGPT 
---------------------------------------------------------------------------------------------------- 
Document 8: 

Multimodal 

Definition: Multimodal is a technique that combines and processes different types of data modes (e.g. text, images, sounds, etc.). It is used to extract or predict richer and more accurate information through interactions between different types of data. 
Examples: A system that analyzes images and descriptive text together to perform more accurate image classification is an example of multimodal technology. 
Associated Keyword: Data Convergence, Artificial Intelligence, Dip Running 
---------------------------------------------------------------------------------------------------- 
Document 9: 

Crawling 

Definition: Crawl is the process of collecting data by visiting web pages in an automated manner. It is often used for search engine optimization or data analysis. 
Example: Crawl is a Google search engine to visit a web site on the Internet to collect and index content. 
Associates: data collection, web scraping, search engine 

Word2Vec 

Definition: Word2Vec is a natural language processing technique that maps words to vector spaces to represent meaningful relationships between words. It produces vectors based on the contextual similarity of words. 
Example: In the Word2Vec model, "King" and "Queen" are represented by vectors in positions close to each other. 
Associated Keywords: natural language processing, embedding, semantic similarity 
LLM (Large Language Model) 
---------------------------------------------------------------------------------------------------- 
Document 10: 

JSON 

Definition: JSON (JavaScript Object Notation) is a lightweight data exchange format that uses readable text for both people and machines to represent data objects. 
Example: {" Name": "Flood Road", "Age": 30, "Occupation": "Developer" } is data in JSON format. 
Associates: Data Exchange, Web Development, API 

Transformer 

Definition: Transformers are a type of deep-learning model used in natural language processing, mainly used for translation, summary, text generation, etc. This is based on the Attention mechanism. 
Example: Google translators use transformer models to perform translations between different languages. 
Associated Keywords: deep learning, natural language processing, Attention 

HuggingFace 
---------------------------------------------------------------------------------------------------- 
Document 11: 

SQL 

Definition: Structured Query Language (SQL) is a programming language for managing data in a database. You can do a variety of things, including data lookup, modification, insertion, deletion, etc. 
Example: SELECT * FROM users WHERE age > 18; views user information over 18 years old. 
Associated Keywords: database, query, data management 

CSV 

Definition: CSV (Comma-Separated Values) is a file format that stores data, and each data value is separated by commas. Used to simply store and exchange tabular data. 
Example: CSV files with headers named name, age, job may contain data such as Hong Gil-dong, 30, developer. 
Associates: data format, file processing, data exchange 

JSON 
---------------------------------------------------------------------------------------------------- 
Document 12: 

HuggingFace 

Definition: HuggingFace is a library that provides a variety of pre-trained models and tools for natural language processing. This helps researchers and developers do NLP work easily. 
Example: You can use HuggingFace's Transformers library to do emotional analysis, text generation, and more. 
Associates: Natural language processing, deep learning, library 

Digital Transformation 

Definition: Digital transformation is the process of leveraging technology to transform a company's services, culture and operations. This focuses on improving the business model and increasing competitiveness through digital technology. 
Example: Innovating data storage and processing by introducing cloud computing is an example of digital transformation. 
Related Keywords: innovation, technology, business model 

Crawling 
---------------------------------------------------------------------------------------------------- 
Document 13: 

Parser 

Definition: Parser is a tool that analyzes a given data (string, files, etc.) and converts it into a structured form. It is used for parsing programming languages or processing file data. 
Example: Parsing an HTML document to create a DOM structure for a web page is an example of parsing. 
Associative keywords: parsing, compiler, data processing 

TF-IDF (Term Frequency-Inverse Document Frequency) 

Definition: TF-IDF is a statistical measure used to evaluate the importance of words within a document. This takes into account the frequency of words in a document and the scarcity of those words in the entire set of documents. 
Example: Words that do not appear frequently in many documents have high TF-IDF values. 
Associates: Natural language processing, information retrieval, data mining 

Deep Learning 
---------------------------------------------------------------------------------------------------- 
Document 14: 

DataFrame 

Definition: DataFrame is a table-shaped data structure consisting of rows and columns, mainly used for data analysis and processing. 
Example: In the Pandas library, DataFrame can have columns of various data types, facilitating data manipulation and analysis. 
Associated Keywords: data analysis, pandas, data processing 

Attention mechanism 

Definition: The Attention mechanism is a technique that allows you to pay more'attention' to important information in deep learning. It is mainly used in sequence data (eg text, time series data). 
Example: In the translation model, the Attention mechanism focuses more on important parts of the input sentence, producing accurate translations. 
Associated Keywords: deep learning, natural language processing, sequence modeling 

Pandas 
---------------------------------------------------------------------------------------------------- 
Document 15: 

Deep Learning 

Definition: Deep learning is a field of machine learning that uses the artificial neural network to solve complex problems. This focuses on learning high-level expressions in data. 
Examples: Dip-learning models are utilized in image recognition, speech recognition, and natural language processing. 
Associated Press: Artificial Neural Network, Machining, Data Analysis 

Schema 

Definition: A schema defines the structure of a database or file, providing a blueprint of how data is stored and organized. 
Example: A table schema in a relational database defines column names, data types, key constraints, and more. 
Associates: database, data modeling, data management 

DataFrame 
```

Now the basics `retriever` for `ContextualCompressionRetriever` I'll wrap it up with. `CrossEncoderReranker` has `HuggingFaceCrossEncoder` Use to rearrange the returned results. -Multilingual support BGE Reranker: [bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3)

Copy

```
from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import CrossEncoderReranker
from langchain_community.cross_encoders import HuggingFaceCrossEncoder

# initialize model
model = HuggingFaceCrossEncoder(model_name="BAAI/bge-reranker-v2-m3")

# select top 3 documents
compressor = CrossEncoderReranker(model=model, top_n=3)

# reset document compression finder
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

# search compressed documents
compressed_docs = compression_retriever.invoke("Word2Vec 에 대해서 알려줄래?")

# print document
pretty_print_docs(compressed_docs)
```

Copy

```
 Document 1: 

Crawling 

Definition: Crawl is the process of collecting data by visiting web pages in an automated manner. It is often used for search engine optimization or data analysis. 
Example: Crawl is a Google search engine to visit a web site on the Internet to collect and index content. 
Associates: data collection, web scraping, search engine 

Word2Vec 

Definition: Word2Vec is a natural language processing technique that maps words to vector spaces to represent meaningful relationships between words. It produces vectors based on the contextual similarity of words. 
Example: In the Word2Vec model, "King" and "Queen" are represented by vectors in positions close to each other. 
Associated Keywords: natural language processing, embedding, semantic similarity 
LLM (Large Language Model) 
---------------------------------------------------------------------------------------------------- 
Document 2: 

Token 

Definition: Token means splitting text into smaller units. This can usually be a word, sentence, or verse. 
Example: Split the sentence "I go to school" into "I", "to school", and "go". 
Associated Keywords: tokenization, natural language processing, parsing 

Tokenizer 

Definition: A talkizer is a tool for splitting text data into tokens. It is used to preprocess data in natural language processing. 
Example: "I love programming."The sentences ["I", "love", "programming", "."]Split to. 
Associated Keywords: tokenization, natural language processing, parsing 

VectorStore 

Definition: Vector Store is a system that stores data converted to vector format. It is used for search, classification and other data analysis tasks. 
Example: Save the word embedding vectors to the database for quick access. 
Associated Keyword: Embedding, Database, Vectorization 

SQL 
---------------------------------------------------------------------------------------------------- 
Document 3: 

Semantic Search 

Definition: A semantic search is a search method that goes beyond a simple keyword match for a user's query and grasps its meaning and returns related results. 
Example: When a user searches for a "solar planet", it returns information about the related planet, such as "Vegetic", "Mars", etc. 
Associates: natural language processing, search algorithms, data mining 

Embedding 

Definition: Embedding is the process of converting text data, such as words or sentences, into a low-dimensional, continuous vector. This allows the computer to understand and process the text. 
Example: The word "apple" is expressed in vectors such as [0.65, -0.23, 0.17]. 
Associated Keywords: natural language processing, vectorization, deep learning 

Token 
```

Copy

```
compressed_docs
```

Copy

```
 [Document (metadata={'source':'./data/appendix-keywords.txt'}, page_content='Crawling\n\n Definition: Crawling is the process of collecting data by visiting web pages in an automated manner. This is often used for search engine optimization or data analysis.\n Example: It is a crawl that Google search engines visit web sites on the Internet to collect and index content.\nAssociation Keywords: data collection, web scraping, search engine\n\nWord2Vec\n\n Definition: Word2Vec is a natural language processing technique that maps words to a vector space to represent a meaningful relationship between words. This creates a vector based on the contextual similarity of the word.\n Example: In the Word2Vec model, "King" and "Queen" are represented by vectors in close positions with each other.\nAssociate: Natural language processing, embedding, semantic similarity\nLLM (Large Language Model)'), Document (metadata={'source':'d. This can usually be a word, sentence, or verse.\n Example: Split the sentence "I go to school" into "I", "To school", "Goes".\nAssociation keyword: tokenization, natural language processing, parsing\n\nTokenizer\n\n Definition:  The talkizer is a tool for splitting text data into tokens. It is used to preprocess data in natural language processing.\n Example: "I love programming."The sentences ["I", "love", "programming", "."]Split to.\n association keyword: tokenization, natural language processing, parsing\n\nVectorStore\n\n Definition: Vector store is a system that stores data converted to vector format. This is used for search, sorting, and other data analysis tasks.\n Example: You can quickly access the word embedding vectors to the database.\Negative keywords: embedding, database, vectorization\n\nSQL'), Document (metadata={'source':'./data/appendix-keywords.txt'}Returns information about related planets such as Jupiter", "Mars", etc..\NAssociation Keywords: natural language processing, search algorithms, data mining\n\nEmbedding\n\n Definition: Embedding converts text data such as words or sentences into low-dimensional continuous vectors It is a process. This allows the computer to understand and process the text.\n Example: The word "apple" [0.65, -0.23,  Expressed in a vector such as 0.17].\NAssociation keyword: natural language processing, vectorization, dipping\n\nToken')] 
```
