# 14. Automating evaluation using online evaluation

#### Online Evaluators <a href="#online-evaluators" id="online-evaluators"></a>

Sometimes you want to evaluate the products recorded in your project.

Copy

```
# installation
# !pip install -qU langsmith langchain-teddynote
```

Copy

```
# Configuration file for managing API KEY as environment variable
from dotenv import load_dotenv

# Load API KEY information
load_dotenv()
```

Copy

```
 True 
```

Copy

```
# LangSmith Set up tracking. https://smith.langchain.com
# !pip install -qU langchain-teddynote
from langchain_teddynote import logging

# Enter a project name.
logging.langsmith("CH16-Auto-Evaluation-Test")
```

Copy

```
 Start tracking LangSmith. 
[Project name] 
CH16-Auto-Evaluation-Test 
```

#### Chain setting for online Evaluation <a href="#evaluation-chain" id="evaluation-chain"></a>

Copy

```
from myrag import PDFRAG
from langchain_openai import ChatOpenAI

# Creating a PDFRAG object
rag = PDFRAG(
    "data/SPRI_AI_Brief_December 2023 issue호_F.pdf",
    ChatOpenAI(model="gpt-4o-mini", temperature=0),
)

# Create a retriever
retriever = rag.create_retriever()

# Create a chain
rag_chain = rag.create_chain(retriever)
```

Copy

```
from langchain_core.runnables import RunnableParallel, RunnablePassthrough

# RunnableParallel Create an object.
evaluation_runnable = RunnableParallel(
    {
        "context": retriever,
        "answer": rag_chain,
        "question": RunnablePassthrough(),
    }
)
```

Run the test chain to see if Runs reflects the results.

Copy

```
# Chain execution
_ = evaluation_runnable.invoke("What is the name of the generative AI developed by Samsung Electronics?")
```

#### Online LLM-as-judge creation <a href="#online-llm-as-judge" id="online-llm-as-judge"></a>

> Online LLM-as-judge creation

![](https://rajasgs.gitbook.io/langchain-for-beginners/~gitbook/image?url=https%3A%2F%2Fwikidocs.net%2Fimages%2Fpage%2F259219%2Fonline-eval-01.png\&width=768\&dpr=4\&quality=100\&sign=3abc550a\&sv=2)

> Secrets & API Keys designation (OpenAI API Key)

![](https://rajasgs.gitbook.io/langchain-for-beginners/~gitbook/image?url=https%3A%2F%2Fwikidocs.net%2Fimages%2Fpage%2F259219%2Fonline-eval-02.png\&width=768\&dpr=4\&quality=100\&sign=161f8687\&sv=2)

> Provider, Model, Prompt Settings

> Provider, Model, Prompt Settings

![](https://rajasgs.gitbook.io/langchain-for-beginners/~gitbook/image?url=https%3A%2F%2Fwikidocs.net%2Fimages%2Fpage%2F259219%2Fonline-eval-03.png\&width=768\&dpr=4\&quality=100\&sign=73732490\&sv=2)

> For facts, specify output.context (change to suit your settings)

![](https://rajasgs.gitbook.io/langchain-for-beginners/~gitbook/image?url=https%3A%2F%2Fwikidocs.net%2Fimages%2Fpage%2F259219%2Fonline-eval-04.png\&width=768\&dpr=4\&quality=100\&sign=9ff57ae5\&sv=2)

> answer designates output.answer (changed to suit your settings)

![](https://rajasgs.gitbook.io/langchain-for-beginners/~gitbook/image?url=https%3A%2F%2Fwikidocs.net%2Fimages%2Fpage%2F259219%2Fonline-eval-05.png\&width=768\&dpr=4\&quality=100\&sign=cc0e06e\&sv=2)

> Preview ensures that data is entered in the correct place
