03. LangSmith tracking settings

Setting up LangSmith tracking

LangSmith LLM application development, monitoring and testing Platform for. If you are starting a project or LangChain learning, LangSmith recommends proceeding after setting up.

LangSmith's tracking function

Tracking is a powerful tool to understand the behavior of LLM applications. LangSmith provides best-in-class tracking, whether or not LangChain is used.

Tracking can help you track down the following issues:

• Unexpected end result

• Why agents are looping

• Why the chain is slower than expected

• Number of tokens used by the agent at each stage

Project unit tracking

You can check the run count, Error incidence, token usage, billing information, etc. on a project basis.

When you click on a project, all Runs that have been run appear.

Detailed step-by-step pursuit of 1 run

After one run, we record in detail about the input/output content of GPT, as well as the search results of retrieve documents. Therefore, after checking the retrieved content of the document, it helps to determine whether the search algorithm should be changed or the prompt.

In addition, at the top, the time it took 1 run (Run) and the token used (5,104), etc., are not marked, and if you get a mouse hover on the token, it will not even indicate the amount charged.

Using LangSmith tracking

The method of using tracking is very simple.

LangSmith API Key issued

1. Access to https://smith.langchain.com/ to proceed with membership.

2. After signing up, you need to go through the process of authenticating your email.

3. Left cogwheel (Setting)-Press "Personal" -"Create API Key" in the middle to issue the API key.

Add your own description to Description and click the Create API Key button to create it.

After copying the generated key, proceed to the next step.

(caution!) Copy the generated key to a safe place to avoid leaking it.

LangSmith key setting on .env

first, .env Enter the key and project information issued by LangSmith in the file.

• LANGCHAIN_TRACING_V2 : "true" Setting it to start tracking.

• LANGCHAIN_ENDPOINT : https://api.smith.langchain.com Does not change.

• LANGCHAIN_API_KEY : In the previous step Key issued Enter.

• LANGCHAIN_PROJECT : Project name When you fill in, all runs (Run) are tracked to that project group.

Enable tracking in Jupyter Notebook or code

How to enable tracking is very simple. Just set the environment variable.

Bring what you set to .env.

from dotenv import load_dotenv

load_dotenv()

If the tracking you set is active, and the API KEY and project people are set up properly, this is enough.

However, if you want to change the project name or change the tracking, you can change it to the code below.

import os

os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_ENDPOINT"] = "https://api.smith.langchain.com"
os.environ["LANGCHAIN_PROJECT"] = "LangChain 프로젝트명"
os.environ["LANGCHAIN_API_KEY"] = "LangChain API KEY 입력"

langchain-teddynote

For the purpose of using langchain related functions more conveniently langchain-teddynote I created a package.

Method of installation

Installation code (run on terminal or on Jupyter Notebook)

pip install langchain-teddynote

LangSmith tracking settings

.env To file LangSmith API key set Should be ( LANGCHAIN_API_KEY )

from langchain_teddynote import logging

# Enter a project name.
logging.langsmith("Desired project name")

Output example

Start tracking LangSmith.
[project name]
langchain tutorial project

When you don't want to chase, Number of tracks turned off There is.

from langchain_teddynote import logging

# If set_enable=False is specified, tracking will not be performed.
logging.langsmith("Langchain Tutorial Project", set_enable=False)