02. Caching (Cache)

Caching

LangChain offers a selective caching layer above LLM.

This is useful for two reasons.

• For LLM providers if multiple requests for the same completion Reduce the number of API calls to reduce costs You can.

• For LLM providers Reduce the number of API calls to speed up your application There is.

Copy

# API KEY a configuration file for managing environment variables
from dotenv import load_dotenv

# API KEY Load information
load_dotenv()

Copy

true

Copy

# LangSmith Set up tracking. https://smith.langchain.com
# !pip install langchain-teddynote
from langchain_teddynote import logging

# Enter a project name.
logging.langsmith("CH04-Models")

Copy

 Start tracking LangSmith. 
[Project name] 
CH04-Models

Generates models and prompts

Copy

from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate

# Create a model.
llm = ChatOpenAI(model_name="gpt-3.5-turbo")

# Generate a prompt.
prompt = PromptTemplate.from_template("{country} about 200 okay, summarize it for me
")

# Create a chain.
chain = prompt | llm

Copy

%%time 
response = chain.invoke({"country": "한국"})
print(response.content)

Copy

Korea is a highly developed country located in East Asia, historically there were various dynasties, including high-ranking, three countries, Goryeo, and Joseon, now known as the Republic of Korea. Seoul is the capital and has high economic power and high-tech power. It is also gaining great popularity worldwide through Hanyu and K-pop, and Korean food, Kimchi, Bulgogi, and Rami are also world famous. North Korea is on the north side of the Korean Peninsula, and relations with South Korea continue to conflict and debate, but efforts toward peace continue. Korea is an attractive country where ancient history and modern culture coexist harmoniously. 
CPU times: user 48.6 ms, sys: 17.1 ms, total: 65.7 ms 
Wall time: 2.88 s 

InMemoryCache

Save the answer to the same question using the inmemory cache, and return the answer stored in the cache.

Copy

%%time
from langchain.globals import set_llm_cache
from langchain.cache import InMemoryCache

# Uses in-memory cache.
set_llm_cache(InMemoryCache())

# Run the chain.
response = chain.invoke({"country": "korea"})
print(response.content)

Copy

Korea is a highly developed and modernized country located in East Asia, economically classified as a developed country and has many world famous companies. In addition, Korea's culture and entertainment industry is gaining popularity worldwide due to the Hanwang craze. Korea boasts a high level of education and innovative skills, and is also known for its high-intensity social competition and poor educational environment. In addition, the relationship with North Korea is still tense due to the division of the Korean Peninsula, and it is a country that is focused on preserving the natural environment and over-consumption with these political issues. 
CPU times: user 92 ms, sys: 14.2 ms, total: 106 ms 
Wall time: 3.97 s 

Copy

%%time
# Run the chain.
response = chain.invoke({"country": "korea"})
print(response.content)

Copy

Korea is a highly developed and modernized country located in East Asia, economically classified as a developed country and has many world famous companies. In addition, Korea's culture and entertainment industry is gaining popularity worldwide due to the Hanwang craze. Korea boasts a high level of education and innovative skills, and is also known for its high-intensity social competition and poor educational environment. In addition, the relationship with North Korea is still tense due to the division of the Korean Peninsula, and it is a country that is focused on preserving the natural environment and over-consumption with these political issues. 
CPU times: user 2.48 ms, sys: 1.21 ms, total: 3.69 ms 
Wall time: 2.86 ms 

SQLite Cache

Copy

from langchain_community.cache import SQLiteCache
from langchain_core.globals import set_llm_cache
import os

# Create a cache directory.
if not os.path.exists("cache"):
    os.makedirs("cache")

# SQLiteCache I use.
set_llm_cache(SQLiteCache(database_path="cache/llm_cache.db"))

Copy

%%time 
# Run to the chain.
response = chain.invoke({"country": "korea"})
print(response.content)

Copy

Korea is an advanced country with an advanced economy located in East Asia. The capital is Seoul, and there are many cities densely populated with a population of about 51 million. Korea has highly developed IT technology, and there are many world famous companies. In addition, the Korean oil craze is spreading worldwide due to cultural content such as K-pop and K-drama. Korea faces environmental issues such as fine dust, but governments and citizens are trying to pursue sustainable development. In addition, the relationship with North Korea is tense on the Korean Peninsula, and efforts to achieve peace and reunification continue. 
CPU times: user 50.6 ms, sys: 27 ms, total: 77.7 ms 
Wall time: 76.2 ms