How to add message history

Prerequisites

This guide assumes familiarity with the following concepts:

• LangChain Expression Language (LCEL)
• Chaining runnables
• Configuring chain parameters at runtime
• Prompt templates
• Chat Messages

Passing conversation state into and out a chain is vital when building a chatbot. The RunnableWithMessageHistory class lets us add message history to certain types of chains. It wraps another Runnable and manages the chat message history for it.

Specifically, it can be used for any Runnable that takes as input one of:

• a sequence of BaseMessages
• a dict with a key that takes a sequence of BaseMessages
• a dict with a key that takes the latest message(s) as a string or sequence of BaseMessages, and a separate key that takes historical messages

And returns as output one of

• a string that can be treated as the contents of an AIMessage
• a sequence of BaseMessage
• a dict with a key that contains a sequence of BaseMessage

Let's take a look at some examples to see how it works. First we construct a runnable (which here accepts a dict as input and returns a message as output):

OpenAI

pip install -qU langchain-openai

import getpass
import os

os.environ["OPENAI_API_KEY"] = getpass.getpass()

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo-0125")

Anthropic

pip install -qU langchain-anthropic

import getpass
import os

os.environ["ANTHROPIC_API_KEY"] = getpass.getpass()

from langchain_anthropic import ChatAnthropic

llm = ChatAnthropic(model="claude-3-sonnet-20240229")

Google

pip install -qU langchain-google-vertexai

import getpass
import os

os.environ["GOOGLE_API_KEY"] = getpass.getpass()

from langchain_google_vertexai import ChatVertexAI

llm = ChatVertexAI(model="gemini-pro")

Cohere

pip install -qU langchain-cohere

import getpass
import os

os.environ["COHERE_API_KEY"] = getpass.getpass()

from langchain_cohere import ChatCohere

llm = ChatCohere(model="command-r")

FireworksAI

pip install -qU langchain-fireworks

import getpass
import os

os.environ["FIREWORKS_API_KEY"] = getpass.getpass()

from langchain_fireworks import ChatFireworks

llm = ChatFireworks(model="accounts/fireworks/models/mixtral-8x7b-instruct")

MistralAI

pip install -qU langchain-mistralai

import getpass
import os

os.environ["MISTRAL_API_KEY"] = getpass.getpass()

from langchain_mistralai import ChatMistralAI

llm = ChatMistralAI(model="mistral-large-latest")

TogetherAI

pip install -qU langchain-openai

import getpass
import os

os.environ["TOGETHER_API_KEY"] = getpass.getpass()

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api.together.xyz/v1",
    api_key=os.environ["TOGETHER_API_KEY"],
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",)

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You're an assistant who's good at {ability}. Respond in 20 words or fewer",
        ),
        MessagesPlaceholder(variable_name="history"),
        ("human", "{input}"),
    ]
)
runnable = prompt | model

API Reference:ChatPromptTemplate | MessagesPlaceholder

To manage the message history, we will need:

1. This runnable;
2. A callable that returns an instance of BaseChatMessageHistory.

Check out the memory integrations page for implementations of chat message histories using Redis and other providers. Here we demonstrate using an in-memory ChatMessageHistory as well as more persistent storage using RedisChatMessageHistory.

In-memory

Below we show a simple example in which the chat history lives in memory, in this case via a global Python dict.

We construct a callable get_session_history that references this dict to return an instance of ChatMessageHistory. The arguments to the callable can be specified by passing a configuration to the RunnableWithMessageHistory at runtime. By default, the configuration parameter is expected to be a single string session_id. This can be adjusted via the history_factory_config kwarg.

Using the single-parameter default:

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

store = {}


def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]


with_message_history = RunnableWithMessageHistory(
    runnable,
    get_session_history,
    input_messages_key="input",
    history_messages_key="history",
)

API Reference:ChatMessageHistory | BaseChatMessageHistory | RunnableWithMessageHistory

info

Note that we've specified input_messages_key (the key to be treated as the latest input message) and history_messages_key (the key to add historical messages to).

When invoking this new runnable, we specify the corresponding chat history via a configuration parameter:

with_message_history.invoke(
    {"ability": "math", "input": "What does cosine mean?"},
    config={"configurable": {"session_id": "abc123"}},
)

AIMessage(content='Cosine is a trigonometric function that represents the ratio of the adjacent side to the hypotenuse of a right triangle.', response_metadata={'id': 'msg_01DH8iRBELVbF3sqM8U5sk8A', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 32, 'output_tokens': 31}}, id='run-e07fc012-a4f6-4e47-8ef8-250f296eba5b-0')

# Remembers
with_message_history.invoke(
    {"ability": "math", "input": "What is its inverse called?"},
    config={"configurable": {"session_id": "abc123"}},
)

AIMessage(content='The inverse of the cosine function is called the arccosine or inverse cosine.', response_metadata={'id': 'msg_015TeeRQBvTvc7XG1JxYqZyq', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 72, 'output_tokens': 22}}, id='run-32ae22ea-3b2f-4d38-8c8a-cb8702e2f3e7-0')

info

Note that in this case the context is preserved via the chat history for the provided session_id, so the model knows that "it" refers to "cosine" in this case.

Now let's try a different session_id

# New session_id --> does not remember.
with_message_history.invoke(
    {"ability": "math", "input": "What is its inverse called?"},
    config={"configurable": {"session_id": "def234"}},
)

AIMessage(content='The inverse of a function is the function that undoes the original function.', response_metadata={'id': 'msg_01M8WbHWg2sjWTz3m3NKqZuF', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 32, 'output_tokens': 18}}, id='run-b64c73d6-03ee-4b0a-85e0-34beb45408d4-0')

When we pass a different session_id, we start a new chat history, so the model does not know what "it" refers to.

Customization

The configuration parameters by which we track message histories can be customized by passing in a list of ConfigurableFieldSpec objects to the history_factory_config parameter. Below, we use two parameters: a user_id and conversation_id.

from langchain_core.runnables import ConfigurableFieldSpec

store = {}


def get_session_history(user_id: str, conversation_id: str) -> BaseChatMessageHistory:
    if (user_id, conversation_id) not in store:
        store[(user_id, conversation_id)] = ChatMessageHistory()
    return store[(user_id, conversation_id)]


with_message_history = RunnableWithMessageHistory(
    runnable,
    get_session_history,
    input_messages_key="input",
    history_messages_key="history",
    history_factory_config=[
        ConfigurableFieldSpec(
            id="user_id",
            annotation=str,
            name="User ID",
            description="Unique identifier for the user.",
            default="",
            is_shared=True,
        ),
        ConfigurableFieldSpec(
            id="conversation_id",
            annotation=str,
            name="Conversation ID",
            description="Unique identifier for the conversation.",
            default="",
            is_shared=True,
        ),
    ],
)

with_message_history.invoke(
    {"ability": "jokes", "input": "Tell me a joke"},
    config={"configurable": {"user_id": "123", "conversation_id": "1"}},
)

API Reference:ConfigurableFieldSpec

AIMessage(content="Why can't a bicycle stand up on its own? It's two-tired!", response_metadata={'id': 'msg_011qHi8pvbNkKhRb9XYRm2kc', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 30, 'output_tokens': 20}}, id='run-5d1d5b5a-ccec-4c2c-b11a-f1953dbe85a3-0')

# remembers
with_message_history.invoke(
    {"ability": "jokes", "input": "What was the joke about?"},
    config={"configurable": {"user_id": "123", "conversation_id": "1"}},
)

AIMessage(content='The joke was about a bicycle not being able to stand up on its own because it\'s "two-tired" (too tired).', response_metadata={'id': 'msg_01LbrkfidZgseBMxxRjQXJQH', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 59, 'output_tokens': 30}}, id='run-8b2ca810-77d7-44b8-b27b-677e0062b19a-0')

# New user_id --> does not remember
with_message_history.invoke(
    {"ability": "jokes", "input": "What was the joke about?"},
    config={"configurable": {"user_id": "456", "conversation_id": "1"}},
)

AIMessage(content="I'm afraid I don't have enough context to provide a relevant joke. As an AI assistant, I don't actually have pre-programmed jokes. I'd be happy to try generating a humorous response if you provide more details about the context.", response_metadata={'id': 'msg_01PgSp46hNJnKyNfNKPDauQ9', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 32, 'output_tokens': 54}}, id='run-ed202892-27e4-4da9-a26d-e0dc16b10940-0')

Note that in this case the context was preserved for the same user_id, but once we changed it, the new chat history was started, even though the conversation_id was the same.

Examples with runnables of different signatures

The above runnable takes a dict as input and returns a BaseMessage. Below we show some alternatives.

Messages input, dict output

from langchain_core.messages import HumanMessage
from langchain_core.runnables import RunnableParallel

chain = RunnableParallel({"output_message": model})


def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]


with_message_history = RunnableWithMessageHistory(
    chain,
    get_session_history,
    output_messages_key="output_message",
)

with_message_history.invoke(
    [HumanMessage(content="What did Simone de Beauvoir believe about free will")],
    config={"configurable": {"session_id": "baz"}},
)

API Reference:HumanMessage | RunnableParallel

{'output_message': AIMessage(content='Simone de Beauvoir was a prominent French existentialist philosopher who had some key beliefs about free will:\n\n1. Radical Freedom: De Beauvoir believed that humans have radical freedom - the ability to choose and define themselves through their actions. She rejected determinism and believed that we are not simply products of our biology, upbringing, or social circumstances.\n\n2. Ambiguity of the Human Condition: However, de Beauvoir also recognized the ambiguity of the human condition. While we have radical freedom, we are also situated beings constrained by our facticity (our given circumstances and limitations). This creates a tension and anguish in the human experience.\n\n3. Responsibility and Bad Faith: With radical freedom comes great responsibility. De Beauvoir criticized "bad faith" - the denial or avoidance of this responsibility by making excuses or pretending we lack free will. She believed we must courageously embrace our freedom and the burdens it entails.\n\n4. Ethical Engagement: For de Beauvoir, freedom is not just an abstract philosophical concept, but something that must be exercised through ethical engagement with the world and others. Our choices and actions have moral implications that we must grapple with.\n\nOverall, de Beauvoir\'s perspective on free will was grounded in existentialist principles - the belief that we are fundamentally free, yet this freedom is fraught with difficulty and responsibility. Her views emphasized the centrality of human agency and the ethical dimensions of our choices.', response_metadata={'id': 'msg_01QFXHx74GSzcMWnWc8YxYSJ', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 20, 'output_tokens': 324}}, id='run-752513bc-2b4f-4cad-87f0-b96fee6ebe43-0')}

with_message_history.invoke(
    [HumanMessage(content="How did this compare to Sartre")],
    config={"configurable": {"session_id": "baz"}},
)

{'output_message': AIMessage(content='Simone de Beauvoir\'s views on free will were quite similar to those of her long-time partner and fellow existentialist philosopher, Jean-Paul Sartre. There are some key parallels and differences:\n\nSimilarities:\n\n1. Radical Freedom: Both de Beauvoir and Sartre believed that humans have radical, unconditioned freedom to choose and define themselves.\n\n2. Rejection of Determinism: They both rejected deterministic views that see humans as products of their circumstances or nature.\n\n3. Emphasis on Responsibility: They agreed that with radical freedom comes great responsibility for one\'s choices and actions.\n\n4. Critique of "Bad Faith": Both philosophers criticized the tendency of people to deny or avoid their freedom through self-deception and making excuses.\n\nDifferences:\n\n1. Gendered Perspectives: While Sartre developed a more gender-neutral existentialist philosophy, de Beauvoir brought a distinctly feminist lens, exploring the unique challenges and experiences of women\'s freedom.\n\n2. Ethical Engagement: De Beauvoir placed more emphasis on the importance of ethical engagement with the world and others, whereas Sartre\'s focus was more individualistic.\n\n3. Ambiguity of the Human Condition: De Beauvoir was more attuned to the ambiguity and tensions inherent in the human condition, whereas Sartre\'s views were sometimes seen as more absolutist.\n\n4. Influence of Phenomenology: De Beauvoir was more influenced by the phenomenological tradition, which shaped her understanding of embodied, situated freedom.\n\nOverall, while Sartre and de Beauvoir shared a core existentialist framework, de Beauvoir\'s unique feminist perspective and emphasis on ethical engagement with others distinguished her views on free will and the human condition.', response_metadata={'id': 'msg_01BEANW4VX6cUWYjkv3CanLz', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 355, 'output_tokens': 388}}, id='run-e786ab3a-1a42-45f3-94a3-f0c591430df3-0')}

Messages input, messages output

RunnableWithMessageHistory(
    model,
    get_session_history,
)

RunnableWithMessageHistory(bound=RunnableBinding(bound=RunnableBinding(bound=RunnableLambda(_enter_history), config={'run_name': 'load_history'})
| RunnableBinding(bound=ChatAnthropic(model='claude-3-haiku-20240307', temperature=0.0, anthropic_api_url='https://api.anthropic.com', anthropic_api_key=SecretStr('**********'), _client=<anthropic.Anthropic object at 0x105682720>, _async_client=<anthropic.AsyncAnthropic object at 0x106a08fe0>), config_factories=[<function Runnable.with_listeners.<locals>.<lambda> at 0x106aeef20>]), config={'run_name': 'RunnableWithMessageHistory'}), get_session_history=<function get_session_history at 0x106aee520>, history_factory_config=[ConfigurableFieldSpec(id='session_id', annotation=<class 'str'>, name='Session ID', description='Unique identifier for a session.', default='', is_shared=True, dependencies=None)])

Dict with single key for all messages input, messages output

from operator import itemgetter

RunnableWithMessageHistory(
    itemgetter("input_messages") | model,
    get_session_history,
    input_messages_key="input_messages",
)

RunnableWithMessageHistory(bound=RunnableBinding(bound=RunnableBinding(bound=RunnableAssign(mapper={
  input_messages: RunnableBinding(bound=RunnableLambda(_enter_history), config={'run_name': 'load_history'})
}), config={'run_name': 'insert_history'})
| RunnableBinding(bound=RunnableLambda(itemgetter('input_messages'))
  | ChatAnthropic(model='claude-3-haiku-20240307', temperature=0.0, anthropic_api_url='https://api.anthropic.com', anthropic_api_key=SecretStr('**********'), _client=<anthropic.Anthropic object at 0x105682720>, _async_client=<anthropic.AsyncAnthropic object at 0x106a08fe0>), config_factories=[<function Runnable.with_listeners.<locals>.<lambda> at 0x106aef560>]), config={'run_name': 'RunnableWithMessageHistory'}), get_session_history=<function get_session_history at 0x106aee520>, input_messages_key='input_messages', history_factory_config=[ConfigurableFieldSpec(id='session_id', annotation=<class 'str'>, name='Session ID', description='Unique identifier for a session.', default='', is_shared=True, dependencies=None)])

Persistent storage

In many cases it is preferable to persist conversation histories. RunnableWithMessageHistory is agnostic as to how the get_session_history callable retrieves its chat message histories. See here for an example using a local filesystem. Below we demonstrate how one could use Redis. Check out the memory integrations page for implementations of chat message histories using other providers.

Setup

We'll need to install Redis if it's not installed already:

%pip install --upgrade --quiet redis

Start a local Redis Stack server if we don't have an existing Redis deployment to connect to:

docker run -d -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

REDIS_URL = "redis://localhost:6379/0"

LangSmith

LangSmith is especially useful for something like message history injection, where it can be hard to otherwise understand what the inputs are to various parts of the chain.

Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after you sign up at the link above, make sure to uncoment the below and set your environment variables to start logging traces:

# os.environ["LANGCHAIN_TRACING_V2"] = "true"
# os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()

Updating the message history implementation just requires us to define a new callable, this time returning an instance of RedisChatMessageHistory:

from langchain_community.chat_message_histories import RedisChatMessageHistory


def get_message_history(session_id: str) -> RedisChatMessageHistory:
    return RedisChatMessageHistory(session_id, url=REDIS_URL)


with_message_history = RunnableWithMessageHistory(
    runnable,
    get_message_history,
    input_messages_key="input",
    history_messages_key="history",
)

API Reference:RedisChatMessageHistory

We can invoke as before:

with_message_history.invoke(
    {"ability": "math", "input": "What does cosine mean?"},
    config={"configurable": {"session_id": "foobar"}},
)

AIMessage(content='Cosine is a trigonometric function that represents the ratio of the adjacent side to the hypotenuse of a right triangle.', response_metadata={'id': 'msg_01DwU2BD8KPLoXeZ6bZPqxxJ', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 164, 'output_tokens': 31}}, id='run-c2a443c4-79b1-4b07-bb42-5e9112e5bbfc-0')

with_message_history.invoke(
    {"ability": "math", "input": "What's its inverse"},
    config={"configurable": {"session_id": "foobar"}},
)

AIMessage(content='The inverse of cosine is called arccosine or inverse cosine.', response_metadata={'id': 'msg_01XYH5iCUokxV1UDhUa8xzna', 'model': 'claude-3-haiku-20240307', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 202, 'output_tokens': 19}}, id='run-97dda3a2-01e3-42e5-8241-f948e7535ffc-0')

tip

Langsmith trace

Looking at the Langsmith trace for the second call, we can see that when constructing the prompt, a "history" variable has been injected which is a list of two messages (our first input and first output).

Next steps

You have now learned one way to manage message history for a runnable.

To learn more, see the other how-to guides on runnables in this section.