LLM Chronicles #3: How Tools Work In Langchain?
Lets see the magic under the hood
I have been fascinated with Langchain’s growing variety of tools which gives LLMs capabilities like creating a Jira ticket, write a PowerBI query or create Zapier flows. Hence I decided to get an understanding of its working in order to build custom tools. It is useful to know how to integrate custom tools in order to make cool internal chatbots.
We will go through an example where we create a Conversational ReAct Agent with access to 3 tools & memory of last 5 messages via ConversationBufferWindowMemory.
Let’s create a predefined tool - google search
from langchain.agents import Tool
search = GoogleSearchAPIWrapper(google_api_key="xx", google_cse_id="xx")
search_tool = Tool(
name = "Search",
func=search.run,
description="useful for when you need to answer questions about current events"
)Let’s create 2 new custom tools - CircumferenceTool & PythagorasTool1. These are math tools which can help LLM do correct calculations.
from typing import Union, Optional
from math import pi, sqrt, cos, sin
from langchain.tools import BaseTool
from langchain.utilities import GoogleSearchAPIWrapper
class CircumferenceTool(BaseTool):
name = "Circumference calculator"
description = "use this tool when you need to calculate a circumference using the radius of a circle"
def _run(self, radius: Union[int, float]):
return float(radius)*2.0*pi
def _arun(self, radius: Union[int, float]):
raise NotImplementedError("This tool does not support async")
desc = (
"use this tool when you need to calculate the length of an hypotenuse "
"given one or two sides of a triangle and/or an angle (in degrees). "
"To use the tool you must provide at least two of the following parameters "
"['adjacent_side', 'opposite_side', 'angle']."
)
class PythagorasTool(BaseTool):
name = "Hypotenuse calculator"
description = desc
def _run(
self,
adjacent_side: Optional[Union[int, float]] = None,
opposite_side: Optional[Union[int, float]] = None,
angle: Optional[Union[int, float]] = None
):
# check for the values we have been given
if adjacent_side and opposite_side:
return sqrt(float(adjacent_side)**2 + float(opposite_side)**2)
elif adjacent_side and angle:
return adjacent_side / cos(float(angle))
elif opposite_side and angle:
return opposite_side / sin(float(angle))
else:
return "Could not calculate the hypotenuse of the triangle. Need two or more of `adjacent_side`, `opposite_side`, or `angle`."
def _arun(self, query: str):
raise NotImplementedError("This tool does not support async")
Notice the two fields in the classes above - name and description. These are used to format the SUFFIX prompt shown below.
SUFFIX = """TOOLS
------
Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:
{{tools}}
{format_instructions}
USER'S INPUT
--------------------
Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):
{{{{input}}}}"""
Define the LLM & memory
import os
from langchain.chat_models import ChatOpenAI
from langchain.chains.conversation.memory import ConversationBufferWindowMemory
OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY') or 'sk-ss'
llm = ChatOpenAI(
openai_api_key=OPENAI_API_KEY,
temperature=0,
model_name='gpt-3.5-turbo'
)
conversational_memory = ConversationBufferWindowMemory(
memory_key='chat_history',
k=5,
return_messages=True
)Define the agent
from langchain.agents import initialize_agent
tools = [CircumferenceTool(), PythagorasTool(), search_tool]
# initialize agent with tools
agent = initialize_agent(
agent='chat-conversational-react-description',
tools=tools,
llm=llm,
verbose=True,
max_iterations=3,
early_stopping_method='generate',
memory=conversational_memory
)Now we override system prompt to make LLM force use the math tools; otherwise there is no guarantee it will actually use tools when needed.
We update the system prompt with - “Unfortunately, Assistant is terrible at maths. When provided with math questions, no matter how simple, assistant always refers to it's trusty tools and absolutely does NOT try to answer math questions by itself”.
sys_msg = """Assistant is a large language model trained by OpenAI.
Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.
Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.
Unfortunately, Assistant is terrible at maths. When provided with math questions, no matter how simple, assistant always refers to it's trusty tools and absolutely does NOT try to answer math questions by itself
Overall, Assistant is a powerful system that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist.
"""
new_prompt = agent.agent.create_prompt(
system_message=sys_msg,
tools=tools
)
agent.agent.llm_chain.prompt = new_promptHere is how the prompt looks after passing the tools.
print(agent.agent.llm_chain.prompt.messages[2].prompt.template)
TOOLS
------
Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:
> Circumference calculator: use this tool when you need to calculate a circumference using the radius of a circle
> Hypotenuse calculator: use this tool when you need to calculate the length of an hypotenuse given one or two sides of a triangle and/or an angle (in degrees). To use the tool you must provide at least two of the following parameters ['adjacent_side', 'opposite_side', 'angle'].
> Search: useful for when you need to answer questions about current events
RESPONSE FORMAT INSTRUCTIONS
----------------------------
When responding to me, please output a response in one of two formats:
**Option 1:**
Use this if you want the human to use a tool.
Markdown code snippet formatted in the following schema:
```json
{{
"action": string \ The action to take. Must be one of Circumference calculator, Hypotenuse calculator
"action_input": string \ The input to the action
}}
```
**Option #2:**
Use this if you want to respond directly to the human. Markdown code snippet formatted in the following schema:
```json
{{
"action": "Final Answer",
"action_input": string \ You should put what you want to return to use here
}}
```
USER'S INPUT
--------------------
Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else):
{input}The input goes to the last line variable {input} and prompt has clear instructions on how to generate the output which is needed for deciding the next action.
Output is parsed to get pair of action & action_input. Tool is used as per the action name with the required input/s. If the action is “Final Answer”, we finish the chain with the value of action_input.
from __future__ import annotations
import json
from typing import Union
from langchain.agents import AgentOutputParser
from langchain.agents.conversational_chat.prompt import FORMAT_INSTRUCTIONS
from langchain.schema import AgentAction, AgentFinish
class ConvoOutputParser(AgentOutputParser):
def get_format_instructions(self) -> str:
return FORMAT_INSTRUCTIONS
def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
cleaned_output = text.strip()
if "```json" in cleaned_output:
_, cleaned_output = cleaned_output.split("```json")
if "```" in cleaned_output:
cleaned_output, _ = cleaned_output.split("```")
if cleaned_output.startswith("```json"):
cleaned_output = cleaned_output[len("```json") :]
if cleaned_output.startswith("```"):
cleaned_output = cleaned_output[len("```") :]
if cleaned_output.endswith("```"):
cleaned_output = cleaned_output[: -len("```")]
cleaned_output = cleaned_output.strip()
response = json.loads(cleaned_output)
action, action_input = response["action"], response["action_input"]
if action == "Final Answer":
return AgentFinish({"output": action_input}, text)
else:
return AgentAction(action, action_input, text)
Example output
```json
{
"action": "Circumference calculator",
"action_input": {
"radius": 7.81
}
}
```This output means we need to use the tool which has name Circumference calculator and pass input radius to it with value 7.81. So the CircumferenceTool returns..
Observation: 49.071677249072565Then we pass the observation to the agent and ask the agent if we were able to get the answer.
TEMPLATE_TOOL_RESPONSE = """TOOL RESPONSE:
---------------------
{observation}
USER'S INPUT
--------------------
Okay, so what is the response to my last comment? If using information obtained from the tools you must mention it explicitly without mentioning the tool names - I have forgotten all TOOL RESPONSES! Remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else."""The agent responds with action equal to “Final Answer”.
```json
{
"action": "Final Answer",
"action_input": "The circumference of a circle with a radius of 7.81mm is approximately 49.07mm."
}
```Because the action is “Final Answer”, we don’t need to use any tool and this answer is shown to the user.
The circumference of a circle with a radius of 7.81mm is approximately 49.07mm.Parting thoughts
Here we learnt how to define our own tools, how tool info is added to the prompt, how the parsing is done to invoke the tool and how to override the LLM to make it surely use the tool. Hope its more clear how this beautiful mechanism works and you can create your own tools now.
Come join Maxpool - A Data Science community to discuss real ML problems!
Connect with me on Medium, Twitter & LinkedIn.
https://github.com/pinecone-io/examples/blob/master/generation/langchain/handbook/07-langchain-tools.ipynb