courses/prompt_engineering_interactive_tutorial/Anthropic 1P/10.2_Appendix_Tool Use.ipynb

Appendix 10.2: Tool Use

• Lesson
• Exercises
• Example Playground

Setup

Run the following setup cell to load your API key and establish the get_completion helper function.

In [ ]:

%pip install anthropic

# Import python's built-in regular expression library
import re
import anthropic

# Retrieve the API_KEY variable from the IPython store
%store -r API_KEY

client = anthropic.Anthropic(api_key=API_KEY)

# Rewrittten to call Claude 3 Sonnet, which is generally better at tool use, and include stop_sequences
def get_completion(messages, system_prompt="", prefill="",stop_sequences=None):
    message = client.messages.create(
        model="claude-3-sonnet-20240229",
        max_tokens=2000,
        temperature=0.0,
        system=system_prompt,
        messages=messages,
        stop_sequences=stop_sequences
    )
    return message.content[0].text

---

Lesson

While it might seem conceptually complex at first, tool use, a.k.a. function calling, is actually quite simple! You already know all the skills necessary to implement tool use, which is really just a combination of substitution and prompt chaining.

In previous substitution exercises, we substituted text into prompts. With tool use, we substitute tool or function results into prompts. Claude can't literally call or access tools and functions. Instead, we have Claude:

1. Output the tool name and arguments it wants to call
2. Halt any further response generation while the tool is called
3. Then we reprompt with the appended tool results

Function calling is useful because it expands Claude's capabilities and enables Claude to handle much more complex, multi-step tasks. Some examples of functions you can give Claude:

• Calculator
• Word counter
• SQL database querying and data retrieval
• Weather API

You can get Claude to do tool use by combining these two elements:

1. A system prompt, in which we give Claude an explanation of the concept of tool use as well as a detailed descriptive list of the tools it has access to
2. The control logic with which to orchestrate and execute Claude's tool use requests

Tool use roadmap

This lesson teaches our current tool use format. However, we will be updating and improving tool use functionality in the near future, including:

• A more streamlined format for function definitions and calls
• More robust error handilgj and edge case coverage
• Tighter integration with the rest of our API
• Better reliability and performance, especially for more complex tool use tasks

Examples

To enable tool use in Claude, we start with the system prompt. In this special tool use system prompt, wet tell Claude:

• The basic premise of tool use and what it entails
• How Claude can call and use the tools it's been given
• A detailed list of tools it has access to in this specific scenario

Here's the first part of the system prompt, explaining tool use to Claude. This part of the system prompt is generalizable across all instances of prompting Claude for tool use. The tool calling structure we're giving Claude (<function_calls> [...] </function_calls>) is a structure Claude has been specifically trained to use, so we recommend that you stick with this.

In [ ]:

system_prompt_tools_general_explanation = """You have access to a set of functions you can use to answer the user's question. This includes access to a
sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external
resources, except by invoking the below functions.

You can invoke one or more functions by writing a "<function_calls>" block like the following as part of your
reply to the user:
<function_calls>
<invoke name="$FUNCTION_NAME">
<antml:parameter name="$PARAMETER_NAME">$PARAMETER_VALUE</parameter>
...
</invoke>
<nvoke name="$FUNCTION_NAME2">
...
</invoke>
</function_calls>

String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that
spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular
expressions.

The output and/or any errors will appear in a subsequent "<function_results>" block, and remain there as part of
your reply to the user.
You may then continue composing the rest of your reply to the user, respond to any errors, or make further function
calls as appropriate.
If a "<function_results>" does NOT appear after your function calls, then they are likely malformatted and not
recognized as a call."""

Here's the second part of the system prompt, which defines the exact tools Claude has access to in this specific situation. In this example, we will be giving Claude a calculator tool, which takes three parameters: two operands and an operator.

Then we combine the two parts of the system prompt.

In [ ]:

system_prompt_tools_specific_tools = """Here are the functions available in JSONSchema format:
<tools>
<tool_description>
<tool_name>calculator</tool_name>
<description>
Calculator function for doing basic arithmetic.
Supports addition, subtraction, multiplication
</description>
<parameters>
<parameter>
<name>first_operand</name>
<type>int</type>
<description>First operand (before the operator)</description>
</parameter>
<parameter>
<name>second_operand</name>
<type>int</type>
<description>Second operand (after the operator)</description>
</parameter>
<parameter>
<name>operator</name>
<type>str</type>
<description>The operation to perform. Must be either +, -, *, or /</description>
</parameter>
</parameters>
</tool_description>
</tools>
"""

system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools

Now we can give Claude a question that requires use of the calculator tool. We will use <function_calls\> in stop_sequences to detect if and when Claude calls the function.

In [ ]:

multiplication_message = {
    "role": "user",
    "content": "Multiply 1,984,135 by 9,343,116"
}

stop_sequences = ["</function_calls>"]

# Get Claude's response
function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)
print(function_calling_response)

Now, we can extract out the parameters from Claude's function call and actually run the function on Claude's behalf.

First we'll define the function's code.

In [ ]:

def do_pairwise_arithmetic(num1, num2, operation):
    if operation == '+':
        return num1 + num2
    elif operation == "-":
        return num1 - num2
    elif operation == "*":
        return num1 * num2
    elif operation == "/":
        return num1 / num2
    else:
        return "Error: Operation not supported."

Then we'll extract the parameters from Claude's function call response. If all the parameters exist, we run the calculator tool.

In [ ]:

def find_parameter(message, parameter_name):
    parameter_start_string = f"name=\"{parameter_name}\">"
    start = message.index(parameter_start_string)
    if start == -1:
        return None
    if start > 0:
        start = start + len(parameter_start_string)
        end = start
        while message[end] != "<":
            end += 1
    return message[start:end]

first_operand = find_parameter(function_calling_response, "first_operand")
second_operand = find_parameter(function_calling_response, "second_operand")
operator = find_parameter(function_calling_response, "operator")

if first_operand and second_operand and operator:
    result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)
    print("---------------- RESULT ----------------")
    print(f"{result:,}")

Now that we have a result, we have to properly format that result so that when we pass it back to Claude, Claude understands what tool that result is in relation to. There is a set format for this that Claude has been trained to recognize:

<function_results>
<result>
<tool_name>{TOOL_NAME}</tool_name>
<stdout>
{TOOL_RESULT}
</stdout>
</result>
</function_results>

Run the cell below to format the above tool result into this structure.

In [ ]:

def construct_successful_function_run_injection_prompt(invoke_results):
    constructed_prompt = (
        "<function_results>\n"
        + '\n'.join(
            f"<result>\n<tool_name>{res['tool_name']}</tool_name>\n<stdout>\n{res['tool_result']}\n</stdout>\n</result>"
            for res in invoke_results
        ) + "\n</function_results>"
    )

    return constructed_prompt

formatted_results = [{
    'tool_name': 'do_pairwise_arithmetic',
    'tool_result': result
}]
function_results = construct_successful_function_run_injection_prompt(formatted_results)
print(function_results)

Now all we have to do is send this result back to Claude by appending the result to the same message chain as before, and we're good!

In [ ]:

full_first_response = function_calling_response + "</function_calls>"

# Construct the full conversation
messages = [multiplication_message,
{
    "role": "assistant",
    "content": full_first_response
},
{
    "role": "user",
    "content": function_results
}]
   
# Print Claude's response
final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)
print("------------- FINAL RESULT -------------")
print(final_response)

Congratulations on running an entire tool use chain end to end!

Now what if we give Claude a question that doesn't that doesn't require using the given tool at all?

In [ ]:

non_multiplication_message = {
    "role": "user",
    "content": "Tell me the capital of France."
}

stop_sequences = ["</function_calls>"]

# Get Claude's response
function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)
print(function_calling_response)

Success! As you can see, Claude knew not to call the function when it wasn't needed.

If you would like to experiment with the lesson prompts without changing any content above, scroll all the way to the bottom of the lesson notebook to visit the Example Playground.

---

Exercises

• Exercise 10.2.1 - SQL

Exercise 10.2.1 - SQL

In this exercise, you'll be writing a tool use prompt for querying and writing to the world's smallest "database". Here's the initialized database, which is really just a dictionary.

In [ ]:

db = {
    "users": [
        {"id": 1, "name": "Alice", "email": "alice@example.com"},
        {"id": 2, "name": "Bob", "email": "bob@example.com"},
        {"id": 3, "name": "Charlie", "email": "charlie@example.com"}
    ],
    "products": [
        {"id": 1, "name": "Widget", "price": 9.99},
        {"id": 2, "name": "Gadget", "price": 14.99},
        {"id": 3, "name": "Doohickey", "price": 19.99}
    ]
}

And here is the code for the functions that write to and from the database.

In [ ]:

def get_user(user_id):
    for user in db["users"]:
        if user["id"] == user_id:
            return user
    return None

def get_product(product_id):
    for product in db["products"]:
        if product["id"] == product_id:
            return product
    return None

def add_user(name, email):
    user_id = len(db["users"]) + 1
    user = {"id": user_id, "name": name, "email": email}
    db["users"].append(user)
    return user

def add_product(name, price):
    product_id = len(db["products"]) + 1
    product = {"id": product_id, "name": name, "price": price}
    db["products"].append(product)
    return product

To solve the exercise, start by defining a system prompt like system_prompt_tools_specific_tools above. Make sure to include the name and description of each tool, along with the name and type and description of each parameter for each function. We've given you some starting scaffolding below.

In [ ]:

system_prompt_tools_specific_tools_sql = """
"""

system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools_sql

When you're ready, you can try out your tool definition system prompt on the examples below. Just run the below cell!

In [ ]:

examples = [
    "Add a user to the database named Deborah.",
    "Add a product to the database named Thingo",
    "Tell me the name of User 2",
    "Tell me the name of Product 3"
]

for example in examples:
    message = {
        "role": "user",
        "content": example
    }

    # Get & print Claude's response
    function_calling_response = get_completion([message], system_prompt=system_prompt, stop_sequences=stop_sequences)
    print(example, "\n----------\n\n", function_calling_response, "\n*********\n*********\n*********\n\n")

If you did it right, the function calling messages should call the add_user, add_product, get_user, and get_product functions correctly.

For extra credit, add some code cells and write parameter-parsing code. Then call the functions with the parameters Claude gives you to see the state of the "database" after the call.

❓ If you want to see a possible solution, run the cell below!

In [ ]:

from hints import exercise_10_2_1_solution; print(exercise_10_2_1_solution)

Congrats!

Congratulations on learning tool use and function calling! Head over to the last appendix section if you would like to learn more about search & RAG.

---

Example Playground

This is an area for you to experiment freely with the prompt examples shown in this lesson and tweak prompts to see how it may affect Claude's responses.

In [ ]:

system_prompt_tools_general_explanation = """You have access to a set of functions you can use to answer the user's question. This includes access to a
sandboxed computing environment. You do NOT currently have the ability to inspect files or interact with external
resources, except by invoking the below functions.

You can invoke one or more functions by writing a "<function_calls>" block like the following as part of your
reply to the user:
<function_calls>
<invoke name="$FUNCTION_NAME">
<antml:parameter name="$PARAMETER_NAME">$PARAMETER_VALUE</parameter>
...
</invoke>
<nvoke name="$FUNCTION_NAME2">
...
</invoke>
</function_calls>

String and scalar parameters should be specified as is, while lists and objects should use JSON format. Note that
spaces for string values are not stripped. The output is not expected to be valid XML and is parsed with regular
expressions.

The output and/or any errors will appear in a subsequent "<function_results>" block, and remain there as part of
your reply to the user.
You may then continue composing the rest of your reply to the user, respond to any errors, or make further function
calls as appropriate.
If a "<function_results>" does NOT appear after your function calls, then they are likely malformatted and not
recognized as a call."""

In [ ]:

system_prompt_tools_specific_tools = """Here are the functions available in JSONSchema format:
<tools>
<tool_description>
<tool_name>calculator</tool_name>
<description>
Calculator function for doing basic arithmetic.
Supports addition, subtraction, multiplication
</description>
<parameters>
<parameter>
<name>first_operand</name>
<type>int</type>
<description>First operand (before the operator)</description>
</parameter>
<parameter>
<name>second_operand</name>
<type>int</type>
<description>Second operand (after the operator)</description>
</parameter>
<parameter>
<name>operator</name>
<type>str</type>
<description>The operation to perform. Must be either +, -, *, or /</description>
</parameter>
</parameters>
</tool_description>
</tools>
"""

system_prompt = system_prompt_tools_general_explanation + system_prompt_tools_specific_tools

In [ ]:

multiplication_message = {
    "role": "user",
    "content": "Multiply 1,984,135 by 9,343,116"
}

stop_sequences = ["</function_calls>"]

# Get Claude's response
function_calling_response = get_completion([multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)
print(function_calling_response)

In [ ]:

def do_pairwise_arithmetic(num1, num2, operation):
    if operation == '+':
        return num1 + num2
    elif operation == "-":
        return num1 - num2
    elif operation == "*":
        return num1 * num2
    elif operation == "/":
        return num1 / num2
    else:
        return "Error: Operation not supported."

In [ ]:

def find_parameter(message, parameter_name):
    parameter_start_string = f"name=\"{parameter_name}\">"
    start = message.index(parameter_start_string)
    if start == -1:
        return None
    if start > 0:
        start = start + len(parameter_start_string)
        end = start
        while message[end] != "<":
            end += 1
    return message[start:end]

first_operand = find_parameter(function_calling_response, "first_operand")
second_operand = find_parameter(function_calling_response, "second_operand")
operator = find_parameter(function_calling_response, "operator")

if first_operand and second_operand and operator:
    result = do_pairwise_arithmetic(int(first_operand), int(second_operand), operator)
    print("---------------- RESULT ----------------")
    print(f"{result:,}")

In [ ]:

def construct_successful_function_run_injection_prompt(invoke_results):
    constructed_prompt = (
        "<function_results>\n"
        + '\n'.join(
            f"<result>\n<tool_name>{res['tool_name']}</tool_name>\n<stdout>\n{res['tool_result']}\n</stdout>\n</result>"
            for res in invoke_results
        ) + "\n</function_results>"
    )

    return constructed_prompt

formatted_results = [{
    'tool_name': 'do_pairwise_arithmetic',
    'tool_result': result
}]
function_results = construct_successful_function_run_injection_prompt(formatted_results)
print(function_results)

In [ ]:

full_first_response = function_calling_response + "</function_calls>"

# Construct the full conversation
messages = [multiplication_message,
{
    "role": "assistant",
    "content": full_first_response
},
{
    "role": "user",
    "content": function_results
}]
   
# Print Claude's response
final_response = get_completion(messages, system_prompt=system_prompt, stop_sequences=stop_sequences)
print("------------- FINAL RESULT -------------")
print(final_response)

In [ ]:

non_multiplication_message = {
    "role": "user",
    "content": "Tell me the capital of France."
}

stop_sequences = ["</function_calls>"]

# Get Claude's response
function_calling_response = get_completion([non_multiplication_message], system_prompt=system_prompt, stop_sequences=stop_sequences)
print(function_calling_response)