courses/tool_use/04_tool_choice.ipynb

Tool choice

The Claude API supports a parameter called tool_choice that allows you to specify how you want Claude to call tools. In this notebook, we'll take a look at how it works and when to use it.

When working with the tool_choice parameter, we have three possible options:

• auto allows Claude to decide whether to call any provided tools or not.
• any tells Claude that it must use one of the provided tools, but doesn't force a particular tool.
• tool allows us to force Claude to always use a particular tool.

This diagram illustrates how each option works:

Let's take a look at each option in detail. We'll start by importing the Anthropic SDK:

In [1]:

import boto3

bedrock_client = boto3.client(service_name='bedrock-runtime', region_name="us-west-2")
model_id = "anthropic.claude-3-5-sonnet-20241022-v2:0"

Auto

Setting tool_choice to auto allows the model to automatically decide whether to use tools or not. This is the default behavior when working with tools if you don't use the tool_choice parameter at all.

To demonstrate this, we're going to provide Claude with a fake web search tool. We will ask Claude questions, some of which would require calling the web search tool and others which Claude should be able to answer on its own.

Let's start by defining a tool called web_search. Please note, to keep this demo simple, we're not actually searching the web here.

In [2]:

def web_search(topic):
    print(f"pretending to search the web for {topic}")

web_search_tool = {
    "toolSpec": {
        "name": "web_search",
        "description": "A tool to retrieve up to date information on a given topic by searching the web",
        "inputSchema": {
            "json": {
                "type": "object",
                "properties": {
                    "topic": {
                        "type": "string",
                        "description": "The topic to search the web for"
                    },
                },
                "required": ["topic"]
            }
        }
    }
}

Next, we write a function that accepts a user_query and passes it along to Claude, along with the web_search_tool.

We also set tool_choice to auto:

In [3]:

tool_choice={"type": "auto"}

Here's the complete function:

In [4]:

from datetime import date

def chat_with_web_search(user_query):

    system_prompt=f"""
    Answer as many questions as you can using your existing knowledge.  
    Only search the web for queries that you can not confidently answer.
    Today's date is {date.today().strftime("%B %d %Y")}
    If you think a user's question involves something in the future that hasn't happened yet, use the search tool.
    """

    messages = [{"role": "user", "content": [{"text": user_query}]}]

    inference_config={"maxTokens":400}
    tool_config = {"tools":[web_search_tool], "toolChoice": {"auto": {}}}

    # Send the message.
    response = bedrock_client.converse(
        modelId=model_id,
        messages=messages,
        system=[{"text": system_prompt}],
        inferenceConfig=inference_config,
        toolConfig=tool_config,
    )

    last_content_block = response["output"]["message"]["content"][-1]
    if "text" in last_content_block:
        print("Claude did NOT call a tool")
        print(f"Assistant: {last_content_block['text']}")
    if "toolUse" in last_content_block:
        print("Claude wants to use a tool")
        print(last_content_block)

Let's start with a question Claude should be able to answer without using the tool:

In [5]:

chat_with_web_search("What color is the sky?")

Claude did NOT call a tool
Assistant: I can answer this question from my general knowledge without needing to search the web.

The sky appears blue during clear daytime conditions due to a phenomenon called Rayleigh scattering. As sunlight travels through the Earth's atmosphere, it collides with gas molecules. These molecules scatter the light in all directions. Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time during the day.

However, the sky can appear different colors depending on conditions and time:
- At sunrise and sunset, the sky often appears red, orange, or pink
- During stormy weather, it may appear grey
- At night, it appears dark or black
- In some locations, pollution can affect the sky's appearance

When we ask "What color is the sky?", Claude does not use the tool. Let's try asking something that Claude should use the web search tool to answer:

In [6]:

chat_with_web_search("Who won the 2024 Miami Grand Prix?")

Claude wants to use a tool
{'toolUse': {'toolUseId': 'tooluse_gzgDZ3KsTJS4BTi6oX2wzg', 'name': 'web_search', 'input': {'topic': 'Who won the 2024 Miami F1 Miami Grand Prix winner'}}}

When we ask "Who won the 2024 Miami Grand Prix?", Claude uses the web search tool!

Let's try a few more examples:

In [7]:

# Claude should NOT need to use the tool for this:
chat_with_web_search("Who won the Superbowl in 2022?")

Claude did NOT call a tool
Assistant: I can answer this without needing to search, as it's a past event.

The Los Angeles Rams won Super Bowl LVI (56) on February 13, 2022, defeating the Cincinnati Bengals 23-20. The game was played at SoFi Stadium in Inglewood, California, and Rams wide receiver Cooper Kupp was named Super Bowl MVP.

In [8]:

# Claude SHOULD use the tool for this:
chat_with_web_search("Who won the Superbowl in 2024?")

Claude did NOT call a tool
Assistant: The Kansas City Chiefs won Super Bowl LVIII (58) on February 11, 2024, defeating the San Francisco 49ers 25-22 in overtime at Allegiant Stadium in Las Vegas, Nevada. It was their second consecutive Super Bowl victory and third in five years. Patrick Mahomes was named Super Bowl MVP as he led the Chiefs to victory with a game-winning touchdown drive in overtime.

Your prompt matters!

When working with tool_choice set to auto, it's important that you spend time to write a detailed prompt. Often, Claude can be over-eager to call tools. Writing a detailed prompt helps Claude determine when to call a tool and when not to. In the above example, we included specific instructions in the system prompt:

In [9]:

system_prompt=f"""
    Answer as many questions as you can using your existing knowledge.  
    Only search the web for queries that you can not confidently answer.
    Today's date is {date.today().strftime("%B %d %Y")}
    If you think a user's question involves something in the future that hasn't happened yet, use the search tool.
"""

---

Forcing a specific tool

We can force Claude to use a particular tool using tool_choice. In the example below, we've defined two simple tools:

• print_sentiment_scores - a tool that "tricks" Claude into generating well-structured JSON output containing sentiment analysis data. For more info on this approach, see Extracting Structured JSON using Claude and Tool Use in the Anthropic Cookbook.
• calculator - a very simple calculator tool that takes two numbers and adds them together .

In [10]:

tools = [
    {
        "toolSpec": {
            "name": "print_sentiment_scores",
            "description": "Prints the sentiment scores of a given tweet or piece of text.",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "positive_score": {"type": "number", "description": "The positive sentiment score, ranging from 0.0 to 1.0."},
                        "negative_score": {"type": "number", "description": "The negative sentiment score, ranging from 0.0 to 1.0."},
                        "neutral_score": {"type": "number", "description": "The neutral sentiment score, ranging from 0.0 to 1.0."}
                    },
                    "required": ["positive_score", "negative_score", "neutral_score"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "calculator",
            "description": "Adds two number",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "num1": {"type": "number", "description": "first number to add"},
                        "num2": {"type": "number", "description": "second number to add"},
                    },
                    "required": ["num1", "num2"]
                }
            }
        }
    }
]

Our goal is to write a function called analyze_tweet_sentiment that takes in a tweet and uses Claude to print a basic sentiment analysis of that tweet. Eventually we will "force" Claude to use the print_sentiment_scores tool, but we'll start by showing what happens when we do not force the tool use.

In this first "bad" version of the analyze_tweet_sentiment function, we provide Claude with both tools. For the sake of comparison, we'll start by setting tool_choice to auto:

In [11]:

tool_choice={"auto": {}}

Please note that we are deliberately not providing Claude with a well-written prompt, to make it easier to see the impact of forcing the use of a particular tool.

In [12]:

def analyze_tweet_sentiment(query):

    response = bedrock_client.converse(
        modelId=model_id,
        messages=[{"role": "user", "content": [{"text": query}]}],
        system=[{"text": system_prompt}],
        inferenceConfig={"maxTokens":4096},
        toolConfig={"tools": tools, "toolChoice": {"auto": {}}},
    )

    print(response)

Let's see what happens when we call the function with the tweet Holy cow, I just made the most incredible meal!

In [13]:

analyze_tweet_sentiment("Holy cow, I just made the most incredible meal!")

{'ResponseMetadata': {'RequestId': '4f4ecfc1-780e-40ba-a7d7-e79f8dd09ff6', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Thu, 14 Nov 2024 19:24:09 GMT', 'content-type': 'application/json', 'content-length': '576', 'connection': 'keep-alive', 'x-amzn-requestid': '4f4ecfc1-780e-40ba-a7d7-e79f8dd09ff6'}, 'RetryAttempts': 0}, 'output': {'message': {'role': 'assistant', 'content': [{'text': "I can sense your excitement about your meal! While I'd love to hear more details about what made it so incredible, I notice you're expressing a strong positive sentiment. I can demonstrate this using the sentiment analysis tool:"}, {'toolUse': {'toolUseId': 'tooluse_T1zbs07wTl2H-_XgAeIWSw', 'name': 'print_sentiment_scores', 'input': {'positive_score': 0.9, 'negative_score': 0.0, 'neutral_score': 0.1}}}]}}, 'stopReason': 'tool_use', 'usage': {'inputTokens': 650, 'outputTokens': 146, 'totalTokens': 796}, 'metrics': {'latencyMs': 3513}}

Claude does not call our print_sentiment_scores tool and instead responds directly with:

"That's great to hear! I don't actually have the capability to assess sentiment from text, but it sounds like you're really excited and proud of the incredible meal you made

Next, let's imagine someone tweets this: I love my cats! I had four and just adopted 2 more! Guess how many I have now?

In [14]:

analyze_tweet_sentiment("I love my cats! I had four and just adopted 2 more! Guess how many I have now?")

{'ResponseMetadata': {'RequestId': 'cf9bed41-bf7b-4e79-9138-43d4493c6df7', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Thu, 14 Nov 2024 19:24:12 GMT', 'content-type': 'application/json', 'content-length': '445', 'connection': 'keep-alive', 'x-amzn-requestid': 'cf9bed41-bf7b-4e79-9138-43d4493c6df7'}, 'RetryAttempts': 0}, 'output': {'message': {'role': 'assistant', 'content': [{'text': 'Let me help you calculate the total number of cats you have now!\n\nYou had 4 cats and adopted 2 more, so let me use the calculator to add these numbers:'}, {'toolUse': {'toolUseId': 'tooluse_Dtmcx1X3QnO4Fb2Kch3mZA', 'name': 'calculator', 'input': {'num1': 4, 'num2': 2}}}]}}, 'stopReason': 'tool_use', 'usage': {'inputTokens': 663, 'outputTokens': 110, 'totalTokens': 773}, 'metrics': {'latencyMs': 2767}}

Clearly, this current implementation is not doing what we want (mostly because we set it up to fail).

So let's force Claude to always use the print_sentiment_scores tool by updating tool_choice:

In [15]:

toolConfig={"tools": tools, "toolChoice": {"tool": {"name":"print_sentiment_scores"}}}

In addition to setting type to tool, we must provide a particular tool name.

In [16]:

def analyze_tweet_sentiment(query):

    response = bedrock_client.converse(
        modelId=model_id,
        messages=[{"role": "user", "content": [{"text": query}]}],
        system=[{"text": system_prompt}],
        inferenceConfig={"maxTokens":4096},
        toolConfig={"tools": tools, "toolChoice": {"tool": {"name":"print_sentiment_scores"}}},
    )

    print(response)

Now if we try prompting Claude with the same prompts from earlier, it's always going to call the print_sentiment_scores tool:

In [17]:

analyze_tweet_sentiment("Holy cow, I just made the most incredible meal!")

{'ResponseMetadata': {'RequestId': '46d415e5-598b-4d4f-9296-6ccd2f20d5ee', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Thu, 14 Nov 2024 19:24:14 GMT', 'content-type': 'application/json', 'content-length': '335', 'connection': 'keep-alive', 'x-amzn-requestid': '46d415e5-598b-4d4f-9296-6ccd2f20d5ee'}, 'RetryAttempts': 0}, 'output': {'message': {'role': 'assistant', 'content': [{'toolUse': {'toolUseId': 'tooluse_Fw_EwfkeQEOslPm5jkdBYA', 'name': 'print_sentiment_scores', 'input': {'positive_score': 0.9, 'negative_score': 0.0, 'neutral_score': 0.1}}}]}}, 'stopReason': 'tool_use', 'usage': {'inputTokens': 658, 'outputTokens': 79, 'totalTokens': 737}, 'metrics': {'latencyMs': 1747}}

Claude calls our print_sentiment_scores tool:

ToolUseBlock(id='toolu_staging_01FMRQ9pZniZqFUGQwTcFU4N', input={'positive_score': 0.9, 'negative_score': 0.0, 'neutral_score': 0.1}, name='print_sentiment_scores', type='tool_use')

Even if we try to trip up Claude with a "Math-y" tweet, it still always calls the print_sentiment_scores tool:

In [18]:

analyze_tweet_sentiment("I love my cats! I had four and just adopted 2 more! Guess how many I have now?")

{'ResponseMetadata': {'RequestId': 'e0f45f28-9e1b-4eaa-af95-3596f681d542', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Thu, 14 Nov 2024 19:24:17 GMT', 'content-type': 'application/json', 'content-length': '443', 'connection': 'keep-alive', 'x-amzn-requestid': 'e0f45f28-9e1b-4eaa-af95-3596f681d542'}, 'RetryAttempts': 0}, 'output': {'message': {'role': 'assistant', 'content': [{'toolUse': {'toolUseId': 'tooluse_JWZGijmlQNuUkjOOYzQrYA', 'name': 'print_sentiment_scores', 'input': {'positive_score': 0.9, 'negative_score': 0.0, 'neutral_score': 0.1}}}, {'toolUse': {'toolUseId': 'tooluse_7NRLuN0kQlqrRWxvnmFieA', 'name': 'calculator', 'input': {'num1': 4, 'num2': 2}}}]}}, 'stopReason': 'tool_use', 'usage': {'inputTokens': 671, 'outputTokens': 132, 'totalTokens': 803}, 'metrics': {'latencyMs': 2518}}

Even though we're forcing Claude to call our print_sentiment_scores tool, we should still employ some basic prompt engineering to give Claude better task context:

In [19]:

def analyze_tweet_sentiment(query):

    prompt = f"""
    Analyze the sentiment in the following tweet: 
    <tweet>{query}</tweet>
    """
    
    response = bedrock_client.converse(
        modelId=model_id,
        messages=[{"role": "user", "content": [{"text": prompt}]}],
        system_prompts=[{"text": system_prompt}],
        inferenceConfig={"maxTokens":4096},
        toolConfig={"tools": tools, "toolChoice": {"tool": {"name":"print_sentiment_scores"}}}
    )

    print(response)

---

Any

The final option for tool_choice is any, which allows us to tell Claude, "You must call a tool, but you can pick which one." Imagine we want to create a SMS chatbot using Claude. The only way for this chatbot to actually "communicate" with a user is via SMS text message.

In the example below, we make a very simple text-messaging assistant that has access to two tools:

• send_text_to_user - sends a text message to a user.
• get_customer_info - looks up customer data based on a username.

The idea is to create a chatbot that always calls one of these tools and never responds with a non-tool response. In all situations, Claude should either respond back by trying to send a text message or calling get_customer_info to get more customer information. To ensure this, we set tool_choice to any:

In [20]:

toolConfig={"tools": tools, "toolChoice": {"any": {}}}

In [21]:

def send_text_to_user(text):
    # Sends a text to the user
    # We'll just print out the text to keep things simple:
    print(f"TEXT MESSAGE SENT: {text}")

def get_customer_info(username):
    return {
        "username": username,
        "email": f"{username}@email.com",
        "purchases": [
            {"id": 1, "product": "computer mouse"},
            {"id": 2, "product": "screen protector"},
            {"id": 3, "product": "usb charging cable"},
        ]
    }

tools = [
    {
        "toolSpec": {    
            "name": "send_text_to_user",
            "description": "Sends a text message to a user",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "text": {"type": "string", "description": "The piece of text to be sent to the user via text message"},
                    },
                    "required": ["text"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "get_customer_info",
            "description": "gets information on a customer based on the customer's username.  Response includes email, username, and previous purchases. Only call this tool once a user has provided you with their username",
            "inputSchema": {
                "json": {
                    "type": "object",
                    "properties": {
                        "username": {"type": "string", "description": "The username of the user in question. "},
                    },
                    "required": ["username"]
                }
            }
        }
    },
]

system_prompt = """
All your communication with a user is done via text message.
Only call tools when you have enough information to accurately call them.  
Do not call the get_customer_info tool until a user has provided you with their username. This is important.
If you do not know a user's username, simply ask a user for their username.
"""

def sms_chatbot(user_message):
    messages = [{"role": "user", "content":[{"text": user_message}]}]
    
    response = bedrock_client.converse(
        modelId=model_id,
        messages=messages,
        system=[{"text": system_prompt}],
        inferenceConfig={"maxTokens":4096},
        toolConfig={"tools": tools, "toolChoice": {"any": {}}},
    )

    if response['stopReason'] == "tool_use":
        last_content_block = response["output"]["message"]["content"][-1]
        if "toolUse" in last_content_block:
            tool_name = last_content_block["toolUse"]["name"]
            tool_inputs = last_content_block["toolUse"]["input"]
            print(f"=======Claude Wants To Call The {tool_name} Tool=======")
            if tool_name == "send_text_to_user":
                send_text_to_user(tool_inputs["text"])
            elif tool_name == "get_customer_info":
                print(get_customer_info(tool_inputs["username"]))
            else:
                print("Oh dear, that tool doesn't exist!")
            
    else:
        print("No tool was called. This shouldn't happen!")
    

Let's start simple:

In [22]:

sms_chatbot("Hey there! How are you?")

=======Claude Wants To Call The send_text_to_user Tool=======
TEXT MESSAGE SENT: Hello! I'm doing well, thank you for asking. I'm here to help you today. Is there something specific I can assist you with? If you'd like me to look up your customer information, I'll just need your username.

Claude responds back by calling the send_text_to_user tool.

Next, we'll ask Claude something a bit trickier:

In [23]:

sms_chatbot("I need help looking up an order")

=======Claude Wants To Call The send_text_to_user Tool=======
TEXT MESSAGE SENT: I'd be happy to help you look up your order information. Could you please provide me with your username so I can access your order history?

Claude wants to send a text message, asking a user to provide their username.

Now, let's see what happens when we provide Claude with our username:

In [24]:

sms_chatbot("I need help looking up an order.  My username is jenny76")

=======Claude Wants To Call The get_customer_info Tool=======
{'username': 'jenny76', 'email': 'jenny76@email.com', 'purchases': [{'id': 1, 'product': 'computer mouse'}, {'id': 2, 'product': 'screen protector'}, {'id': 3, 'product': 'usb charging cable'}]}

Claude calls the get_customer_info tool, just as we hoped!

Even if we send Claude a gibberish message, it will still call one of our tools:

In [25]:

sms_chatbot("askdj aksjdh asjkdbhas kjdhas 1+1 ajsdh")

=======Claude Wants To Call The send_text_to_user Tool=======
TEXT MESSAGE SENT: I apologize, but I couldn't understand your message. Could you please rephrase your question clearly? I'm here to help with customer information and sending messages. If you'd like to access your customer information, please provide your username.