Files

T

Elie Schoppik 0a2d2dcfeb adding files for bedrock-converse

2024-11-26 12:58:55 -05:00

2.6 MiB

Raw Permalink Blame History

Tool use with multiple tools

Now we're going to take our tool use to another level! We're going to provide Claude with a suite of tools it can select from. Our goal is to build a simple customer support chatbot for a fictional electronics company called TechNova. It will have access to simple tools including:

get_user to look up user details by email, username, or phone number
get_order_by_id to look up an order directly by its order ID
get_customer_orders to look up all the orders belonging to a customer
cancel_order cancels an order given a specific order ID

This chatbot will be quite limited, but it demonstrates some key pieces of working with multiple tools.

Here's an image showcasing an example interaction with the chatbot.

Here's another interaction with the chatbot where we've explicitly printed out a message any time Claude uses a tool, to make it easier to understand what's happening behind the scenes:

Here's a diagram showing the flow of information for a potential single chat message:

important note: this chatbot is extremely simple and allows anyone to cancel an order as long as they have the username or email of the user who placed the order. This implementation is for educational purposes and should not be integrated into a real codebase without alteration!

Our fake database

Before we start working with Claude, we'll begin by defining a very simple FakeDatabase class that will hold some fake customers and orders, as well as providing methods for us to interact with the data. In the real world, the chatbot would presumably be connected to one or more actual databases.

In [1]:

class FakeDatabase:
    def __init__(self):
        self.customers = [
            {"id": "1213210", "name": "John Doe", "email": "john@gmail.com", "phone": "123-456-7890", "username": "johndoe"},
            {"id": "2837622", "name": "Priya Patel", "email": "priya@candy.com", "phone": "987-654-3210", "username": "priya123"},
            {"id": "3924156", "name": "Liam Nguyen", "email": "lnguyen@yahoo.com", "phone": "555-123-4567", "username": "liamn"},
            {"id": "4782901", "name": "Aaliyah Davis", "email": "aaliyahd@hotmail.com", "phone": "111-222-3333", "username": "adavis"},
            {"id": "5190753", "name": "Hiroshi Nakamura", "email": "hiroshi@gmail.com", "phone": "444-555-6666", "username": "hiroshin"},
            {"id": "6824095", "name": "Fatima Ahmed", "email": "fatimaa@outlook.com", "phone": "777-888-9999", "username": "fatimaahmed"},
            {"id": "7135680", "name": "Alejandro Rodriguez", "email": "arodriguez@protonmail.com", "phone": "222-333-4444", "username": "alexr"},
            {"id": "8259147", "name": "Megan Anderson", "email": "megana@gmail.com", "phone": "666-777-8888", "username": "manderson"},
            {"id": "9603481", "name": "Kwame Osei", "email": "kwameo@yahoo.com", "phone": "999-000-1111", "username": "kwameo"},
            {"id": "1057426", "name": "Mei Lin", "email": "meilin@gmail.com", "phone": "333-444-5555", "username": "mlin"}
        ]

        self.orders = [
            {"id": "24601", "customer_id": "1213210", "product": "Wireless Headphones", "quantity": 1, "price": 79.99, "status": "Shipped"},
            {"id": "13579", "customer_id": "1213210", "product": "Smartphone Case", "quantity": 2, "price": 19.99, "status": "Processing"},
            {"id": "97531", "customer_id": "2837622", "product": "Bluetooth Speaker", "quantity": 1, "price": "49.99", "status": "Shipped"}, 
            {"id": "86420", "customer_id": "3924156", "product": "Fitness Tracker", "quantity": 1, "price": 129.99, "status": "Delivered"},
            {"id": "54321", "customer_id": "4782901", "product": "Laptop Sleeve", "quantity": 3, "price": 24.99, "status": "Shipped"},
            {"id": "19283", "customer_id": "5190753", "product": "Wireless Mouse", "quantity": 1, "price": 34.99, "status": "Processing"},
            {"id": "74651", "customer_id": "6824095", "product": "Gaming Keyboard", "quantity": 1, "price": 89.99, "status": "Delivered"},
            {"id": "30298", "customer_id": "7135680", "product": "Portable Charger", "quantity": 2, "price": 29.99, "status": "Shipped"},
            {"id": "47652", "customer_id": "8259147", "product": "Smartwatch", "quantity": 1, "price": 199.99, "status": "Processing"},
            {"id": "61984", "customer_id": "9603481", "product": "Noise-Cancelling Headphones", "quantity": 1, "price": 149.99, "status": "Shipped"},
            {"id": "58243", "customer_id": "1057426", "product": "Wireless Earbuds", "quantity": 2, "price": 99.99, "status": "Delivered"},
            {"id": "90357", "customer_id": "1213210", "product": "Smartphone Case", "quantity": 1, "price": 19.99, "status": "Shipped"},
            {"id": "28164", "customer_id": "2837622", "product": "Wireless Headphones", "quantity": 2, "price": 79.99, "status": "Processing"}
        ]

    def get_user(self, key, value):
        if key in {"email", "phone", "username"}:
            for customer in self.customers:
                if customer[key] == value:
                    return customer
            return f"Couldn't find a user with {key} of {value}"
        else:
            raise ValueError(f"Invalid key: {key}")

    def get_order_by_id(self, order_id):
        for order in self.orders:
            if order["id"] == order_id:
                return order
        return None
    
    def get_customer_orders(self, customer_id):
        return [order for order in self.orders if order["customer_id"] == customer_id]

    def cancel_order(self, order_id):
        order = self.get_order_by_id(order_id)
        if order:
            if order["status"] == "Processing":
                order["status"] = "Cancelled"
                return "Cancelled the order"
            else:
                return "Order has already shipped.  Can't cancel it."
        return "Can't find that order!"

We'll create an instance of the FakeDatabase:

In [2]:

db = FakeDatabase()

Let's make sure our get_user method works as intended. It's expecting us to pass in a key that is one of:

"email"
"username"
"phone"

It also expects a corresponding value that it will use to perform a basic search, hopefully returning the matching user.

In [3]:

db.get_user("email", "john@gmail.com")

Out [3]:

{'id': '1213210',
 'name': 'John Doe',
 'email': 'john@gmail.com',
 'phone': '123-456-7890',
 'username': 'johndoe'}

In [4]:

db.get_user("username", "adavis")

Out [4]:

{'id': '4782901',
 'name': 'Aaliyah Davis',
 'email': 'aaliyahd@hotmail.com',
 'phone': '111-222-3333',
 'username': 'adavis'}

In [5]:

db.get_user("phone", "666-777-8888")

Out [5]:

{'id': '8259147',
 'name': 'Megan Anderson',
 'email': 'megana@gmail.com',
 'phone': '666-777-8888',
 'username': 'manderson'}

We can also get a list of all orders that belong to a particular customer by calling get_customer_orders and passing in a customer ID:

In [6]:

db.get_customer_orders("1213210")

Out [6]:

[{'id': '24601',
  'customer_id': '1213210',
  'product': 'Wireless Headphones',
  'quantity': 1,
  'price': 79.99,
  'status': 'Shipped'},
 {'id': '13579',
  'customer_id': '1213210',
  'product': 'Smartphone Case',
  'quantity': 2,
  'price': 19.99,
  'status': 'Processing'},
 {'id': '90357',
  'customer_id': '1213210',
  'product': 'Smartphone Case',
  'quantity': 1,
  'price': 19.99,
  'status': 'Shipped'}]

In [7]:

db.get_customer_orders("9603481")

Out [7]:

[{'id': '61984',
  'customer_id': '9603481',
  'product': 'Noise-Cancelling Headphones',
  'quantity': 1,
  'price': 149.99,
  'status': 'Shipped'}]

We also can look up a single order directly if we have the order ID:

In [8]:

db.get_order_by_id('24601')

Out [8]:

{'id': '24601',
 'customer_id': '1213210',
 'product': 'Wireless Headphones',
 'quantity': 1,
 'price': 79.99,
 'status': 'Shipped'}

Lastly, we can use the cancel_order method to cancel an order. Orders can only be cancelled if they are "processing". We can't cancel an order that has already shipped! The method expects us to pass in an order_id of the order we want to cancel.

In [9]:

#Let's look up an order that has a status of processing:
db.get_order_by_id("47652")

Out [9]:

{'id': '47652',
 'customer_id': '8259147',
 'product': 'Smartwatch',
 'quantity': 1,
 'price': 199.99,
 'status': 'Processing'}

In [10]:

#Now let's cancel it! 
db.cancel_order("47652")

Out [10]:

'Cancelled the order'

In [11]:

# It's status should now be "Cancelled"
db.get_order_by_id("47652")

Out [11]:

{'id': '47652',
 'customer_id': '8259147',
 'product': 'Smartwatch',
 'quantity': 1,
 'price': 199.99,
 'status': 'Cancelled'}

Writing our tools

Now that we've tested our (very) simple fake database functionality, let's write JSON schemas that define the structure of the tools. Let's take a look at a very simple tool that corresponds to our get_order_by_id method:

In [12]:

tool1 = {
    "toolSpec": {
        "name": "get_order_by_id",
        "description": "Retrieves the details of a specific order based on the order ID. Returns the order ID, product name, quantity, price, and order status.",
        "inputSchema": {
            "json": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "string",
                        "description": "The unique identifier for the order."
                    }
                },
                "required": ["order_id"]
            }
        }
    }
    }

As always, we include a name, a description, and an overview of the inputs. In this case, there is a single input, order_id, and it is required.

Now let's take a look at a more complex tool that corresponds to the get_user method. Recall that this method has two arguments:

A key argument that is one of the following strings:
- "email"
- "username"
- "phone"
A value argument which is the term we'll be using to search (the actual email, phone number, or username)

Here's the corresponding tool definition:

In [13]:

tool2 = {
    "toolSpec": {
        "name": "get_user",
        "description": "Looks up a user by email, phone, or username.",
        "inputSchema": {
            "json": {
                "type": "object",
                "properties": {
                    "key": {
                        "type": "string",
                        "enum": ["email", "phone", "username"],
                        "description": "The attribute to search for a user by (email, phone, or username)."
                    },
                    "value": {
                        "type": "string",
                        "description": "The value to match for the specified attribute."
                    }
                },
                "required": ["key", "value"]
            }
            }
    }
    }

Pay special attention to the way we define the set of possible valid options for key using enum in the schema.

We still have two more tools to write, but to save time, we'll just provide you the complete list of tools ready for us to use. (You're welcome to try defining them youself first as an exercise!)

In [14]:

tools = [
    {
        "toolSpec": {
            "name": "get_user",
            "description": "Looks up a user by email, phone, or username.",
            "inputSchema": {
                "json": {     
                    "type": "object",
                    "properties": {
                        "key": {
                            "type": "string",
                            "enum": ["email", "phone", "username"],
                            "description": "The attribute to search for a user by (email, phone, or username)."
                        },
                        "value": {
                            "type": "string",
                            "description": "The value to match for the specified attribute."
                        }
                    },
                    "required": ["key", "value"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "get_order_by_id",
            "description": "Retrieves the details of a specific order based on the order ID. Returns the order ID, product name, quantity, price, and order status.",
            "inputSchema": {
                "json": {    
                    "type": "object",
                    "properties": {
                        "order_id": {
                            "type": "string",
                            "description": "The unique identifier for the order."
                        }
                    },
                    "required": ["order_id"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "get_customer_orders",
            "description": "Retrieves the list of orders belonging to a user based on a user's customer id.",
            "inputSchema": {
                "json": {     
                    "type": "object",
                    "properties": {
                        "customer_id": {
                            "type": "string",
                            "description": "The customer_id belonging to the user"
                        }
                    },
                    "required": ["customer_id"]
                }
            }
        }
    },
    {
        "toolSpec": {
            "name": "cancel_order",
            "description": "Cancels an order based on a provided order_id.  Only orders that are 'processing' can be cancelled",
            "inputSchema": {
                "json": {     
                    "type": "object",
                    "properties": {
                        "order_id": {
                            "type": "string",
                            "description": "The order_id pertaining to a particular order"
                        }
                    },
                    "required": ["order_id"]
                }
            }
        }
    }
]

Giving our tools to Claude

Next up, we need to do a few things:

Tell Claude about our tools and send the user's chat message to Claude.
Handle the response we get back from Claude:
- If Claude doesn't want to use a tool:
  - Print Claude's output to the user.
  - Ask the user for their next message.
- If Claude does want to use a tool
  - Verify that Claude called one of the appropriate tools.
  - Execute the underlying function, like get_user or cancel_order.
  - Send Claude the result of running the tool.

Eventually, the goal is to write a command line script that will create an interactive chatbot that will run in a loop. But to keep things simple, we'll start by writing the basic code linearly. We'll end by creating an interactive chatbot script.

We start with a function that can translate Claude's tool use response into an actual method call on our db:

In [15]:

def process_tool_call(tool_name, tool_input):
    if tool_name == "get_user":
        return db.get_user(tool_input["key"], tool_input["value"])
    elif tool_name == "get_order_by_id":
        return db.get_order_by_id(tool_input["order_id"])
    elif tool_name == "get_customer_orders":
        return db.get_customer_orders(tool_input["customer_id"])
    elif tool_name == "cancel_order":
        return db.cancel_order(tool_input["order_id"])

Let's start with a simple demo that shows Claude can decide which tool to use when presented with a list of multiple tools. We'll ask Claude Can you look up my orders? My email is john@gmail.com and see what happens!

In [16]:

import boto3

bedrock_client = boto3.client(service_name='bedrock-runtime', region_name="us-west-2")
model_id = "anthropic.claude-3-5-sonnet-20241022-v2:0"

messages = [{"role": "user", "content": [{"text": "Can you look up my orders? My email is john@gmail.com"}]}]

inference_config={"maxTokens":400}
tool_config = {"tools":tools}

# Send the message.
response = bedrock_client.converse(
    modelId=model_id,
    messages=messages,
    inferenceConfig=inference_config,
    toolConfig=tool_config,
)

In [17]:

print(response["stopReason"])

tool_use

Claude wants to use the get_user tool.

Now we'll write the basic logic to update our messages list, call the appropriate tool, and update our messages list again with the tool results.

In [18]:

import json 

# Update messages to include Claude's response
messages.append(response["output"]["message"])

print(response)

#If Claude stops because it wants to use a tool:
if response["stopReason"] == "tool_use":
    tool_use = response["output"]["message"]["content"][-1]["toolUse"] #Naive approach assumes only 1 tool is called at a time
    tool_name = tool_use["name"]
    tool_input = tool_use["input"]
    print("Claude wants to use the {tool_name} tool")
    print(f"Tool Input:")
    print(json.dumps(tool_input, indent=2))

    #Actually run the underlying tool functionality on our db
    tool_result = process_tool_call(tool_name, tool_input)

    print(f"\nTool Result:")
    print(json.dumps(tool_result, indent=2))

    #Add our tool_result message:
    tool_response = {
        "role": "user",
        "content": [
            {
                "toolResult": {
                    "toolUseId": tool_use['toolUseId'],
                    "content": [
                        {
                            "text": json.dumps(tool_result)
                        }
                    ]
                }
            }
        ]
    }
    
    messages.append(tool_response)
else: 
    #If Claude does NOT want to use a tool, just print out the text reponse
    print("\nTechNova Support:" + f"{response['output']['message']['content'][0]['text']}")

{'ResponseMetadata': {'RequestId': 'c0fa41c8-f176-4d42-80c6-7f91b730e9eb', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Thu, 14 Nov 2024 19:23:58 GMT', 'content-type': 'application/json', 'content-length': '483', 'connection': 'keep-alive', 'x-amzn-requestid': 'c0fa41c8-f176-4d42-80c6-7f91b730e9eb'}, 'RetryAttempts': 0}, 'output': {'message': {'role': 'assistant', 'content': [{'text': "I'll help you look up your orders. Let me first find your user information using your email, and then I can retrieve your orders.\n\nFirst, let's look up your user details:"}, {'toolUse': {'toolUseId': 'tooluse_fJ7fxaWMQmqlI8W3tRXfTw', 'name': 'get_user', 'input': {'key': 'email', 'value': 'john@gmail.com'}}}]}}, 'stopReason': 'tool_use', 'usage': {'inputTokens': 728, 'outputTokens': 114, 'totalTokens': 842}, 'metrics': {'latencyMs': 2726}}
Claude wants to use the {tool_name} tool
Tool Input:
{
  "key": "email",
  "value": "john@gmail.com"
}

Tool Result:
{
  "id": "1213210",
  "name": "John Doe",
  "email": "john@gmail.com",
  "phone": "123-456-7890",
  "username": "johndoe"
}

This is what our messages list looks like now:

In [19]:

messages

Out [19]:

[{'role': 'user',
  'content': [{'text': 'Can you look up my orders? My email is john@gmail.com'}]},
 {'role': 'assistant',
  'content': [{'text': "I'll help you look up your orders. Let me first find your user information using your email, and then I can retrieve your orders.\n\nFirst, let's look up your user details:"},
   {'toolUse': {'toolUseId': 'tooluse_fJ7fxaWMQmqlI8W3tRXfTw',
     'name': 'get_user',
     'input': {'key': 'email', 'value': 'john@gmail.com'}}}]},
 {'role': 'user',
  'content': [{'toolResult': {'toolUseId': 'tooluse_fJ7fxaWMQmqlI8W3tRXfTw',
     'content': [{'text': '{"id": "1213210", "name": "John Doe", "email": "john@gmail.com", "phone": "123-456-7890", "username": "johndoe"}'}]}}]}]

Now we'll send a second request to Claude using the updated messages list:

In [20]:

response2 = bedrock_client.converse(
    modelId=model_id,
    messages=messages,
    inferenceConfig=inference_config,
    toolConfig=tool_config,
)

In [21]:

response2

Out [21]:

{'ResponseMetadata': {'RequestId': 'c3ae3ff2-0715-43e7-a1a0-553e4dada84b',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Thu, 14 Nov 2024 19:24:00 GMT',
   'content-type': 'application/json',
   'content-length': '360',
   'connection': 'keep-alive',
   'x-amzn-requestid': 'c3ae3ff2-0715-43e7-a1a0-553e4dada84b'},
  'RetryAttempts': 0},
 'output': {'message': {'role': 'assistant',
   'content': [{'text': "Now, I'll retrieve your orders using your customer ID:"},
    {'toolUse': {'toolUseId': 'tooluse_Nb553Vc3SSOwD86cJID1SQ',
      'name': 'get_customer_orders',
      'input': {'customer_id': '1213210'}}}]}},
 'stopReason': 'tool_use',
 'usage': {'inputTokens': 898, 'outputTokens': 73, 'totalTokens': 971},
 'metrics': {'latencyMs': 1787}}

Now that Claude has the result of the get_user tool, including the user's ID, it wants to call get_customer_orders to look up the orders that correspond to this particular customer. It's picking the right tool for the job. Note: there are many potential issues and places Claude could go wrong here, but this is a start!

Writing an interactive script

It's a lot easier to interact with this chatbot via an interactive command line script.

Here's all of the code from above combined into a single function that starts a loop-based chat session (you'll probably want to run this as its own script from the command line):

In [22]:

import boto3

bedrock_client = boto3.client(service_name='bedrock-runtime', region_name="us-west-2")
model_id = "anthropic.claude-3-5-sonnet-20241022-v2:0"

class FakeDatabase:
    def __init__(self):
        self.customers = [
            {"id": "1213210", "name": "John Doe", "email": "john@gmail.com", "phone": "123-456-7890", "username": "johndoe"},
            {"id": "2837622", "name": "Priya Patel", "email": "priya@candy.com", "phone": "987-654-3210", "username": "priya123"},
            {"id": "3924156", "name": "Liam Nguyen", "email": "lnguyen@yahoo.com", "phone": "555-123-4567", "username": "liamn"},
            {"id": "4782901", "name": "Aaliyah Davis", "email": "aaliyahd@hotmail.com", "phone": "111-222-3333", "username": "adavis"},
            {"id": "5190753", "name": "Hiroshi Nakamura", "email": "hiroshi@gmail.com", "phone": "444-555-6666", "username": "hiroshin"},
            {"id": "6824095", "name": "Fatima Ahmed", "email": "fatimaa@outlook.com", "phone": "777-888-9999", "username": "fatimaahmed"},
            {"id": "7135680", "name": "Alejandro Rodriguez", "email": "arodriguez@protonmail.com", "phone": "222-333-4444", "username": "alexr"},
            {"id": "8259147", "name": "Megan Anderson", "email": "megana@gmail.com", "phone": "666-777-8888", "username": "manderson"},
            {"id": "9603481", "name": "Kwame Osei", "email": "kwameo@yahoo.com", "phone": "999-000-1111", "username": "kwameo"},
            {"id": "1057426", "name": "Mei Lin", "email": "meilin@gmail.com", "phone": "333-444-5555", "username": "mlin"}
        ]

        self.orders = [
            {"id": "24601", "customer_id": "1213210", "product": "Wireless Headphones", "quantity": 1, "price": 79.99, "status": "Shipped"},
            {"id": "13579", "customer_id": "1213210", "product": "Smartphone Case", "quantity": 2, "price": 19.99, "status": "Processing"},
            {"id": "97531", "customer_id": "2837622", "product": "Bluetooth Speaker", "quantity": 1, "price": "49.99", "status": "Shipped"}, 
            {"id": "86420", "customer_id": "3924156", "product": "Fitness Tracker", "quantity": 1, "price": 129.99, "status": "Delivered"},
            {"id": "54321", "customer_id": "4782901", "product": "Laptop Sleeve", "quantity": 3, "price": 24.99, "status": "Shipped"},
            {"id": "19283", "customer_id": "5190753", "product": "Wireless Mouse", "quantity": 1, "price": 34.99, "status": "Processing"},
            {"id": "74651", "customer_id": "6824095", "product": "Gaming Keyboard", "quantity": 1, "price": 89.99, "status": "Delivered"},
            {"id": "30298", "customer_id": "7135680", "product": "Portable Charger", "quantity": 2, "price": 29.99, "status": "Shipped"},
            {"id": "47652", "customer_id": "8259147", "product": "Smartwatch", "quantity": 1, "price": 199.99, "status": "Processing"},
            {"id": "61984", "customer_id": "9603481", "product": "Noise-Cancelling Headphones", "quantity": 1, "price": 149.99, "status": "Shipped"},
            {"id": "58243", "customer_id": "1057426", "product": "Wireless Earbuds", "quantity": 2, "price": 99.99, "status": "Delivered"},
            {"id": "90357", "customer_id": "1213210", "product": "Smartphone Case", "quantity": 1, "price": 19.99, "status": "Shipped"},
            {"id": "28164", "customer_id": "2837622", "product": "Wireless Headphones", "quantity": 2, "price": 79.99, "status": "Processing"}
        ]

    def get_user(self, key, value):
        if key in {"email", "phone", "username"}:
            for customer in self.customers:
                if customer[key] == value:
                    return customer
            return f"Couldn't find a user with {key} of {value}"
        else:
            raise ValueError(f"Invalid key: {key}")

    def get_order_by_id(self, order_id):
        for order in self.orders:
            if order["id"] == order_id:
                return order
        return None
    
    def get_customer_orders(self, customer_id):
        return [order for order in self.orders if order["customer_id"] == customer_id]

    def cancel_order(self, order_id):
        order = self.get_order_by_id(order_id)
        if order:
            if order["status"] == "Processing":
                order["status"] = "Cancelled"
                return "Cancelled the order"
            else:
                return "Order has already shipped.  Can't cancel it."
        return "Can't find that order!"

db = FakeDatabase()

def process_tool_call(tool_name, tool_input):
    if tool_name == "get_user":
        return db.get_user(tool_input["key"], tool_input["value"])
    elif tool_name == "get_order_by_id":
        return db.get_order_by_id(tool_input["order_id"])
    elif tool_name == "get_customer_orders":
        return db.get_customer_orders(tool_input["customer_id"])
    elif tool_name == "cancel_order":
        return db.cancel_order(tool_input["order_id"])
    
def simple_chat():
    user_message = input("\nUser: ")
    messages = [{"role": "user", "content": [{"text": user_message}]}]
    while True:
        #If the last message is from the assistant, get another input from the user
        if messages[-1].get("role") == "assistant":
            user_message = input("\nUser: ")
            messages.append({"role": "user", "content": [{"text": user_message}]})

        #Send a request to Claude
        response = bedrock_client.converse(
            modelId=model_id,
            messages=messages,
            toolConfig={"tools": tools},
        )
        # Update messages to include Claude's response
        messages.append(
            response["output"]["message"]
        )

        #If Claude stops because it wants to use a tool:
        if response["stopReason"] == "tool_use":
            tool_use = response["output"]["message"]["content"][-1]["toolUse"] #Naive approach assumes only 1 tool is called at a time
            tool_name = tool_use["name"]
            tool_input = tool_use["input"]
            print("Claude wants to use the {tool_name} tool")
            print(f"Tool Input:")
            print(json.dumps(tool_input, indent=2))

            #Actually run the underlying tool functionality on our db
            tool_result = process_tool_call(tool_name, tool_input)

            print(f"\nTool Result:")
            print(json.dumps(tool_result, indent=2))

            #Add our tool_result message:
            tool_response = {
                "role": "user",
                "content": [
                    {
                        "toolResult": {
                            "toolUseId": tool_use['toolUseId'],
                            "content": [
                                {
                                    "text": json.dumps(tool_result)
                                }
                            ]
                        }
                    }
                ]
            }
            
            messages.append(tool_response)
        else: 
            #If Claude does NOT want to use a tool, just print out the text reponse
            print("\nTechNova Support:" + f"{response['output']['message']['content'][0]['text']}")

# Start the chat!!
simple_chat()

[0;31m---------------------------------------------------------------------------[0m
[0;31mValidationException[0m                       Traceback (most recent call last)
Cell [0;32mIn[22], line 135[0m
[1;32m    132[0m             [38;5;28mprint[39m([38;5;124m"[39m[38;5;130;01m\n[39;00m[38;5;124mTechNova Support:[39m[38;5;124m"[39m [38;5;241m+[39m [38;5;124mf[39m[38;5;124m"[39m[38;5;132;01m{[39;00mresponse[[38;5;124m'[39m[38;5;124moutput[39m[38;5;124m'[39m][[38;5;124m'[39m[38;5;124mmessage[39m[38;5;124m'[39m][[38;5;124m'[39m[38;5;124mcontent[39m[38;5;124m'[39m][[38;5;241m0[39m][[38;5;124m'[39m[38;5;124mtext[39m[38;5;124m'[39m][38;5;132;01m}[39;00m[38;5;124m"[39m)
[1;32m    134[0m [38;5;66;03m# Start the chat!![39;00m
[0;32m--> 135[0m [43msimple_chat[49m[43m([49m[43m)[49m

Cell [0;32mIn[22], line 87[0m, in [0;36msimple_chat[0;34m()[0m
[1;32m     84[0m     messages[38;5;241m.[39mappend({[38;5;124m"[39m[38;5;124mrole[39m[38;5;124m"[39m: [38;5;124m"[39m[38;5;124muser[39m[38;5;124m"[39m, [38;5;124m"[39m[38;5;124mcontent[39m[38;5;124m"[39m: [{[38;5;124m"[39m[38;5;124mtext[39m[38;5;124m"[39m: user_message}]})
[1;32m     86[0m [38;5;66;03m#Send a request to Claude[39;00m
[0;32m---> 87[0m response [38;5;241m=[39m [43mbedrock_client[49m[38;5;241;43m.[39;49m[43mconverse[49m[43m([49m
[1;32m     88[0m [43m    [49m[43mmodelId[49m[38;5;241;43m=[39;49m[43mmodel_id[49m[43m,[49m
[1;32m     89[0m [43m    [49m[43mmessages[49m[38;5;241;43m=[39;49m[43mmessages[49m[43m,[49m
[1;32m     90[0m [43m    [49m[43mtoolConfig[49m[38;5;241;43m=[39;49m[43m{[49m[38;5;124;43m"[39;49m[38;5;124;43mtools[39;49m[38;5;124;43m"[39;49m[43m:[49m[43m [49m[43mtools[49m[43m}[49m[43m,[49m
[1;32m     91[0m [43m[49m[43m)[49m
[1;32m     92[0m [38;5;66;03m# Update messages to include Claude's response[39;00m
[1;32m     93[0m messages[38;5;241m.[39mappend(
[1;32m     94[0m     response[[38;5;124m"[39m[38;5;124moutput[39m[38;5;124m"[39m][[38;5;124m"[39m[38;5;124mmessage[39m[38;5;124m"[39m]
[1;32m     95[0m )

File [0;32m/opt/homebrew/Caskroom/miniforge/base/envs/py311/lib/python3.11/site-packages/botocore/client.py:569[0m, in [0;36mClientCreator._create_api_method.<locals>._api_call[0;34m(self, *args, **kwargs)[0m
[1;32m    565[0m     [38;5;28;01mraise[39;00m [38;5;167;01mTypeError[39;00m(
[1;32m    566[0m         [38;5;124mf[39m[38;5;124m"[39m[38;5;132;01m{[39;00mpy_operation_name[38;5;132;01m}[39;00m[38;5;124m() only accepts keyword arguments.[39m[38;5;124m"[39m
[1;32m    567[0m     )
[1;32m    568[0m [38;5;66;03m# The "self" in this scope is referring to the BaseClient.[39;00m
[0;32m--> 569[0m [38;5;28;01mreturn[39;00m [38;5;28;43mself[39;49m[38;5;241;43m.[39;49m[43m_make_api_call[49m[43m([49m[43moperation_name[49m[43m,[49m[43m [49m[43mkwargs[49m[43m)[49m

File [0;32m/opt/homebrew/Caskroom/miniforge/base/envs/py311/lib/python3.11/site-packages/botocore/client.py:1023[0m, in [0;36mBaseClient._make_api_call[0;34m(self, operation_name, api_params)[0m
[1;32m   1019[0m     error_code [38;5;241m=[39m error_info[38;5;241m.[39mget([38;5;124m"[39m[38;5;124mQueryErrorCode[39m[38;5;124m"[39m) [38;5;129;01mor[39;00m error_info[38;5;241m.[39mget(
[1;32m   1020[0m         [38;5;124m"[39m[38;5;124mCode[39m[38;5;124m"[39m
[1;32m   1021[0m     )
[1;32m   1022[0m     error_class [38;5;241m=[39m [38;5;28mself[39m[38;5;241m.[39mexceptions[38;5;241m.[39mfrom_code(error_code)
[0;32m-> 1023[0m     [38;5;28;01mraise[39;00m error_class(parsed_response, operation_name)
[1;32m   1024[0m [38;5;28;01melse[39;00m:
[1;32m   1025[0m     [38;5;28;01mreturn[39;00m parsed_response

[0;31mValidationException[0m: An error occurred (ValidationException) when calling the Converse operation: The text field in the ContentBlock object at messages.0.content.0 is blank. Add text to the text field, and try again.

Here's an example conversation:

As you can see, the chatbot is calling the correct tools when needed, but the actual chat responses are not ideal. We probably don't want a customer-facing chatbot telling users about the specific tools it's going to call!

Prompt enhancements

We can improve the chat experience by providing our chatbot with a solid system prompt. One of the first things we should do is give Claude some context and tell it that it's acting as a customer service assistant for TechNova.

Here's a start for our system prompt:

In [ ]:

system_prompt = """
You are a customer support chat bot for an online retailer called TechNova. 
Your job is to help users look up their account, orders, and cancel orders.
Be helpful and brief in your responses.
"""

Don't forget to pass this prompt into the system parameter when making requests to Claude!

Here's a sample conversation after adding in the above system prompt details:

Notice that the assistant knows it is a support bot for TechNova, and it no longer tells the user about its tools. It might also be worth explicitly telling Claude not to reference tools at all.

Note: The =======Claude wants to use the cancel_order_tool======= lines are logging we added to the script, not Claude's actual outputs!

If you play with this script enough, you'll likely notice other issues. One very obvious and very problematic issue is illustrated by the following exchange:

The screenshot above shows the entire conversation. The user asks for help canceling an order and states that they don't know the order ID. Claude should follow up and ask for the customer's email, phone number, or username to look up the customer and then find the matching orders. Instead, Claude just decides to call the get_user tool without actually knowing any information about the user. Claude made up an email address and tried to use it, but of course didn't find a matching customer.

To prevent this, we'll update the system prompt to include language along the lines of:

In [ ]:

system_prompt = """
You are a customer support chat bot for an online retailer called TechNova. 
Your job is to help users look up their account, orders, and cancel orders.
Be helpful and brief in your responses.
You have access to a set of tools, but only use them when needed.  
If you do not have enough information to use a tool correctly, ask a user follow up questions to get the required inputs.
Do not call any of the tools unless you have the required data from a user. 
"""

Here's a screenshot of a conversation with the assistant that uses the updated system prompt:

Much better!

Final version

Here's a screenshot of a longer conversation with a version of this script that colorizes the output.

Closing notes

This is an educational demo made to illustrate the tool use workflow. This script and prompt are NOT ready for production. The assistant responds with things like "You can find the order ID on the order confirmation email" without actually knowing if that's true. In the real world, we would need to provide Claude with lots of background knowledge about our particular company (at a bare minimum). We also need to rigorously test the chatbot and make sure it behaves well in all situations. Also, it's probably a bad idea to make it so easy for anyone to cancel an order! It would be pretty easy to cancel a lot of people's orders if you had access to their email, username, or phone number. In the real world, we would probably want some form of authentication. Long story short: this is a demo and not a complete chatbot implementation!

Exercise

Add functionality to our assistant (and underlying FakeDatabase class) that allows a user to update their email and phone number.
Define a new tool called get_user_info that combines the functionality of get_user and get_customer_orders. This tool should take a user's email, phone, or username as input and return the user's information along with their order history all in one go.
Add error handling and input validation!

Bonus

Update the chatbot to use an ACTUAL database instead of our FakeDatabase class.

2.6 MiB Raw Permalink Blame History