llama-stack-mirror/docs/source/building_applications/tools.md

Tools

Tools are functions that can be invoked by an agent to perform tasks. They are organized into tool groups and registered with specific providers. Each tool group represents a collection of related tools from a single provider. They are organized into groups so that state can be externalized: the collection operates on the same state typically. An example of this would be a "db_access" tool group that contains tools for interacting with a database. "list_tables", "query_table", "insert_row" could be examples of tools in this group.

Tools are treated as any other resource in llama stack like models. You can register them, have providers for them etc.

When instatiating an agent, you can provide it a list of tool groups that it has access to. Agent gets the corresponding tool definitions for the specified tool groups and passes them along to the model.

Refer to the Building AI Applications notebook for more examples on how to use tools.

Types of Tool Group providers

There are three types of providers for tool groups that are supported by Llama Stack.

1. Built-in providers
2. Model Context Protocol (MCP) providers
3. Client provided tools

Built-in providers

Built-in providers come packaged with Llama Stack. These providers provide common functionalities like web search, code interpretation, and computational capabilities.

Web Search providers

There are three web search providers that are supported by Llama Stack.

1. Brave Search
2. Bing Search
3. Tavily Search

Example client SDK call to register a "websearch" toolgroup that is provided by brave-search.

# Register Brave Search tool group
client.toolgroups.register(
    toolgroup_id="builtin::websearch",
    provider_id="brave-search",
    args={"max_results": 5},
)

The tool requires an API key which can be provided either in the configuration or through the request header X-LlamaStack-Provider-Data. The format of the header is {"<provider_name>_api_key": <your api key>}.

Code Interpreter

The Code Interpreter allows execution of Python code within a controlled environment.

# Register Code Interpreter tool group
client.toolgroups.register(
    toolgroup_id="builtin::code_interpreter", provider_id="code_interpreter"
)

Features:

• Secure execution environment using bwrap sandboxing
• Matplotlib support for generating plots
• Disabled dangerous system operations
• Configurable execution timeouts

⚠️ Important: The code interpreter tool can operate in a controlled enviroment locally or on Podman containers. To ensure proper functionality in containerised environments:

• The container requires privileged access (e.g., --privileged).
• Users without sufficient permissions may encounter permission errors. (bwrap: Can't mount devpts on /newroot/dev/pts: Permission denied)
• 🔒 Security Warning: Privileged mode grants elevated access and bypasses security restrictions. Use only in local, isolated, or controlled environments.

WolframAlpha

The WolframAlpha tool provides access to computational knowledge through the WolframAlpha API.

# Register WolframAlpha tool group
client.toolgroups.register(
    toolgroup_id="builtin::wolfram_alpha", provider_id="wolfram-alpha"
)

Example usage:

result = client.tool_runtime.invoke_tool(
    tool_name="wolfram_alpha", args={"query": "solve x^2 + 2x + 1 = 0"}
)

Memory

The Memory tool enables retrieval of context from various types of memory banks (vector, key-value, keyword, and graph).

# Register Memory tool group
client.toolgroups.register(
    toolgroup_id="builtin::memory",
    provider_id="memory",
    args={"max_chunks": 5, "max_tokens_in_context": 4096},
)

Features:

• Support for multiple memory bank types
• Configurable query generation
• Context retrieval with token limits

Note: By default, llama stack run.yaml defines toolgroups for web search, code interpreter and memory, that are provided by tavily-search, code-interpreter and memory providers.

Model Context Protocol (MCP) Tools

MCP tools are special tools that can interact with llama stack over model context protocol. These tools are dynamically discovered from an MCP endpoint and can be used to extend the agent's capabilities.

Refer to https://github.com/modelcontextprotocol/servers for available MCP servers.

# Register MCP tools
client.toolgroups.register(
    toolgroup_id="builtin::filesystem",
    provider_id="model-context-protocol",
    mcp_endpoint=URL(uri="http://localhost:8000/sse"),
)

MCP tools require:

• A valid MCP endpoint URL
• The endpoint must implement the Model Context Protocol
• Tools are discovered dynamically from the endpoint

Tools provided by the client

These tools are registered along with the agent config and are specific to the agent for which they are registered. The main difference between these tools and the tools provided by the built-in providers is that the execution of these tools is handled by the client and the agent transfers the tool call to the client and waits for the result from the client.

# Example agent config with client provided tools
config = AgentConfig(
    toolgroups=[
        "builtin::websearch",
    ],
    client_tools=[ToolDef(name="client_tool", description="Client provided tool")],
)

Refer to llama-stack-apps for an example of how to use client provided tools.

Tool Structure

Each tool has the following components:

• name: Unique identifier for the tool
• description: Human-readable description of the tool's functionality
• parameters: List of parameters the tool accepts
  • name: Parameter name
  • parameter_type: Data type (string, number, etc.)
  • description: Parameter description
  • required: Whether the parameter is required (default: true)
  • default: Default value if any

Example tool definition:

{
    "name": "web_search",
    "description": "Search the web for information",
    "parameters": [
        {
            "name": "query",
            "parameter_type": "string",
            "description": "The query to search for",
            "required": True,
        }
    ],
}

Tool Invocation

Tools can be invoked using the invoke_tool method:

result = client.tool_runtime.invoke_tool(
    tool_name="web_search", kwargs={"query": "What is the capital of France?"}
)

The result contains:

• content: The tool's output
• error_message: Optional error message if the tool failed
• error_code: Optional error code if the tool failed

Listing Available Tools

You can list all available tools or filter by tool group:

# List all tools
all_tools = client.tools.list_tools()

# List tools in a specific group
group_tools = client.tools.list_tools(toolgroup_id="search_tools")

Simple Example: Using an Agent with the Code-Interpreter Tool

from llama_stack_client.lib.agents.agent import Agent
from llama_stack_client.types.agent_create_params import AgentConfig

# Configure the AI agent with necessary parameters
agent_config = AgentConfig(
    name="code-interpreter",
    description="A code interpreter agent for executing Python code snippets",
    instructions="""
    You are a highly reliable, concise, and precise assistant.
    Always show the generated code, never generate your own code, and never anticipate results.
    """,
    model="meta-llama/Llama-3.2-3B-Instruct",
    toolgroups=["builtin::code_interpreter"],
    max_infer_iters=5,
    enable_session_persistence=False,
)

# Instantiate the AI agent with the given configuration
agent = Agent(client, agent_config)

# Start a session
session_id = agent.create_session("tool_session")

# Send a query to the AI agent for code execution
response = agent.create_turn(
    messages=[{"role": "user", "content": "Run this code: print(3 ** 4 - 5 * 2)"}],
    session_id=session_id,
)