llama-stack-mirror/docs/docs/building_applications/tools.mdx

---
title: Tools
description: Extend agent capabilities with external tools and function calling
sidebar_label: Tools
sidebar_position: 6
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Tools

Tools are functions that can be invoked by an agent to perform tasks. They are organized into tool groups and registered with specific providers. Each tool group represents a collection of related tools from a single provider. They are organized into groups so that state can be externalized: the collection operates on the same state typically.

An example of this would be a "db_access" tool group that contains tools for interacting with a database. "list_tables", "query_table", "insert_row" could be examples of tools in this group.

Tools are treated as any other resource in llama stack like models. You can register them, have providers for them etc.

When instantiating an agent, you can provide it a list of tool groups that it has access to. Agent gets the corresponding tool definitions for the specified tool groups and passes them along to the model.

Refer to the [Building AI Applications](https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.ipynb) notebook for more examples on how to use tools.

## Server-side vs. Client-side Tool Execution

Llama Stack allows you to use both server-side and client-side tools. With server-side tools, `agent.create_turn` can perform execution of the tool calls emitted by the model transparently giving the user the final answer desired. If client-side tools are provided, the tool call is sent back to the user for execution and optional continuation using the `agent.resume_turn` method.

## Server-side Tools

Llama Stack provides built-in providers for some common tools. These include web search, math, and RAG capabilities.

### Web Search

You have three providers to execute the web search tool calls generated by a model: Brave Search, Bing Search, and Tavily Search.

To indicate that the web search tool calls should be executed by brave-search, you can point the "builtin::websearch" toolgroup to the "brave-search" provider.

```python
client.toolgroups.register(
    toolgroup_id="builtin::websearch",
    provider_id="brave-search",
    args={"max_results": 5},
)
```

The tool requires an API key which can be provided either in the configuration or through the request header `X-LlamaStack-Provider-Data`. The format of the header is:
```
{"<provider_name>_api_key": <your api key>}
```

### Math

The WolframAlpha tool provides access to computational knowledge through the WolframAlpha API.

```python
client.toolgroups.register(
    toolgroup_id="builtin::wolfram_alpha",
    provider_id="wolfram-alpha"
)
```

Example usage:
```python
result = client.tool_runtime.invoke_tool(
    tool_name="wolfram_alpha",
    args={"query": "solve x^2 + 2x + 1 = 0"}
)
```

### RAG

The RAG tool enables retrieval of context from various types of memory banks (vector, key-value, keyword, and graph).

```python
# Register Memory tool group
client.toolgroups.register(
    toolgroup_id="builtin::rag",
    provider_id="faiss",
    args={"max_chunks": 5, "max_tokens_in_context": 4096},
)
```

Features:
- Support for multiple memory bank types
- Configurable query generation
- Context retrieval with token limits

:::note[Default Configuration]
By default, llama stack run.yaml defines toolgroups for web search, wolfram alpha and rag, that are provided by tavily-search, wolfram-alpha and rag providers.
:::

## Model Context Protocol (MCP)

[MCP](https://github.com/modelcontextprotocol) is an upcoming, popular standard for tool discovery and execution. It is a protocol that allows tools to be dynamically discovered from an MCP endpoint and can be used to extend the agent's capabilities.

### Using Remote MCP Servers

You can find some popular remote MCP servers [here](https://github.com/jaw9c/awesome-remote-mcp-servers). You can register them as toolgroups in the same way as local providers.

```python
client.toolgroups.register(
    toolgroup_id="mcp::deepwiki",
    provider_id="model-context-protocol",
    mcp_endpoint=URL(uri="https://mcp.deepwiki.com/sse"),
)
```

Note that most of the more useful MCP servers need you to authenticate with them. Many of them use OAuth2.0 for authentication. You can provide authorization headers to send to the MCP server using the "Provider Data" abstraction provided by Llama Stack. When making an agent call,

```python
agent = Agent(
    ...,
    tools=["mcp::deepwiki"],
    extra_headers={
        "X-LlamaStack-Provider-Data": json.dumps(
            {
                "mcp_headers": {
                    "http://mcp.deepwiki.com/sse": {
                        "Authorization": "Bearer <your_access_token>",
                    },
                },
            }
        ),
    },
)
agent.create_turn(...)
```

### Running Your Own MCP Server

Here's an example of how to run a simple MCP server that exposes a File System as a set of tools to the Llama Stack agent.

<Tabs>
<TabItem value="setup" label="Server Setup">

```shell
# Start your MCP server
mkdir /tmp/content
touch /tmp/content/foo
touch /tmp/content/bar
npx -y supergateway --port 8000 --stdio 'npx -y @modelcontextprotocol/server-filesystem /tmp/content'
```

</TabItem>
<TabItem value="register" label="Registration">

```python
# Register the MCP server as a tool group
client.toolgroups.register(
    toolgroup_id="mcp::filesystem",
    provider_id="model-context-protocol",
    mcp_endpoint=URL(uri="http://localhost:8000/sse"),
)
```

</TabItem>
</Tabs>

## Adding Custom (Client-side) Tools

When you want to use tools other than the built-in tools, you just need to implement a python function with a docstring. The content of the docstring will be used to describe the tool and the parameters and passed along to the generative model.

```python
# Example tool definition
def my_tool(input: int) -> int:
    """
    Runs my awesome tool.

    :param input: some int parameter
    """
    return input * 2
```

:::tip[Documentation Best Practices]
We employ python docstrings to describe the tool and the parameters. It is important to document the tool and the parameters so that the model can use the tool correctly. It is recommended to experiment with different docstrings to see how they affect the model's behavior.
:::

Once defined, simply pass the tool to the agent config. `Agent` will take care of the rest (calling the model with the tool definition, executing the tool, and returning the result to the model for the next iteration).

```python
# Example agent config with client provided tools
agent = Agent(client, ..., tools=[my_tool])
```

Refer to [llama-stack-apps](https://github.com/meta-llama/llama-stack-apps/) for an example of how to use client provided tools.

## Tool Invocation

Tools can be invoked using the `invoke_tool` method:

```python
result = client.tool_runtime.invoke_tool(
    tool_name="web_search",
    kwargs={"query": "What is the capital of France?"}
)
```

The result contains:
- `content`: The tool's output
- `error_message`: Optional error message if the tool failed
- `error_code`: Optional error code if the tool failed

## Listing Available Tools

You can list all available tools or filter by tool group:

```python
# List all tools
all_tools = client.tools.list_tools()

# List tools in a specific group
group_tools = client.tools.list_tools(toolgroup_id="search_tools")
```

## Complete Examples

### Web Search Agent

<Tabs>
<TabItem value="setup" label="Setup & Configuration">

1. Start by registering a Tavily API key at [Tavily](https://tavily.com/).
2. [Optional] Provide the API key directly to the Llama Stack server
```bash
export TAVILY_SEARCH_API_KEY="your key"
```
```bash
--env TAVILY_SEARCH_API_KEY=${TAVILY_SEARCH_API_KEY}
```

</TabItem>
<TabItem value="implementation" label="Implementation">

```python
from llama_stack_client.lib.agents.agent import Agent
from llama_stack_client.types.agent_create_params import AgentConfig
from llama_stack_client.lib.agents.event_logger import EventLogger
from llama_stack_client import LlamaStackClient

client = LlamaStackClient(
    base_url=f"http://localhost:8321",
    provider_data={
        "tavily_search_api_key": "your_TAVILY_SEARCH_API_KEY"
    },  # Set this from the client side. No need to provide it if it has already been configured on the Llama Stack server.
)

agent = Agent(
    client,
    model="meta-llama/Llama-3.2-3B-Instruct",
    instructions=(
        "You are a web search assistant, must use websearch tool to look up the most current and precise information available. "
    ),
    tools=["builtin::websearch"],
)

session_id = agent.create_session("websearch-session")

response = agent.create_turn(
    messages=[
        {"role": "user", "content": "How did the USA perform in the last Olympics?"}
    ],
    session_id=session_id,
)
for log in EventLogger().log(response):
    log.print()
```

</TabItem>
</Tabs>

### WolframAlpha Math Agent

<Tabs>
<TabItem value="setup" label="Setup & Configuration">

1. Start by registering for a WolframAlpha API key at [WolframAlpha Developer Portal](https://developer.wolframalpha.com/access).
2. Provide the API key either when starting the Llama Stack server:
    ```bash
    --env WOLFRAM_ALPHA_API_KEY=${WOLFRAM_ALPHA_API_KEY}
    ```
    or from the client side:
    ```python
    client = LlamaStackClient(
        base_url="http://localhost:8321",
        provider_data={"wolfram_alpha_api_key": wolfram_api_key},
    )
    ```

</TabItem>
<TabItem value="implementation" label="Implementation">

```python
# Configure the tools in the Agent by setting tools=["builtin::wolfram_alpha"]
agent = Agent(
    client,
    model="meta-llama/Llama-3.2-3B-Instruct",
    instructions="You are a mathematical assistant that can solve complex equations.",
    tools=["builtin::wolfram_alpha"],
)

session_id = agent.create_session("math-session")

# Example user query
response = agent.create_turn(
    messages=[{"role": "user", "content": "Solve x^2 + 2x + 1 = 0 using WolframAlpha"}],
    session_id=session_id,
)
```

</TabItem>
</Tabs>

## Best Practices

### 🛠️ **Tool Selection**
- Use **server-side tools** for production applications requiring reliability and security
- Use **client-side tools** for development, prototyping, or specialized integrations
- Combine multiple tool types for comprehensive functionality

### 📝 **Documentation**
- Write clear, detailed docstrings for custom tools
- Include parameter descriptions and expected return types
- Test tool descriptions with the model to ensure proper usage

### 🔐 **Security**
- Store API keys securely using environment variables or secure configuration
- Use the `X-LlamaStack-Provider-Data` header for dynamic authentication
- Validate tool inputs and outputs for security

### 🔄 **Error Handling**
- Implement proper error handling in custom tools
- Use structured error responses with meaningful messages
- Monitor tool performance and reliability

## Related Resources

- **[Agents](./agent)** - Building intelligent agents with tools
- **[RAG (Retrieval Augmented Generation)](./rag)** - Using knowledge retrieval tools
- **[Agent Execution Loop](./agent_execution_loop)** - Understanding tool execution flow
- **[Building AI Applications Notebook](https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.ipynb)** - Comprehensive examples
- **[Llama Stack Apps Examples](https://github.com/meta-llama/llama-stack-apps)** - Real-world tool implementations