mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-03 19:57:35 +00:00
c71ce8df61
# What does this PR do? - Migrates the remaining documentation sections to the new documentation format <!-- Provide a short summary of what this PR does and why. Link to relevant issues if applicable. --> <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> ## Test Plan - Partial migration <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* -->
112 lines
3.2 KiB
Text
112 lines
3.2 KiB
Text
---
|
|
title: Agents
|
|
description: Build powerful AI applications with the Llama Stack agent framework
|
|
sidebar_label: Agents
|
|
sidebar_position: 3
|
|
---
|
|
|
|
import Tabs from '@theme/Tabs';
|
|
import TabItem from '@theme/TabItem';
|
|
|
|
# Agents
|
|
|
|
An Agent in Llama Stack is a powerful abstraction that allows you to build complex AI applications.
|
|
|
|
The Llama Stack agent framework is built on a modular architecture that allows for flexible and powerful AI applications. This document explains the key components and how they work together.
|
|
|
|
## Core Concepts
|
|
|
|
### 1. Agent Configuration
|
|
|
|
Agents are configured using the `AgentConfig` class, which includes:
|
|
|
|
- **Model**: The underlying LLM to power the agent
|
|
- **Instructions**: System prompt that defines the agent's behavior
|
|
- **Tools**: Capabilities the agent can use to interact with external systems
|
|
- **Safety Shields**: Guardrails to ensure responsible AI behavior
|
|
|
|
```python
|
|
from llama_stack_client import Agent
|
|
|
|
# Create the agent
|
|
agent = Agent(
|
|
llama_stack_client,
|
|
model="meta-llama/Llama-3-70b-chat",
|
|
instructions="You are a helpful assistant that can use tools to answer questions.",
|
|
tools=["builtin::code_interpreter", "builtin::rag/knowledge_search"],
|
|
)
|
|
```
|
|
|
|
### 2. Sessions
|
|
|
|
Agents maintain state through sessions, which represent a conversation thread:
|
|
|
|
```python
|
|
# Create a session
|
|
session_id = agent.create_session(session_name="My conversation")
|
|
```
|
|
|
|
### 3. Turns
|
|
|
|
Each interaction with an agent is called a "turn" and consists of:
|
|
|
|
- **Input Messages**: What the user sends to the agent
|
|
- **Steps**: The agent's internal processing (inference, tool execution, etc.)
|
|
- **Output Message**: The agent's response
|
|
|
|
<Tabs>
|
|
<TabItem value="streaming" label="Streaming Response">
|
|
|
|
```python
|
|
from llama_stack_client import AgentEventLogger
|
|
|
|
# Create a turn with streaming response
|
|
turn_response = agent.create_turn(
|
|
session_id=session_id,
|
|
messages=[{"role": "user", "content": "Tell me about Llama models"}],
|
|
)
|
|
for log in AgentEventLogger().log(turn_response):
|
|
log.print()
|
|
```
|
|
|
|
</TabItem>
|
|
<TabItem value="non-streaming" label="Non-Streaming Response">
|
|
|
|
```python
|
|
from rich.pretty import pprint
|
|
|
|
# Non-streaming API
|
|
response = agent.create_turn(
|
|
session_id=session_id,
|
|
messages=[{"role": "user", "content": "Tell me about Llama models"}],
|
|
stream=False,
|
|
)
|
|
print("Inputs:")
|
|
pprint(response.input_messages)
|
|
print("Output:")
|
|
pprint(response.output_message.content)
|
|
print("Steps:")
|
|
pprint(response.steps)
|
|
```
|
|
|
|
</TabItem>
|
|
</Tabs>
|
|
|
|
### 4. Steps
|
|
|
|
Each turn consists of multiple steps that represent the agent's thought process:
|
|
|
|
- **Inference Steps**: The agent generating text responses
|
|
- **Tool Execution Steps**: The agent using tools to gather information
|
|
- **Shield Call Steps**: Safety checks being performed
|
|
|
|
## Agent Execution Loop
|
|
|
|
Refer to the [Agent Execution Loop](./agent_execution_loop) for more details on what happens within an agent turn.
|
|
|
|
## Related Resources
|
|
|
|
- **[Agent Execution Loop](./agent_execution_loop)** - Understanding the internal processing flow
|
|
- **[RAG (Retrieval Augmented Generation)](./rag)** - Building knowledge-enhanced agents
|
|
- **[Tools Integration](./tools)** - Extending agent capabilities with external tools
|
|
- **[Safety Guardrails](./safety)** - Implementing responsible AI practices
|