phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-11 19:56:03 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/docs/notebooks/llamastack_agents_getting_started_examples.ipynb

1031 lines
36 KiB
Text
Raw Blame History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Llama Stack Agents - Complete Guide\n",
"\n",
"This notebook provides a comprehensive guide to building agents with Llama Stack, covering:\n",
"\n",
"1. **Basic Agent Example** - Simple agent creation and conversation\n",
"2. **Advanced Agent Features** - RAG, multi-turn conversations\n",
"3. **MCP Tools Integration** - Model Context Protocol tools for extended capabilities\n",
"\n",
"## Prerequisites\n",
"\n",
"Before running this notebook, ensure:\n",
"- ✅ Llama Stack server is running: `llama stack run starter --port 8321`\n",
"- ✅ Ollama (or vllm) is running\n",
"- ✅ A model is available: `llama3.3:70b`\n",
"\n",
"## Setup"
]
},
{
"cell_type": "code",
"execution_count": 119,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Client initialized successfully!\n",
" Base URL: http://localhost:8321\n"
]
}
],
"source": [
"# Import required libraries\n",
"import json\n",
"from typing import Any, Dict\n",
"\n",
"from llama_stack_client import LlamaStackClient, Agent\n",
"from llama_stack_client.types import UserMessage\n",
"\n",
"# Initialize client\n",
"client = LlamaStackClient(base_url=\"http://localhost:8321\")\n",
"\n",
"print(\"✅ Client initialized successfully!\")\n",
"print(f\" Base URL: http://localhost:8321\")"
]
},
{
"cell_type": "code",
"execution_count": 120,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Created agent successfully\n"
]
}
],
"source": [
"# Create a basic agent using the Agent class\n",
"agent = Agent(\n",
" client=client,\n",
" model=\"ollama/llama3.3:70b\",\n",
" instructions=\"You are a helpful AI assistant that can answer questions and help with tasks.\",\n",
")\n",
"\n",
"print(\"✅ Created agent successfully\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"# Part 1: Basic Agent Example\n",
"\n",
"Let's start with a simple agent that can answer questions. This demonstrates:\n",
"- Agent creation with basic configuration\n",
"- Session management\n",
"- Streaming responses"
]
},
{
"cell_type": "code",
"execution_count": 121,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/conversations \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Created session: conv_fc6e84e0499f522e30a4c91e6a3f13d6f03fa326386c403b\n"
]
}
],
"source": [
"# Create agent session\n",
"basic_session_id = agent.create_session(session_name=\"basic_example_session\")\n",
"\n",
"print(f\"✅ Created session: {basic_session_id}\")"
]
},
{
"cell_type": "code",
"execution_count": 122,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/responses \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"User: What is the capital of France? Please explain briefly.\n",
"\n",
"Assistant: The capital of France is Paris. It's a city known for its iconic landmarks like the Eiffel Tower, art museums, and historic architecture, serving as the country's political, cultural, and economic center.The capital of France is Paris. It's a city known for its iconic landmarks like the Eiffel Tower, art museums, and historic architecture, serving as the country's political, cultural, and economic center.\n",
"\n",
"✅ Response captured: 204 characters\n"
]
}
],
"source": [
"# Send a message to the agent with streaming\n",
"query = \"What is the capital of France? Please explain briefly.\"\n",
"\n",
"print(f\"User: {query}\\n\")\n",
"print(\"Assistant: \", end='')\n",
"\n",
"# Create a turn with streaming\n",
"response = agent.create_turn(\n",
" session_id=basic_session_id,\n",
" messages=[UserMessage(content=query, role=\"user\")],\n",
" stream=True,\n",
")\n",
"\n",
"# Stream the response\n",
"output_text = \"\"\n",
"for chunk in response:\n",
" if chunk.event.event_type == \"turn_completed\":\n",
" output_text = chunk.event.final_text\n",
" print(output_text)\n",
" break\n",
" elif chunk.event.event_type == \"step_progress\":\n",
" # Print text deltas as they arrive\n",
" if hasattr(chunk.event.delta, 'text'):\n",
" print(chunk.event.delta.text, end='', flush=True)\n",
"\n",
"print(f\"\\n✅ Response captured: {len(output_text)} characters\")"
]
},
{
"cell_type": "code",
"execution_count": 123,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: DELETE http://localhost:8321/v1/conversations/conv_fc6e84e0499f522e30a4c91e6a3f13d6f03fa326386c403b \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Session cleaned up\n"
]
}
],
"source": [
"# Clean up the session\n",
"client.conversations.delete(conversation_id=basic_session_id)\n",
"print(\"✅ Session cleaned up\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"# Part 2: Advanced Agent Features\n",
"\n",
"Now let's explore more advanced capabilities:\n",
"- Multi-turn conversations with context memory\n",
"- RAG (Retrieval-Augmented Generation) patterns"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2.1 Multi-Turn Conversation\n",
"\n",
"Demonstrate how agents can maintain context across multiple conversation turns."
]
},
{
"cell_type": "code",
"execution_count": 124,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Created conversation agent\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/conversations \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Created session: conv_53b1fa277c2b51b59846a499a4a8f4fb264c86d50fe03b30\n"
]
}
],
"source": [
"# Create agent for multi-turn conversation\n",
"conv_agent = Agent(\n",
" client=client,\n",
" model=\"ollama/llama3.3:70b\",\n",
" instructions=\"You are a helpful assistant that remembers context from previous messages.\",\n",
")\n",
"\n",
"print(\"✅ Created conversation agent\")\n",
"\n",
"conv_session_id = conv_agent.create_session(session_name=\"multi_turn_session\")\n",
"print(f\"✅ Created session: {conv_session_id}\")"
]
},
{
"cell_type": "code",
"execution_count": 125,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"============================================================\n",
"Turn 1\n",
"============================================================\n",
"User: My name is Alice and I'm learning about AI.\n",
"Assistant: "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/responses \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Nice to meet you, Alice! Learning about AI can be a fascinating topic. There's so much to explore, from machine learning and natural language processing to computer vision and robotics. What specific area of AI are you most interested in or would you like to start with the basics? I'm here to help and provide guidance as you learn.Nice to meet you, Alice! Learning about AI can be a fascinating topic. There's so much to explore, from machine learning and natural language processing to computer vision and robotics. What specific area of AI are you most interested in or would you like to start with the basics? I'm here to help and provide guidance as you learn.\n",
"\n",
"============================================================\n",
"Turn 2\n",
"============================================================\n",
"User: What are some good resources for beginners?\n",
"Assistant: "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/responses \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"As a beginner, it's essential to start with resources that are easy to understand and provide a solid foundation in AI concepts. Here are some popular resources:\n",
"\n",
"**Online Courses:**\n",
"\n",
"1. **Coursera - Machine Learning by Andrew Ng**: A fantastic course that covers the basics of machine learning.\n",
"2. **edX - Introduction to Artificial Intelligence**: A broad introduction to AI, covering topics like computer vision, robotics, and natural language processing.\n",
"3. **Udemy - Artificial Intelligence for Beginners**: A beginner-friendly course that explores AI fundamentals.\n",
"\n",
"**Websites and Blogs:**\n",
"\n",
"1. **Towards Data Science**: A great blog that features articles on machine learning, AI, and data science.\n",
"2. **AI Alignment Forum**: A website that discusses the latest developments in AI research and applications.\n",
"3. **KDnuggets**: A popular blog that covers AI, machine learning, and data science topics.\n",
"\n",
"**Books:**\n",
"\n",
"1. **\"Artificial Intelligence: A Modern Approach\" by Stuart Russell and Peter Norvig**: A comprehensive textbook on AI.\n",
"2. **\"Deep Learning\" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville**: A book that focuses on deep learning techniques.\n",
"3. **\"Life 3.0: Being Human in the Age of Artificial Intelligence\" by Max Tegmark**: A thought-provoking book that explores the future of AI.\n",
"\n",
"**YouTube Channels:**\n",
"\n",
"1. **3Blue1Brown (Grant Sanderson)**: Animated explanations of machine learning and AI concepts.\n",
"2. **Crash Course - Artificial Intelligence**: Engaging video lessons on AI basics.\n",
"3. **Siraj Raval**: A channel that offers explanations of AI and machine learning concepts.\n",
"\n",
"Remember, Alice, these resources are just a starting point. As you progress in your learning journey, you'll discover more advanced topics and specialized areas within AI.\n",
"\n",
"What do you think? Is there a particular resource or topic that catches your attention?As a beginner, it's essential to start with resources that are easy to understand and provide a solid foundation in AI concepts. Here are some popular resources:\n",
"\n",
"**Online Courses:**\n",
"\n",
"1. **Coursera - Machine Learning by Andrew Ng**: A fantastic course that covers the basics of machine learning.\n",
"2. **edX - Introduction to Artificial Intelligence**: A broad introduction to AI, covering topics like computer vision, robotics, and natural language processing.\n",
"3. **Udemy - Artificial Intelligence for Beginners**: A beginner-friendly course that explores AI fundamentals.\n",
"\n",
"**Websites and Blogs:**\n",
"\n",
"1. **Towards Data Science**: A great blog that features articles on machine learning, AI, and data science.\n",
"2. **AI Alignment Forum**: A website that discusses the latest developments in AI research and applications.\n",
"3. **KDnuggets**: A popular blog that covers AI, machine learning, and data science topics.\n",
"\n",
"**Books:**\n",
"\n",
"1. **\"Artificial Intelligence: A Modern Approach\" by Stuart Russell and Peter Norvig**: A comprehensive textbook on AI.\n",
"2. **\"Deep Learning\" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville**: A book that focuses on deep learning techniques.\n",
"3. **\"Life 3.0: Being Human in the Age of Artificial Intelligence\" by Max Tegmark**: A thought-provoking book that explores the future of AI.\n",
"\n",
"**YouTube Channels:**\n",
"\n",
"1. **3Blue1Brown (Grant Sanderson)**: Animated explanations of machine learning and AI concepts.\n",
"2. **Crash Course - Artificial Intelligence**: Engaging video lessons on AI basics.\n",
"3. **Siraj Raval**: A channel that offers explanations of AI and machine learning concepts.\n",
"\n",
"Remember, Alice, these resources are just a starting point. As you progress in your learning journey, you'll discover more advanced topics and specialized areas within AI.\n",
"\n",
"What do you think? Is there a particular resource or topic that catches your attention?\n",
"\n",
"============================================================\n",
"Turn 3\n",
"============================================================\n",
"User: Can you remind me what my name is?\n",
"Assistant: "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/responses \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Your name is Alice! I remember you told me that when we started our conversation about learning AI. How's your exploration of AI resources going so far, Alice? Do you have any questions or need help with anything specific?Your name is Alice! I remember you told me that when we started our conversation about learning AI. How's your exploration of AI resources going so far, Alice? Do you have any questions or need help with anything specific?\n",
"\n",
"✅ Completed 3 conversational turns with context retention\n"
]
}
],
"source": [
"# Conversation turns that build on each other\n",
"conversation_turns = [\n",
" \"My name is Alice and I'm learning about AI.\",\n",
" \"What are some good resources for beginners?\",\n",
" \"Can you remind me what my name is?\",\n",
"]\n",
"\n",
"for i, query in enumerate(conversation_turns, 1):\n",
" print(f\"\\n{'='*60}\")\n",
" print(f\"Turn {i}\")\n",
" print(f\"{'='*60}\")\n",
" print(f\"User: {query}\")\n",
"\n",
" response = conv_agent.create_turn(\n",
" session_id=conv_session_id,\n",
" messages=[UserMessage(content=query, role=\"user\")],\n",
" stream=True,\n",
" )\n",
"\n",
" print(\"Assistant: \", end='')\n",
" for chunk in response:\n",
" if chunk.event.event_type == \"turn_completed\":\n",
" output = chunk.event.final_text\n",
" print(output)\n",
" break\n",
" elif chunk.event.event_type == \"step_progress\":\n",
" if hasattr(chunk.event.delta, 'text'):\n",
" print(chunk.event.delta.text, end='', flush=True)\n",
"\n",
"print(f\"\\n✅ Completed {len(conversation_turns)} conversational turns with context retention\")"
]
},
{
"cell_type": "code",
"execution_count": 126,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: DELETE http://localhost:8321/v1/conversations/conv_53b1fa277c2b51b59846a499a4a8f4fb264c86d50fe03b30 \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Session cleaned up\n"
]
}
],
"source": [
"# Cleanup\n",
"client.conversations.delete(conversation_id=conv_session_id)\n",
"print(\"✅ Session cleaned up\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2.2 RAG (Retrieval-Augmented Generation) Pattern\n",
"\n",
"Demonstrate how to provide context to the agent for more accurate responses."
]
},
{
"cell_type": "code",
"execution_count": 127,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Knowledge base: 3 documents\n",
" - doc1: Llama Stack is Meta's comprehensive framework for ...\n",
" - doc2: MCP (Model Context Protocol) is a standardized pro...\n",
" - doc3: RAG (Retrieval-Augmented Generation) combines info...\n"
]
}
],
"source": [
"# Sample knowledge base documents\n",
"documents = [\n",
" {\n",
" \"doc_id\": \"doc1\",\n",
" \"content\": \"Llama Stack is Meta's comprehensive framework for building LLM applications. \"\n",
" \"It provides standardized APIs for inference, safety, agents, and more.\",\n",
" \"metadata\": {\"category\": \"framework\", \"source\": \"docs\"}\n",
" },\n",
" {\n",
" \"doc_id\": \"doc2\",\n",
" \"content\": \"MCP (Model Context Protocol) is a standardized protocol that allows AI models \"\n",
" \"to interact with external tools and data sources in a consistent way.\",\n",
" \"metadata\": {\"category\": \"protocol\", \"source\": \"docs\"}\n",
" },\n",
" {\n",
" \"doc_id\": \"doc3\",\n",
" \"content\": \"RAG (Retrieval-Augmented Generation) combines information retrieval with \"\n",
" \"language generation to provide more accurate and contextual responses.\",\n",
" \"metadata\": {\"category\": \"technique\", \"source\": \"docs\"}\n",
" },\n",
"]\n",
"\n",
"print(f\"Knowledge base: {len(documents)} documents\")\n",
"for doc in documents:\n",
" print(f\" - {doc['doc_id']}: {doc['content'][:50]}...\")"
]
},
{
"cell_type": "code",
"execution_count": 128,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Created RAG agent\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/conversations \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Created session: conv_0361da3cbb5e8a9d9ce4ae6404e16f66724e6a18f8c01077\n"
]
}
],
"source": [
"# Create RAG-enabled agent\n",
"rag_agent = Agent(\n",
" client=client,\n",
" model=\"ollama/llama3.3:70b\",\n",
" instructions=(\n",
" \"You are a helpful AI assistant with access to a knowledge base. \"\n",
" \"When answering questions, use the provided context from the knowledge base. \"\n",
" \"If the context doesn't contain relevant information, say so.\"\n",
" ),\n",
")\n",
"\n",
"print(\"✅ Created RAG agent\")\n",
"\n",
"rag_session_id = rag_agent.create_session(session_name=\"rag_session\")\n",
"print(f\"✅ Created session: {rag_session_id}\")"
]
},
{
"cell_type": "code",
"execution_count": 129,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Query: What is Llama Stack and what does it provide?\n",
"Retrieved 1 relevant document(s)\n",
"\n",
"Answer: "
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/responses \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"According to the provided context from Document 1, Llama Stack is Meta's comprehensive framework for building LLM (Large Language Model) applications. It provides standardized APIs (Application Programming Interfaces) for various functions, including inference, safety, agents, and more.According to the provided context from Document 1, Llama Stack is Meta's comprehensive framework for building LLM (Large Language Model) applications. It provides standardized APIs (Application Programming Interfaces) for various functions, including inference, safety, agents, and more.\n",
"\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: DELETE http://localhost:8321/v1/conversations/conv_0361da3cbb5e8a9d9ce4ae6404e16f66724e6a18f8c01077 \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Session cleaned up\n"
]
}
],
"source": [
"# Query with context\n",
"query = \"What is Llama Stack and what does it provide?\"\n",
"\n",
"# Simulate retrieval (in production, use vector search)\n",
"relevant_docs = [doc for doc in documents if \"llama stack\" in doc[\"content\"].lower()]\n",
"context = \"\\n\\n\".join([f\"Document {i+1}:\\n{doc['content']}\"\n",
" for i, doc in enumerate(relevant_docs)])\n",
"\n",
"# Create prompt with retrieved context\n",
"prompt_with_context = f\"\"\"Context from knowledge base:\n",
"{context}\n",
"\n",
"Question: {query}\n",
"\n",
"Please answer based on the provided context.\"\"\"\n",
"\n",
"print(f\"Query: {query}\")\n",
"print(f\"Retrieved {len(relevant_docs)} relevant document(s)\\n\")\n",
"print(\"Answer: \", end='')\n",
"\n",
"response = rag_agent.create_turn(\n",
" session_id=rag_session_id,\n",
" messages=[UserMessage(content=prompt_with_context, role=\"user\")],\n",
" stream=True,\n",
")\n",
"\n",
"for chunk in response:\n",
" if chunk.event.event_type == \"turn_completed\":\n",
" output = chunk.event.final_text\n",
" print(output)\n",
" break\n",
" elif chunk.event.event_type == \"step_progress\":\n",
" if hasattr(chunk.event.delta, 'text'):\n",
" print(chunk.event.delta.text, end='', flush=True)\n",
"\n",
"print(\"\\n\")\n",
"client.conversations.delete(conversation_id=rag_session_id)\n",
"print(\"✅ Session cleaned up\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"# Part 3: MCP (Model Context Protocol) Tools\n",
"\n",
"MCP provides a standardized way for AI models to interact with external tools and data sources.\n",
"\n",
"We'll demonstrate:\n",
"- Defining MCP-compatible tools\n",
"- Agent tool selection\n",
"- Tool execution and response handling"
]
},
{
"cell_type": "code",
"execution_count": 130,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created 3 MCP tools:\n",
" - get_weather: Get current weather information for a specified location\n",
" - execute_code: Execute Python code and return the result\n",
" - web_search: Search the web for information\n"
]
}
],
"source": [
"def create_mcp_tools():\n",
" \"\"\"Create MCP-compatible tool definitions.\"\"\"\n",
" return [\n",
" {\n",
" \"tool_name\": \"get_weather\",\n",
" \"description\": \"Get current weather information for a specified location\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"location\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"City and state/country, e.g., 'San Francisco, CA'\"\n",
" },\n",
" \"unit\": {\n",
" \"type\": \"string\",\n",
" \"enum\": [\"celsius\", \"fahrenheit\"],\n",
" \"description\": \"Temperature unit\",\n",
" \"default\": \"fahrenheit\"\n",
" }\n",
" },\n",
" \"required\": [\"location\"]\n",
" }\n",
" },\n",
" {\n",
" \"tool_name\": \"execute_code\",\n",
" \"description\": \"Execute Python code and return the result\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"code\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"Python code to execute\"\n",
" }\n",
" },\n",
" \"required\": [\"code\"]\n",
" }\n",
" },\n",
" {\n",
" \"tool_name\": \"web_search\",\n",
" \"description\": \"Search the web for information\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"query\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"Search query\"\n",
" }\n",
" },\n",
" \"required\": [\"query\"]\n",
" }\n",
" },\n",
" ]\n",
"\n",
"tools = create_mcp_tools()\n",
"print(f\"Created {len(tools)} MCP tools:\")\n",
"for tool in tools:\n",
" print(f\" - {tool['tool_name']}: {tool['description']}\")"
]
},
{
"cell_type": "code",
"execution_count": 132,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MCP tool configuration ready\n",
" Server: http://localhost:3000/sse\n",
" Format: MCP server-based\n",
"\n",
" To use MCP tools:\n",
" 1. Set up your MCP server\n",
" 2. Update MCP_SERVER_URL and MCP_ACCESS_TOKEN above\n",
" 3. Pass mcp_tools to Agent(tools=mcp_tools)\n"
]
}
],
"source": [
"# Example 2: MCP Server Configuration (0.3.0 format)\n",
"\n",
"# MCP server configuration\n",
"# Replace with your actual MCP server URL and credentials\n",
"MCP_SERVER_URL = \"https://api.example.com/mcp\" # Your MCP server endpoint\n",
"MCP_ACCESS_TOKEN = \"your-token-here\" # Your authentication token\n",
"\n",
"MCP_ACCESS_TOKEN = \"YOUR_ACCESS_TOKEN_HERE\"\n",
"## ran an MCP server locally, you can replace this field with your mcp server url\n",
"MCP_SERVER_URL = \"http://localhost:3000/sse\" \n",
"#MCP_SERVER_URL = \"https://mcp.deepwiki.com/sse\"\n",
"mcp_tools = [\n",
" {\n",
" \"type\": \"mcp\",\n",
" \"server_url\": MCP_SERVER_URL,\n",
" \"server_label\": \"weather\",\n",
" \"headers\": {\n",
" \"Authorization\": f\"Bearer {MCP_ACCESS_TOKEN}\",\n",
" },\n",
" }\n",
"]\n",
"\n",
"\n",
"print(\"MCP tool configuration ready\")\n",
"print(f\" Server: {MCP_SERVER_URL}\")\n",
"print(\" Format: MCP server-based\")\n",
"print(\"\\n To use MCP tools:\")\n",
"print(\" 1. Set up your MCP server\")\n",
"print(\" 2. Update MCP_SERVER_URL and MCP_ACCESS_TOKEN above\")\n",
"print(\" 3. Pass mcp_tools to Agent(tools=mcp_tools)\")"
]
},
{
"cell_type": "code",
"execution_count": 133,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tool execution simulator ready\n"
]
}
],
"source": [
"def simulate_tool_execution(tool_name: str, arguments: Dict[str, Any]) -> str:\n",
" \"\"\"Simulate tool execution (replace with real implementations).\"\"\"\n",
" if tool_name == \"get_weather\":\n",
" location = arguments.get(\"location\", \"Unknown\")\n",
" unit = arguments.get(\"unit\", \"fahrenheit\")\n",
" temp = \"72°F\" if unit == \"fahrenheit\" else \"22°C\"\n",
" return json.dumps({\n",
" \"location\": location,\n",
" \"temperature\": temp,\n",
" \"condition\": \"Partly cloudy\",\n",
" \"humidity\": \"65%\",\n",
" \"wind\": \"10 mph NW\"\n",
" })\n",
" elif tool_name == \"execute_code\":\n",
" code = arguments.get(\"code\", \"\")\n",
" return json.dumps({\n",
" \"status\": \"success\",\n",
" \"output\": f\"Code execution simulated for: {code[:50]}...\"\n",
" })\n",
" elif tool_name == \"web_search\":\n",
" query = arguments.get(\"query\", \"\")\n",
" return json.dumps({\n",
" \"status\": \"success\",\n",
" \"results\": [\n",
" {\"title\": f\"Result {i+1}\", \"url\": f\"https://example.com/{i+1}\",\n",
" \"snippet\": f\"Information about {query}\"}\n",
" for i in range(3)\n",
" ]\n",
" })\n",
" return json.dumps({\"error\": \"Unknown tool\"})\n",
"\n",
"print(\"Tool execution simulator ready\")"
]
},
{
"cell_type": "code",
"execution_count": 145,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/conversations \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Created MCP agent\n",
"✅ Created session: conv_d86d8242813c4019f1176754067d0e393923fcc364429f5b\n"
]
}
],
"source": [
"mcp_agent = Agent(\n",
" client=client,\n",
" model=\"ollama/llama3.3:70b\",\n",
" instructions=\"You are a helpful AI assistant that can answer questions and help with various tasks.\",\n",
" tools=mcp_tools # you can set this field to tools when experimenting with the tools created by create_mcp_tools above.\n",
")\n",
"\n",
"print(\"Created MCP agent\")\n",
"\n",
"mcp_session_id = mcp_agent.create_session(session_name=\"mcp_tools_session\")\n",
"print(f\"✅ Created session: {mcp_session_id}\")"
]
},
{
"cell_type": "code",
"execution_count": 146,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/responses \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"======================================================================\n",
"MCP TOOL EXAMPLE\n",
"======================================================================\n",
"\n",
" User: What's the weather like in New York City?\n",
"\n",
" Assistant: \n",
"\n",
" [Tool Execution Started]\n",
"\n",
"\n",
" [Tool Execution Started]\n",
"The current weather in New York City is 52°F with a slight chance of very light rain and winds of 12 mph NE. Tomorrow's forecast shows a high of 55°F with a slight chance of very light rain and winds of 14 to 17 mph NE. Over the next few days, the city can expect temperatures ranging from 49°F to 61°F with varying chances of rain and wind speeds. It's recommended to check the latest forecasts for the most up-to-date information.The current weather in New York City is 52°F with a slight chance of very light rain and winds of 12 mph NE. Tomorrow's forecast shows a high of 55°F with a slight chance of very light rain and winds of 14 to 17 mph NE. Over the next few days, the city can expect temperatures ranging from 49°F to 61°F with varying chances of rain and wind speeds. It's recommended to check the latest forecasts for the most up-to-date information.\n",
"\n",
"\n",
" Summary: Used 2 tool(s) to answer the query\n"
]
}
],
"source": [
"# Example: Weather query that should trigger tool usage\n",
"query = \"What's the weather like in New York City?\"\n",
"\n",
"print(f\"{'='*70}\")\n",
"print(f\"MCP TOOL EXAMPLE\")\n",
"print(f\"{'='*70}\")\n",
"print(f\"\\n User: {query}\")\n",
"\n",
"response = mcp_agent.create_turn(\n",
" session_id=mcp_session_id,\n",
" messages=[UserMessage(content=query, role=\"user\")],\n",
" stream=True,\n",
")\n",
"\n",
"print(\"\\n Assistant: \", end='')\n",
"tool_calls_made = []\n",
"\n",
"for chunk in response:\n",
" event_type = chunk.event.event_type\n",
"\n",
" if event_type == \"step_started\":\n",
" if chunk.event.step_type == \"tool_execution\":\n",
" print(f\"\\n\\n [Tool Execution Started]\")\n",
"\n",
" elif event_type == \"step_progress\":\n",
" # Check for tool call deltas\n",
" if hasattr(chunk.event.delta, 'delta_type'):\n",
" if chunk.event.delta.delta_type == \"tool_call_issued\":\n",
" tool_calls_made.append(chunk.event.delta)\n",
" result = simulate_tool_execution(\n",
" chunk.event.delta.tool_name,\n",
" json.loads(chunk.event.delta.arguments)\n",
" )\n",
" if hasattr(chunk.event.delta, 'text'):\n",
" print(chunk.event.delta.text, end='', flush=True)\n",
"\n",
" elif event_type == \"turn_completed\":\n",
" output = chunk.event.final_text\n",
" if output:\n",
" print(output)\n",
"\n",
"print()\n",
"if tool_calls_made:\n",
" print(f\"\\n Summary: Used {len(tool_calls_made)} tool(s) to answer the query\")"
]
},
{
"cell_type": "code",
"execution_count": 147,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: DELETE http://localhost:8321/v1/conversations/conv_d86d8242813c4019f1176754067d0e393923fcc364429f5b \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Session cleaned up\n"
]
}
],
"source": [
"# Cleanup\n",
"client.conversations.delete(conversation_id=mcp_session_id)\n",
"print(\"✅ Session cleaned up\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"# Summary\n",
"\n",
"This notebook demonstrated three levels of Llama Stack agent capabilities:\n",
"\n",
"## 1. Basic Agent\n",
"- ✅ Simple agent creation\n",
"- ✅ Session management \n",
"- ✅ Streaming responses\n",
"\n",
"## 2. Advanced Features\n",
"- ✅ Multi-turn conversations\n",
"- ✅ RAG (Retrieval-Augmented Generation) pattern\n",
"- ✅ Custom knowledge base integration\n",
"\n",
"## 3. MCP Tools Integration\n",
"- ✅ MCP-compatible tool definitions\n",
"- ✅ Automatic tool selection by the agent\n",
"- ✅ Tool execution and response handling\n",
"- ✅ Real-time streaming with tool calls\n",
"\n",
"\n",
"## Resources\n",
"\n",
"- [Llama Stack Documentation](https://llama-stack.readthedocs.io/)\n",
"- [Llama Stack GitHub](https://github.com/meta-llama/llama-stack)\n",
"- [MCP Protocol Specification](https://modelcontextprotocol.io/)\n",
"- [Ollama Documentation](https://ollama.ai/)"
]
}
],
"metadata": {
"fileHeader": "",
"fileUid": "92b7454e-a941-41f0-bd02-6d5e728f20f1",
"isAdHoc": false,
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in a new issue View git blame Copy permalink