phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 12:07:34 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

lint

Browse source
This commit is contained in:
Kai Wu 2025-09-10 14:58:17 -07:00
parent 7fc14dd97e
commit f0966efec4
3 changed files with 2103 additions and 230 deletions
Download patch file Download diff file Expand all files Collapse all files

396
docs/notebooks/crewai/Llama_Stack_CrewAI.ipynb
View file

@ -13,14 +13,14 @@
"\n", "\n",
"### Overview\n", "### Overview\n",
"\n", "\n",
"- **LlamaStack**: Provides the infrastructure for running LLMs and vector databases.\n", "- **LlamaStack**: Provides the infrastructure for running LLMs and vector store.\n",
"- **CrewAI**: Offers a framework for orchestrating agents and tasks.\n", "- **CrewAI**: Offers a framework for orchestrating agents and tasks.\n",
"- **Integration**: Leverages LlamaStack's OpenAI-compatible API with CrewAI.\n", "- **Integration**: Leverages LlamaStack's OpenAI-compatible API with CrewAI.\n",
"\n", "\n",
"### What You Will Learn\n", "### What You Will Learn\n",
"\n", "\n",
"1. How to set up and start the LlamaStack server using the Together AI provider.\n", "1. How to set up and start the LlamaStack server using the Together AI provider.\n",
"2. How to create and manage vector databases within LlamaStack.\n", "2. How to create and manage vector stores within LlamaStack.\n",
"3. How to build RAG tool with CrewAI by utilizing the LlamaStack server.\n", "3. How to build RAG tool with CrewAI by utilizing the LlamaStack server.\n",
"4. How to query the RAG tool for effective information retrieval and generation.\n", "4. How to query the RAG tool for effective information retrieval and generation.\n",
"\n", "\n",
@ -38,7 +38,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 1, "execution_count": 2,
"id": "5b6a6a17-b931-4bea-8273-0d6e5563637a", "id": "5b6a6a17-b931-4bea-8273-0d6e5563637a",
"metadata": { "metadata": {
"colab": { "colab": {
@ -59,7 +59,7 @@
] ]
}, },
{ {
"name": "stdin", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"TOGETHER_API_KEY environment variable is not set. Please enter your API key: ········\n" "TOGETHER_API_KEY environment variable is not set. Please enter your API key: ········\n"
@ -109,7 +109,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 2, "execution_count": 3,
"id": "dd2dacf3-ec8b-4cc7-8ff4-b5b6ea4a6e9e", "id": "dd2dacf3-ec8b-4cc7-8ff4-b5b6ea4a6e9e",
"metadata": { "metadata": {
"colab": { "colab": {
@ -128,32 +128,32 @@
"Environment '/Users/kaiwu/work/kaiwu/llama-stack/.venv' already exists, re-using it.\n", "Environment '/Users/kaiwu/work/kaiwu/llama-stack/.venv' already exists, re-using it.\n",
"Virtual environment /Users/kaiwu/work/kaiwu/llama-stack/.venv is already active\n", "Virtual environment /Users/kaiwu/work/kaiwu/llama-stack/.venv is already active\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n", "\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2mAudited \u001b[1m1 package\u001b[0m \u001b[2min 206ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m1 package\u001b[0m \u001b[2min 186ms\u001b[0m\u001b[0m\n",
"Installing pip dependencies\n", "Installing pip dependencies\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n", "\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2K\u001b[2mResolved \u001b[1m185 packages\u001b[0m \u001b[2min 2.02s\u001b[0m\u001b[0m \u001b[0m\n", "\u001b[2K\u001b[2mResolved \u001b[1m185 packages\u001b[0m \u001b[2min 1.29s\u001b[0m\u001b[0m \u001b[0m\n",
"\u001b[2mUninstalled \u001b[1m3 packages\u001b[0m \u001b[2min 247ms\u001b[0m\u001b[0m\n", "\u001b[2mUninstalled \u001b[1m3 packages\u001b[0m \u001b[2min 285ms\u001b[0m\u001b[0m\n",
"\u001b[2K\u001b[2mInstalled \u001b[1m3 packages\u001b[0m \u001b[2min 97ms\u001b[0m\u001b[0m \u001b[0m\n", "\u001b[2K\u001b[2mInstalled \u001b[1m3 packages\u001b[0m \u001b[2min 105ms\u001b[0m\u001b[0m \u001b[0m\n",
" \u001b[31m-\u001b[39m \u001b[1mprotobuf\u001b[0m\u001b[2m==5.29.5\u001b[0m\n", " \u001b[31m-\u001b[39m \u001b[1mprotobuf\u001b[0m\u001b[2m==5.29.5\u001b[0m\n",
" \u001b[32m+\u001b[39m \u001b[1mprotobuf\u001b[0m\u001b[2m==5.29.4\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1mprotobuf\u001b[0m\u001b[2m==5.29.4\u001b[0m\n",
" \u001b[31m-\u001b[39m \u001b[1mruamel-yaml\u001b[0m\u001b[2m==0.18.14\u001b[0m\n", " \u001b[31m-\u001b[39m \u001b[1mruamel-yaml\u001b[0m\u001b[2m==0.18.14\u001b[0m\n",
" \u001b[32m+\u001b[39m \u001b[1mruamel-yaml\u001b[0m\u001b[2m==0.17.40\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1mruamel-yaml\u001b[0m\u001b[2m==0.17.40\u001b[0m\n",
" \u001b[31m-\u001b[39m \u001b[1mruff\u001b[0m\u001b[2m==0.12.5\u001b[0m\n", " \u001b[31m-\u001b[39m \u001b[1mruff\u001b[0m\u001b[2m==0.12.5\u001b[0m\n",
" \u001b[32m+\u001b[39m \u001b[1mruff\u001b[0m\u001b[2m==0.9.10\u001b[0m\n", " \u001b[32m+\u001b[39m \u001b[1mruff\u001b[0m\u001b[2m==0.9.10\u001b[0m\n",
"Installing special provider module: torch torchtune>=0.5.0 torchao>=0.12.0 --extra-index-url https://download.pytorch.org/whl/cpu\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2mAudited \u001b[1m3 packages\u001b[0m \u001b[2min 169ms\u001b[0m\u001b[0m\n",
"Installing special provider module: sentence-transformers --no-deps\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2mAudited \u001b[1m1 package\u001b[0m \u001b[2min 43ms\u001b[0m\u001b[0m\n",
"Installing special provider module: torch torchvision torchao>=0.12.0 --extra-index-url https://download.pytorch.org/whl/cpu\n", "Installing special provider module: torch torchvision torchao>=0.12.0 --extra-index-url https://download.pytorch.org/whl/cpu\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n", "\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2mAudited \u001b[1m3 packages\u001b[0m \u001b[2min 29ms\u001b[0m\u001b[0m\n", "\u001b[2mAudited \u001b[1m3 packages\u001b[0m \u001b[2min 66ms\u001b[0m\u001b[0m\n",
"Installing special provider module: sentence-transformers --no-deps\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2mAudited \u001b[1m1 package\u001b[0m \u001b[2min 63ms\u001b[0m\u001b[0m\n",
"Installing special provider module: torch torchtune>=0.5.0 torchao>=0.12.0 --extra-index-url https://download.pytorch.org/whl/cpu\n",
"\u001b[2mUsing Python 3.12.9 environment at: /Users/kaiwu/work/kaiwu/llama-stack/.venv\u001b[0m\n",
"\u001b[2mAudited \u001b[1m3 packages\u001b[0m \u001b[2min 95ms\u001b[0m\u001b[0m\n",
"\u001b[32mBuild Successful!\u001b[0m\n", "\u001b[32mBuild Successful!\u001b[0m\n",
"\u001b[34mYou can find the newly-built distribution here: /Users/kaiwu/.llama/distributions/starter/starter-run.yaml\u001b[0m\n", "\u001b[34mYou can find the newly-built distribution here: /Users/kaiwu/.llama/distributions/starter/starter-run.yaml\u001b[0m\n",
"\u001b[32mYou can run the new Llama Stack distro via: \u001b[34mllama stack run /Users/kaiwu/.llama/distributions/starter/starter-run.yaml --image-type venv\u001b[0m\u001b[0m\n", "\u001b[32mYou can run the new Llama Stack distro via: \u001b[34mllama stack run /Users/kaiwu/.llama/distributions/starter/starter-run.yaml --image-type venv\u001b[0m\u001b[0m\n",
"Starting Llama Stack server with PID: 97967\n", "Starting Llama Stack server with PID: 26783\n",
"Waiting for server to start.................\n", "Waiting for server to start..........\n",
"Server is ready!\n" "Server is ready!\n"
] ]
} }
@ -231,7 +231,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 3, "execution_count": 4,
"id": "ab4eff97-4565-4c73-b1b3-0020a4c7e2a5", "id": "ab4eff97-4565-4c73-b1b3-0020a4c7e2a5",
"metadata": { "metadata": {
"id": "ab4eff97-4565-4c73-b1b3-0020a4c7e2a5" "id": "ab4eff97-4565-4c73-b1b3-0020a4c7e2a5"
@ -260,7 +260,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 4, "execution_count": 5,
"id": "880443ef-ac3c-48b1-a80a-7dab5b25ac61", "id": "880443ef-ac3c-48b1-a80a-7dab5b25ac61",
"metadata": { "metadata": {
"id": "880443ef-ac3c-48b1-a80a-7dab5b25ac61", "id": "880443ef-ac3c-48b1-a80a-7dab5b25ac61",
@ -268,6 +268,13 @@
"scrolled": true "scrolled": true
}, },
"outputs": [ "outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Available models:\n"
]
},
{ {
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
@ -280,7 +287,12 @@
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Available models:\n", "- llama_api/Llama-3.3-70B-Instruct\n",
"- llama_api/Llama-4-Maverick-17B-128E-Instruct-FP8\n",
"- llama_api/Llama-4-Scout-17B-16E-Instruct-FP8\n",
"- meta-llama/Llama-3.3-70B-Instruct\n",
"- meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8\n",
"- meta-llama/Llama-4-Scout-17B-16E-Instruct-FP8\n",
"- fireworks/accounts/fireworks/models/llama-v3p1-8b-instruct\n", "- fireworks/accounts/fireworks/models/llama-v3p1-8b-instruct\n",
"- fireworks/accounts/fireworks/models/llama-v3p1-70b-instruct\n", "- fireworks/accounts/fireworks/models/llama-v3p1-70b-instruct\n",
"- fireworks/accounts/fireworks/models/llama-v3p1-405b-instruct\n", "- fireworks/accounts/fireworks/models/llama-v3p1-405b-instruct\n",
@ -375,21 +387,21 @@
"id": "b0f28603-3207-4157-b731-638d93cd82b5" "id": "b0f28603-3207-4157-b731-638d93cd82b5"
}, },
"source": [ "source": [
"### 4. Vector Database Setup\n", "### 4. Vector Store Setup\n",
"\n", "\n",
"#### Register a Vector Database\n", "#### Create a Vector Store with File Upload\n",
"\n", "\n",
"Create a FAISS vector database for storing document embeddings:\n", "Create a vector store using the OpenAI-compatible vector stores API:\n",
"\n", "\n",
"- **Vector DB ID**: Unique identifier for the database\n", "- **Vector Store**: OpenAI-compatible vector store for document storage\n",
"- **Provider**: FAISS (Facebook AI Similarity Search)\n", "- **File Upload**: Automatic chunking and embedding of uploaded files\n",
"- **Embedding Model**: Sentence Transformers model for text embeddings\n", "- **Embedding Model**: Sentence Transformers model for text embeddings\n",
"- **Dimensions**: 384-dimensional embeddings" "- **Dimensions**: 384-dimensional embeddings"
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 5, "execution_count": 6,
"id": "0f241d81-19a7-451f-ac4e-2869a29300d1", "id": "0f241d81-19a7-451f-ac4e-2869a29300d1",
"metadata": { "metadata": {
"id": "0f241d81-19a7-451f-ac4e-2869a29300d1", "id": "0f241d81-19a7-451f-ac4e-2869a29300d1",
@ -400,57 +412,30 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/vector-dbs \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/files \"HTTP/1.1 200 OK\"\n",
"INFO:httpx:HTTP Request: GET http://0.0.0.0:8321/v1/vector-dbs \"HTTP/1.1 200 OK\"\n" "INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/files \"HTTP/1.1 200 OK\"\n",
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/files \"HTTP/1.1 200 OK\"\n"
] ]
}, },
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"Registered new vector DB: VectorDBRegisterResponse(embedding_dimension=384, embedding_model='sentence-transformers/all-MiniLM-L6-v2', identifier='acme_docs', provider_id='faiss', type='vector_db', provider_resource_id='acme_docs_v2', vector_db_name=None, owner=None, source='via_register_api')\n", "File(id='file-489db9aae0424745960e3408ff0f477f', bytes=41, created_at=1757540912, expires_at=1789076912, filename='shipping_policy.txt', object='file', purpose='assistants')\n",
"Existing vector DBs: [VectorDBListResponseItem(embedding_dimension=384, embedding_model='sentence-transformers/all-MiniLM-L6-v2', identifier='acme_docs', provider_id='faiss', type='vector_db', provider_resource_id='acme_docs_v2', vector_db_name=None)]\n" "File(id='file-b2f38b0e164347f5a2b6bbe211e33ff3', bytes=48, created_at=1757540912, expires_at=1789076912, filename='returns_policy.txt', object='file', purpose='assistants')\n",
"File(id='file-6f6f157d165a4078b4abef66a095ccd6', bytes=45, created_at=1757540912, expires_at=1789076912, filename='support.txt', object='file', purpose='assistants')\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/vector_stores \"HTTP/1.1 200 OK\"\n"
] ]
} }
], ],
"source": [ "source": [
"# Register a new clean vector database\n", "from io import BytesIO\n",
"vector_db = client.vector_dbs.register(\n",
" vector_db_id=\"acme_docs\", # Use a new unique name\n",
" provider_id=\"faiss\",\n",
" provider_vector_db_id=\"acme_docs_v2\",\n",
" embedding_model=\"sentence-transformers/all-MiniLM-L6-v2\",\n",
" embedding_dimension=384,\n",
")\n",
"print(\"Registered new vector DB:\", vector_db)\n",
"\n",
"# List all registered vector databases\n",
"dbs = client.vector_dbs.list()\n",
"print(\"Existing vector DBs:\", dbs)"
]
},
{
"cell_type": "markdown",
"id": "pcgjqzfr3eo",
"metadata": {
"id": "pcgjqzfr3eo"
},
"source": [
"#### Prepare Sample Documents\n",
"\n",
"Create LLAMA Stack Chunks for FAISS vector store"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "5a0a6619-c9fb-4938-8ff3-f84304eed91e",
"metadata": {
"id": "5a0a6619-c9fb-4938-8ff3-f84304eed91e"
},
"outputs": [],
"source": [
"from llama_stack_client.types.vector_io_insert_params import Chunk\n",
"\n", "\n",
"docs = [\n", "docs = [\n",
" (\"Acme ships globally in 3-5 business days.\", {\"title\": \"Shipping Policy\"}),\n", " (\"Acme ships globally in 3-5 business days.\", {\"title\": \"Shipping Policy\"}),\n",
@ -458,62 +443,22 @@
" (\"Support is available 24/7 via chat and email.\", {\"title\": \"Support\"}),\n", " (\"Support is available 24/7 via chat and email.\", {\"title\": \"Support\"}),\n",
"]\n", "]\n",
"\n", "\n",
"# Convert to Chunk objects\n", "file_ids = []\n",
"chunks = []\n", "for content, metadata in docs:\n",
"for _, (content, metadata) in enumerate(docs):\n", " with BytesIO(content.encode()) as file_buffer:\n",
" # Transform metadata to required format with document_id from title\n", " file_buffer.name = f\"{metadata['title'].replace(' ', '_').lower()}.txt\"\n",
" metadata = {\"document_id\": metadata[\"title\"]}\n", " create_file_response = client.files.create(file=file_buffer, purpose=\"assistants\")\n",
" chunk = Chunk(\n", " print(create_file_response)\n",
" content=content, # Required[InterleavedContent]\n", " file_ids.append(create_file_response.id)\n",
" metadata=metadata, # Required[Dict]\n",
" )\n",
" chunks.append(chunk)"
]
},
{
"cell_type": "markdown",
"id": "6bg3sm2ko5g",
"metadata": {
"id": "6bg3sm2ko5g"
},
"source": [
"#### Insert Documents into Vector Database\n",
"\n", "\n",
"Store the prepared documents in the FAISS vector database. This process:\n", "# Create vector store with files\n",
"1. Generates embeddings for each document\n", "vector_store = client.vector_stores.create(\n",
"2. Stores embeddings with metadata\n", " name=\"acme_docs\",\n",
"3. Enables semantic search capabilities" " file_ids=file_ids,\n",
] " embedding_model=\"sentence-transformers/all-MiniLM-L6-v2\",\n",
}, " embedding_dimension=384,\n",
{ " provider_id=\"faiss\"\n",
"cell_type": "code", ")"
"execution_count": 7,
"id": "0e8740d8-b809-44b9-915f-1e0200e3c3f1",
"metadata": {
"id": "0e8740d8-b809-44b9-915f-1e0200e3c3f1",
"outputId": "ebc275ae-54dd-4566-edb0-f3a067bfcf03"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/vector-io/insert \"HTTP/1.1 200 OK\"\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Documents inserted: None\n"
]
}
],
"source": [
"# Insert chunks into FAISS vector store\n",
"\n",
"response = client.vector_io.insert(vector_db_id=\"acme_docs\", chunks=chunks)\n",
"print(\"Documents inserted:\", response)"
] ]
}, },
{ {
@ -525,12 +470,12 @@
"source": [ "source": [
"#### Test Vector Search\n", "#### Test Vector Search\n",
"\n", "\n",
"Query the vector database to verify it's working correctly. This performs semantic search to find relevant documents based on the query." "Query the vector store to verify it's working correctly. This performs semantic search to find relevant documents based on the query."
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 8, "execution_count": 7,
"id": "4a5e010c-eeeb-4020-a957-74d6d1cba342", "id": "4a5e010c-eeeb-4020-a957-74d6d1cba342",
"metadata": { "metadata": {
"id": "4a5e010c-eeeb-4020-a957-74d6d1cba342", "id": "4a5e010c-eeeb-4020-a957-74d6d1cba342",
@ -541,32 +486,27 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/vector-io/query \"HTTP/1.1 200 OK\"\n" "INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/vector_stores/vs_dab05212-db05-402c-91ef-57e41797406b/search \"HTTP/1.1 200 OK\"\n"
] ]
}, },
{ {
"name": "stdout", "name": "stdout",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"metadata : {'document_id': 'Shipping Policy'}\n", "Acme ships globally in 3-5 business days.\n",
"content : Acme ships globally in 3-5 business days.\n", "Returns are accepted within 30 days of purchase.\n"
"metadata : {'document_id': 'Returns Policy'}\n",
"content : Returns are accepted within 30 days of purchase.\n",
"metadata : {'document_id': 'Support'}\n",
"content : Support is available 24/7 via chat and email.\n"
] ]
} }
], ],
"source": [ "source": [
"# Query chunks from FAISS vector store\n", "search_response = client.vector_stores.search(\n",
"\n", " vector_store_id=vector_store.id,\n",
"query_chunk_response = client.vector_io.query(\n", " query=\"How long does shipping take?\",\n",
" vector_db_id=\"acme_docs\",\n", " max_num_results=2\n",
" query=\"How long does Acme take to ship orders?\",\n",
")\n", ")\n",
"for chunk in query_chunk_response.chunks:\n", "for result in search_response.data:\n",
" print(\"metadata\", \":\", chunk.metadata)\n", " content = result.content[0].text\n",
" print(\"content\", \":\", chunk.content)" " print(content)"
] ]
}, },
{ {
@ -589,7 +529,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 9, "execution_count": 8,
"id": "c378bd10-09c2-417c-bdfc-1e0a2dd19084", "id": "c378bd10-09c2-417c-bdfc-1e0a2dd19084",
"metadata": { "metadata": {
"id": "c378bd10-09c2-417c-bdfc-1e0a2dd19084", "id": "c378bd10-09c2-417c-bdfc-1e0a2dd19084",
@ -631,7 +571,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 10, "execution_count": 9,
"id": "f88ffb5a-657b-4916-9375-c6ddc156c25e", "id": "f88ffb5a-657b-4916-9375-c6ddc156c25e",
"metadata": { "metadata": {
"id": "f88ffb5a-657b-4916-9375-c6ddc156c25e", "id": "f88ffb5a-657b-4916-9375-c6ddc156c25e",
@ -642,22 +582,22 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"\u001b[92m10:30:35 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n", "\u001b[92m14:49:56 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:LiteLLM:\n", "INFO:LiteLLM:\n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:30:41 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n", "\u001b[92m14:50:01 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n",
"INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n" "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n"
] ]
}, },
{ {
"data": { "data": {
"text/plain": [ "text/plain": [
"\"In the Andes' gentle breeze, a llama's soft eyes gaze with peaceful ease, its fur a warm and fuzzy tease. With steps both gentle and serene, the llama roams, a symbol of calm, its beauty truly unseen.\"" "\"In the Andes' gentle breeze, a llama's soft eyes gaze with peaceful ease, its fur a warm and fuzzy tease. With steps both gentle and serene, the llama roams, a symbol of calm, its beauty pure and supreme.\""
] ]
}, },
"execution_count": 10, "execution_count": 9,
"metadata": {}, "metadata": {},
"output_type": "execute_result" "output_type": "execute_result"
} }
@ -680,15 +620,15 @@
"source": [ "source": [
"#### Create CrewAI Custom Tool\n", "#### Create CrewAI Custom Tool\n",
"\n", "\n",
"Define a custom CrewAI tool, `LlamaStackRAGTool`, to encapsulate the logic for querying the LlamaStack vector database. This tool will be used by the CrewAI agent to perform retrieval during the RAG process.\n", "Define a custom CrewAI tool, `LlamaStackRAGTool`, to encapsulate the logic for querying the LlamaStack vector store. This tool will be used by the CrewAI agent to perform retrieval during the RAG process.\n",
"\n", "\n",
"- **Input Schema**: Defines the expected input parameters for the tool, such as the user query, the vector database ID, and optional parameters like `top_k` and `score_threshold`.\n", "- **Input Schema**: Defines the expected input parameters for the tool, such as the user query, the vector store ID, and optional parameters like `top_k`.\n",
"- **Tool Logic**: Implements the `_run` method, which takes the user query and vector database ID, calls the LlamaStack client's `vector_io.query` method, and formats the retrieved documents into a human-readable string for the LLM to use as context." "- **Tool Logic**: Implements the `_run` method, which takes the user query and vector store ID, calls the LlamaStack client's `vector_stores.search` method, and formats the retrieved documents into a human-readable string for the LLM to use as context."
] ]
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 11, "execution_count": 16,
"id": "08de540f-ed47-405a-a9c5-16505f4c88c8", "id": "08de540f-ed47-405a-a9c5-16505f4c88c8",
"metadata": { "metadata": {
"id": "08de540f-ed47-405a-a9c5-16505f4c88c8" "id": "08de540f-ed47-405a-a9c5-16505f4c88c8"
@ -700,11 +640,11 @@
"from pydantic import BaseModel, Field\n", "from pydantic import BaseModel, Field\n",
"\n", "\n",
"# ---------- 1. Input schema ----------\n", "# ---------- 1. Input schema ----------\n",
"class RAGToolInput(BaseModel):\n", "class VectorStoreRAGToolInput(BaseModel):\n",
" \"\"\"Input schema for LlamaStackRAGTool.\"\"\"\n", " \"\"\"Input schema for LlamaStackVectorStoreRAGTool.\"\"\"\n",
" query: str = Field(..., description=\"The user query for RAG search\")\n", " query: str = Field(..., description=\"The user query for RAG search\")\n",
" vector_db_id: str = Field(...,\n", " vector_store_id: str = Field(...,\n",
" description=\"ID of the vector database to search inside the Llama-Stack server\",\n", " description=\"ID of the vector store to search inside the Llama-Stack server\",\n",
" )\n", " )\n",
" top_k: Optional[int] = Field(\n", " top_k: Optional[int] = Field(\n",
" default=5,\n", " default=5,\n",
@ -716,44 +656,41 @@
" )\n", " )\n",
"\n", "\n",
"# ---------- 2. The tool ----------\n", "# ---------- 2. The tool ----------\n",
"class LlamaStackRAGTool(BaseTool):\n", "class LlamaStackVectorStoreRAGTool(BaseTool):\n",
" name: str = \"Llama Stack RAG tool\"\n", " name: str = \"Llama Stack Vector Store RAG tool\"\n",
" description: str = (\n", " description: str = (\n",
" \"This tool calls a Llama-Stack endpoint for retrieval-augmented generation. \"\n", " \"This tool calls a Llama-Stack endpoint for retrieval-augmented generation using a vector store. \"\n",
" \"It takes a natural-language query and returns the most relevant documents.\"\n", " \"It takes a natural-language query and returns the most relevant documents.\"\n",
" )\n", " )\n",
" args_schema: Type[BaseModel] = RAGToolInput\n", " args_schema: Type[BaseModel] = VectorStoreRAGToolInput\n",
" client: Any\n", " client: Any\n",
" vector_db_id: str = \"\"\n", " vector_store_id: str = \"\"\n",
" top_k: int = 5\n", " top_k: int = 5\n",
" score_threshold: Optional[float] = None\n",
"\n", "\n",
" def _run(self, **kwargs: Any) -> str:\n", " def _run(self, **kwargs: Any) -> str:\n",
" # 1. Resolve parameters (use instance defaults when not supplied)\n", " # 1. Resolve parameters (use instance defaults when not supplied)\n",
" print(kwargs)\n",
" query: str = kwargs.get(\"query\") # Required – schema enforces presence\n", " query: str = kwargs.get(\"query\") # Required – schema enforces presence\n",
" vector_db_id: str = kwargs.get(\"vector_db_id\", self.vector_db_id)\n", " vector_store_id: str = kwargs.get(\"vector_store_id\", self.vector_store_id)\n",
" top_k: int = kwargs.get(\"top_k\", self.top_k)\n", " top_k: int = kwargs.get(\"top_k\", self.top_k)\n",
" score_threshold: float | None = kwargs.get(\n", " if vector_store_id == \"\":\n",
" \"score_threshold\", self.score_threshold\n", " print('vector_store_id is empty, please specify which vector_store to search')\n",
" )\n",
" if vector_db_id == \"\":\n",
" print('vector_db_id is empty, please specify which vector_db to search')\n",
" return \"No documents found.\"\n", " return \"No documents found.\"\n",
" # 2. Issue request to Llama-Stack\n", " # 2. Issue request to Llama-Stack\n",
" response: List[dict] = self.client.vector_io.query(\n", " response = self.client.vector_stores.search(\n",
" vector_db_id=vector_db_id,\n", " vector_store_id=vector_store_id,\n",
" query=query,\n", " query=query,\n",
" params={\"max_chunks\": top_k, \"score_threshold\": score_threshold}\n", " max_num_results=top_k,\n",
" )\n", " )\n",
"\n", "\n",
" # 3. Massage results into a single human-readable string\n", " # 3. Massage results into a single human-readable string\n",
" if not response:\n", " if not response or not response.data:\n",
" return \"No documents found.\"\n", " return \"No documents found.\"\n",
"\n", "\n",
" docs: List[str] = []\n", " docs: List[str] = []\n",
" for metadata,content in response:\n", " for result in response.data:\n",
" docs.append(f\"metadata: {metadata}, content: {content}\")\n", " content = result.content[0].text if result.content else \"No content\"\n",
" filename = result.filename if result.filename else {}\n",
" docs.append(f\"filename: {filename}, content: {content}\")\n",
" return \"\\n\".join(docs)\n" " return \"\\n\".join(docs)\n"
] ]
}, },
@ -771,7 +708,7 @@
"Construct a CrewAI pipeline that orchestrates the RAG process. This pipeline includes:\n", "Construct a CrewAI pipeline that orchestrates the RAG process. This pipeline includes:\n",
"\n", "\n",
"1. **Agent Definition**: Defining a CrewAI agent with a specific role (`RAG assistant`), goal, backstory, and the LlamaStack LLM and the custom RAG tool.\n", "1. **Agent Definition**: Defining a CrewAI agent with a specific role (`RAG assistant`), goal, backstory, and the LlamaStack LLM and the custom RAG tool.\n",
"2. **Task Definition**: Defining a CrewAI task for the agent to perform. The task description includes placeholders for the user query and vector database ID, which will be provided during execution. The task's expected output is an answer to the question based on the retrieved context.\n", "2. **Task Definition**: Defining a CrewAI task for the agent to perform. The task description includes placeholders for the user query and vector store ID, which will be provided during execution. The task's expected output is an answer to the question based on the retrieved context.\n",
"3. **Crew Definition**: Creating a CrewAI `Crew` object with the defined task and agent. This crew represents the complete RAG pipeline.\n", "3. **Crew Definition**: Creating a CrewAI `Crew` object with the defined task and agent. This crew represents the complete RAG pipeline.\n",
"\n", "\n",
"**CrewAI workflow**:\n", "**CrewAI workflow**:\n",
@ -780,7 +717,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 12, "execution_count": 17,
"id": "9684427d-dcc7-4544-9af5-8b110d014c42", "id": "9684427d-dcc7-4544-9af5-8b110d014c42",
"metadata": { "metadata": {
"id": "9684427d-dcc7-4544-9af5-8b110d014c42" "id": "9684427d-dcc7-4544-9af5-8b110d014c42"
@ -796,10 +733,10 @@
" backstory=\"You are an experienced search assistant specializing in finding relevant information from documentation and vector_db to answer user questions accurately.\",\n", " backstory=\"You are an experienced search assistant specializing in finding relevant information from documentation and vector_db to answer user questions accurately.\",\n",
" allow_delegation=False,\n", " allow_delegation=False,\n",
" llm=llamastack_llm,\n", " llm=llamastack_llm,\n",
" tools=[LlamaStackRAGTool(client=client)])\n", " tools=[LlamaStackVectorStoreRAGTool(client=client)])\n",
"# ---- 4. Wrap everything in a Crew task ---------------------------\n", "# ---- 4. Wrap everything in a Crew task ---------------------------\n",
"task = Task(\n", "task = Task(\n",
" description=\"Answer the following questions: {query}, using the RAG_tool to search the provided vector_db_id {vector_db_id} if needed\",\n", " description=\"Answer the following questions: {query}, using the RAG_tool to search the provided vector_store_id {vector_store_id} if needed\",\n",
" expected_output=\"An answer to the question with provided context\",\n", " expected_output=\"An answer to the question with provided context\",\n",
" agent=agent,\n", " agent=agent,\n",
")\n", ")\n",
@ -820,7 +757,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 13, "execution_count": 18,
"id": "03322188-9509-446a-a4a8-ce3bb83ec87c", "id": "03322188-9509-446a-a4a8-ce3bb83ec87c",
"metadata": { "metadata": {
"colab": { "colab": {
@ -839,7 +776,7 @@
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">Crew Execution Started</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">Crew Execution Started</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008080; text-decoration-color: #008080\">crew</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008080; text-decoration-color: #008080\">crew</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008080; text-decoration-color: #008080\">146be674-7521-4db6-bb70-287b22aa21aa</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008080; text-decoration-color: #008080\">091cf919-5c4b-4168-ac49-65fe5e8faa9e</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
@ -851,7 +788,7 @@
"\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[1;36mCrew Execution Started\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[1;36mCrew Execution Started\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[36mcrew\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[36mcrew\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[36m146be674-7521-4db6-bb70-287b22aa21aa\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[36m091cf919-5c4b-4168-ac49-65fe5e8faa9e\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n",
@ -877,7 +814,7 @@
{ {
"data": { "data": {
"application/vnd.jupyter.widget-view+json": { "application/vnd.jupyter.widget-view+json": {
"model_id": "0fd1807480b443948be37ef922f4ad0f", "model_id": "cb8f60c158fb4a0496e78e4d596ac4c8",
"version_major": 2, "version_major": 2,
"version_minor": 0 "version_minor": 0
}, },
@ -892,23 +829,25 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"\u001b[92m10:31:29 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n", "\u001b[92m14:55:09 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:LiteLLM:\n", "INFO:LiteLLM:\n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:31:30 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n", "\u001b[92m14:55:11 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n",
"INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n" "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n"
] ]
}, },
{ {
"data": { "data": {
"text/html": [ "text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">{'query': 'How long does shipping take?', 'vector_db_id': 'acme_docs', 'top_k': 5, 'score_threshold': 0.5}\n", "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">{'query': 'How long does shipping take?', 'vector_store_id': 'vs_dab05212-db05-402c-91ef-57e41797406b', 'top_k': 1,\n",
"'score_threshold': 0.0}\n",
"</pre>\n" "</pre>\n"
], ],
"text/plain": [ "text/plain": [
"{'query': 'How long does shipping take?', 'vector_db_id': 'acme_docs', 'top_k': 5, 'score_threshold': 0.5}\n" "{'query': 'How long does shipping take?', 'vector_store_id': 'vs_dab05212-db05-402c-91ef-57e41797406b', 'top_k': 1,\n",
"'score_threshold': 0.0}\n"
] ]
}, },
"metadata": {}, "metadata": {},
@ -918,13 +857,13 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/vector-io/query \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/vector_stores/vs_dab05212-db05-402c-91ef-57e41797406b/search \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:31:31 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n", "\u001b[92m14:55:11 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:LiteLLM:\n", "INFO:LiteLLM:\n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:31:33 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n", "\u001b[92m14:55:12 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n",
"INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n" "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n"
] ]
}, },
@ -944,7 +883,7 @@
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008000; text-decoration-color: #008000\">╭──────────────────────────────────────────────── Task Completion ────────────────────────────────────────────────╮</span>\n", "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008000; text-decoration-color: #008000\">╭──────────────────────────────────────────────── Task Completion ────────────────────────────────────────────────╮</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Task Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Task Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">b1362fb4-fe9b-479d-a01d-18bf38406780</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">cf3f4f08-744c-4aee-9387-e9eb70624fc1</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Agent: </span><span style=\"color: #008000; text-decoration-color: #008000\">RAG assistant</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Agent: </span><span style=\"color: #008000; text-decoration-color: #008000\">RAG assistant</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
@ -956,7 +895,7 @@
"\u001b[32m╭─\u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m Task Completion \u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m─╮\u001b[0m\n", "\u001b[32m╭─\u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m Task Completion \u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m─╮\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[1;32mTask Completed\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[1;32mTask Completed\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mb1362fb4-fe9b-479d-a01d-18bf38406780\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mcf3f4f08-744c-4aee-9387-e9eb70624fc1\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mAgent: \u001b[0m\u001b[32mRAG assistant\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mAgent: \u001b[0m\u001b[32mRAG assistant\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
@ -987,10 +926,9 @@
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Crew Execution Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Crew Execution Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">crew</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">crew</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008000; text-decoration-color: #008000\">146be674-7521-4db6-bb70-287b22aa21aa</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008000; text-decoration-color: #008000\">091cf919-5c4b-4168-ac49-65fe5e8faa9e</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Final Output: Acme ships globally in 3-5 business days. Returns are accepted within 30 days of purchase. </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Final Output: Acme ships globally in 3-5 business days.</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Support is available 24/7 via chat and email.</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
@ -1001,10 +939,9 @@
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[1;32mCrew Execution Completed\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[1;32mCrew Execution Completed\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mcrew\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mcrew\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[32m146be674-7521-4db6-bb70-287b22aa21aa\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[32m091cf919-5c4b-4168-ac49-65fe5e8faa9e\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mFinal Output: Acme ships globally in 3-5 business days. Returns are accepted within 30 days of purchase. \u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mFinal Output: Acme ships globally in 3-5 business days.\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mSupport is available 24/7 via chat and email.\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n" "\u001b[32m╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n"
@ -1031,13 +968,13 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"❓ How long does shipping take?\n", "❓ How long does shipping take?\n",
"💡 Acme ships globally in 3-5 business days. Returns are accepted within 30 days of purchase. Support is available 24/7 via chat and email.\n" "💡 Acme ships globally in 3-5 business days.\n"
] ]
} }
], ],
"source": [ "source": [
"query = \"How long does shipping take?\"\n", "query = \"How long does shipping take?\"\n",
"response = crew.kickoff(inputs={\"query\": query,\"vector_db_id\": \"acme_docs\"})\n", "response = crew.kickoff(inputs={\"query\": query,\"vector_store_id\": vector_store.id})\n",
"print(\"❓\", query)\n", "print(\"❓\", query)\n",
"print(\"💡\", response)" "print(\"💡\", response)"
] ]
@ -1054,7 +991,7 @@
}, },
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 14, "execution_count": 19,
"id": "61995550-bb0b-46a8-a5d0-023207475d60", "id": "61995550-bb0b-46a8-a5d0-023207475d60",
"metadata": { "metadata": {
"colab": { "colab": {
@ -1073,7 +1010,7 @@
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">Crew Execution Started</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">Crew Execution Started</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008080; text-decoration-color: #008080\">crew</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008080; text-decoration-color: #008080\">crew</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008080; text-decoration-color: #008080\">146be674-7521-4db6-bb70-287b22aa21aa</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008080; text-decoration-color: #008080\">091cf919-5c4b-4168-ac49-65fe5e8faa9e</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
"<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n", "<span style=\"color: #008080; text-decoration-color: #008080\">│</span> <span style=\"color: #008080; text-decoration-color: #008080\">│</span>\n",
@ -1085,7 +1022,7 @@
"\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[1;36mCrew Execution Started\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[1;36mCrew Execution Started\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[36mcrew\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[36mcrew\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[36m146be674-7521-4db6-bb70-287b22aa21aa\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[36m091cf919-5c4b-4168-ac49-65fe5e8faa9e\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n",
"\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n", "\u001b[36m│\u001b[0m \u001b[36m│\u001b[0m\n",
@ -1111,7 +1048,7 @@
{ {
"data": { "data": {
"application/vnd.jupyter.widget-view+json": { "application/vnd.jupyter.widget-view+json": {
"model_id": "e1e2dc3abcd745fa80b32ef32b264396", "model_id": "60b83042bfc14a75b555537d13147372",
"version_major": 2, "version_major": 2,
"version_minor": 0 "version_minor": 0
}, },
@ -1126,23 +1063,25 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"\u001b[92m10:31:38 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n", "\u001b[92m14:55:19 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:LiteLLM:\n", "INFO:LiteLLM:\n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:31:39 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n", "\u001b[92m14:55:21 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n",
"INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n" "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n"
] ]
}, },
{ {
"data": { "data": {
"text/html": [ "text/html": [
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">{'query': 'return policy after 40 days', 'vector_db_id': 'acme_docs', 'top_k': 1, 'score_threshold': 0.5}\n", "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">{'query': 'return policy after 40 days', 'vector_store_id': 'vs_dab05212-db05-402c-91ef-57e41797406b', 'top_k': 1, \n",
"'score_threshold': 0.5}\n",
"</pre>\n" "</pre>\n"
], ],
"text/plain": [ "text/plain": [
"{'query': 'return policy after 40 days', 'vector_db_id': 'acme_docs', 'top_k': 1, 'score_threshold': 0.5}\n" "{'query': 'return policy after 40 days', 'vector_store_id': 'vs_dab05212-db05-402c-91ef-57e41797406b', 'top_k': 1, \n",
"'score_threshold': 0.5}\n"
] ]
}, },
"metadata": {}, "metadata": {},
@ -1152,13 +1091,13 @@
"name": "stderr", "name": "stderr",
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/vector-io/query \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://0.0.0.0:8321/v1/openai/v1/vector_stores/vs_dab05212-db05-402c-91ef-57e41797406b/search \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:31:39 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n", "\u001b[92m14:55:22 - LiteLLM:INFO\u001b[0m: utils.py:3258 - \n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:LiteLLM:\n", "INFO:LiteLLM:\n",
"LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n", "LiteLLM completion() model= together/meta-llama/Llama-3.3-70B-Instruct-Turbo; provider = openai\n",
"INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n", "INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions \"HTTP/1.1 200 OK\"\n",
"\u001b[92m10:31:41 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n", "\u001b[92m14:55:22 - LiteLLM:INFO\u001b[0m: utils.py:1260 - Wrapper: Completed Call, calling success_handler\n",
"INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n" "INFO:LiteLLM:Wrapper: Completed Call, calling success_handler\n"
] ]
}, },
@ -1178,7 +1117,7 @@
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008000; text-decoration-color: #008000\">╭──────────────────────────────────────────────── Task Completion ────────────────────────────────────────────────╮</span>\n", "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"color: #008000; text-decoration-color: #008000\">╭──────────────────────────────────────────────── Task Completion ────────────────────────────────────────────────╮</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Task Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Task Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">b1362fb4-fe9b-479d-a01d-18bf38406780</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">cf3f4f08-744c-4aee-9387-e9eb70624fc1</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Agent: </span><span style=\"color: #008000; text-decoration-color: #008000\">RAG assistant</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Agent: </span><span style=\"color: #008000; text-decoration-color: #008000\">RAG assistant</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
@ -1190,7 +1129,7 @@
"\u001b[32m╭─\u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m Task Completion \u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m─╮\u001b[0m\n", "\u001b[32m╭─\u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m Task Completion \u001b[0m\u001b[32m───────────────────────────────────────────────\u001b[0m\u001b[32m─╮\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[1;32mTask Completed\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[1;32mTask Completed\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mb1362fb4-fe9b-479d-a01d-18bf38406780\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mcf3f4f08-744c-4aee-9387-e9eb70624fc1\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mAgent: \u001b[0m\u001b[32mRAG assistant\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mAgent: \u001b[0m\u001b[32mRAG assistant\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
@ -1221,10 +1160,9 @@
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Crew Execution Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000; font-weight: bold\">Crew Execution Completed</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">crew</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Name: </span><span style=\"color: #008000; text-decoration-color: #008000\">crew</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008000; text-decoration-color: #008000\">146be674-7521-4db6-bb70-287b22aa21aa</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">ID: </span><span style=\"color: #008000; text-decoration-color: #008000\">091cf919-5c4b-4168-ac49-65fe5e8faa9e</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Tool Args: </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Final Output: Returns are accepted within 30 days of purchase. Therefore, you cannot return a product after </span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">Final Output: Returns are accepted within 30 days of purchase.</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #c0c0c0; text-decoration-color: #c0c0c0\">40 days.</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">│</span> <span style=\"color: #008000; text-decoration-color: #008000\">│</span>\n",
"<span style=\"color: #008000; text-decoration-color: #008000\">╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n", "<span style=\"color: #008000; text-decoration-color: #008000\">╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</span>\n",
@ -1235,10 +1173,9 @@
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[1;32mCrew Execution Completed\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[1;32mCrew Execution Completed\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mcrew\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mName: \u001b[0m\u001b[32mcrew\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[32m146be674-7521-4db6-bb70-287b22aa21aa\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mID: \u001b[0m\u001b[32m091cf919-5c4b-4168-ac49-65fe5e8faa9e\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mTool Args: \u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37mFinal Output: Returns are accepted within 30 days of purchase. Therefore, you cannot return a product after \u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[37mFinal Output: Returns are accepted within 30 days of purchase.\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[37m40 days.\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n", "\u001b[32m│\u001b[0m \u001b[32m│\u001b[0m\n",
"\u001b[32m╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n" "\u001b[32m╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n"
@ -1265,13 +1202,13 @@
"output_type": "stream", "output_type": "stream",
"text": [ "text": [
"❓ Can I return a product after 40 days?\n", "❓ Can I return a product after 40 days?\n",
"💡 Returns are accepted within 30 days of purchase. Therefore, you cannot return a product after 40 days.\n" "💡 Returns are accepted within 30 days of purchase.\n"
] ]
} }
], ],
"source": [ "source": [
"query = \"Can I return a product after 40 days?\"\n", "query = \"Can I return a product after 40 days?\"\n",
"response = crew.kickoff(inputs={\"query\": query,\"vector_db_id\": \"acme_docs\"})\n", "response = crew.kickoff(inputs={\"query\": query,\"vector_store_id\": vector_store.id})\n",
"print(\"❓\", query)\n", "print(\"❓\", query)\n",
"print(\"💡\", response)" "print(\"💡\", response)"
] ]
@ -1287,31 +1224,30 @@
"\n", "\n",
"We have successfully built a RAG system that combines:\n", "We have successfully built a RAG system that combines:\n",
"\n", "\n",
"- **LlamaStack** for infrastructure (LLM serving + vector database)\n", "- **LlamaStack** for infrastructure (LLM serving + vector store)\n",
"- **CrewAI** for orchestration (agents, tasks, and tools)\n", "- **CrewAI** for orchestration (agents, tasks, and tools)\n",
"- **Together AI** for high-quality language models\n", "- **Together AI** for high-quality language models\n",
"\n", "\n",
"### Key Benefits\n", "### Key Benefits\n",
"\n", "\n",
"1. **Unified Infrastructure**: A single server for LLMs and vector databases simplifies deployment and management.\n", "1. **Unified Infrastructure**: A single server for LLMs and vector stores simplifies deployment and management.\n",
"2. **OpenAI Compatibility**: Enables easy integration with existing libraries and frameworks that support the OpenAI API standard, such as CrewAI.\n", "2. **OpenAI Compatibility**: Enables easy integration with existing libraries and frameworks that support the OpenAI API standard, such as CrewAI.\n",
"3. **Multi-Provider Support**: Offers the flexibility to switch between different LLM and embedding providers without altering the core application logic.\n", "3. **Multi-Provider Support**: Offers the flexibility to switch between different LLM and embedding providers without altering the core application logic.\n",
"4. **Production Ready**: LlamaStack includes features designed for production environments, such as built-in safety shields and monitoring capabilities.\n", "4. **Production Ready**: LlamaStack includes features designed for production environments, such as built-in safety shields and monitoring capabilities.\n",
"\n", "\n",
"### Next Steps\n",
"\n",
"- Implement more sophisticated document processing and chunking techniques.\n",
"- Add conversation memory within the CrewAI pipeline for multi-turn interactions.\n",
"- Integrate advanced safety filtering and monitoring using LlamaStack's features.\n",
"- Scale the system to handle larger document collections and higher query volumes.\n",
"- Integrate the RAG system with web frameworks like FastAPI or Streamlit to build interactive applications.\n",
"\n",
"---\n",
"\n", "\n",
"##### 🔧 Cleanup\n", "##### 🔧 Cleanup\n",
"\n", "\n",
"Remember to stop the LlamaStack server process when you are finished to free up resources. You can use the `kill_llama_stack_server()` helper function defined earlier in the notebook." "Remember to stop the LlamaStack server process when you are finished to free up resources. You can use the `kill_llama_stack_server()` helper function defined earlier in the notebook."
] ]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a21270b4-b0a7-4481-96a5-044f908de363",
"metadata": {},
"outputs": [],
"source": []
} }
], ],
"metadata": { "metadata": {

1
docs/notebooks/crewai/customer_policies.txt Normal file
View file

@ -0,0 +1 @@
Acme ships globally in 3-5 business days.

1936
docs/notebooks/crewai/llama_stack_server.log Normal file
View file

File diff suppressed because it is too large Load diff