phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-15 22:47:59 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

standardized port and also included pre-req for all notebooks

Browse source
This commit is contained in:
Justin Lee 2024-11-05 16:38:46 -08:00
parent d0baf24999
commit b556cd91fd
8 changed files with 177 additions and 42 deletions
Download patch file Download diff file Expand all files Collapse all files

48
docs/zero_to_hero_guide/00_Inference101.ipynb
View file

@ -7,7 +7,10 @@
"source": [
"# Llama Stack Inference Guide\n",
"\n",
"This document provides instructions on how to use Llama Stack's `chat_completion` function for generating text using the `Llama3.2-11B-Vision-Instruct` model. Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/).\n",
"This document provides instructions on how to use Llama Stack's `chat_completion` function for generating text using the `Llama3.2-11B-Vision-Instruct` model. \n",
"\n",
"Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html).\n",
"\n",
"\n",
"### Table of Contents\n",
"1. [Quickstart](#quickstart)\n",
@ -25,7 +28,36 @@
"## Quickstart\n",
"\n",
"This section walks through each step to set up and make a simple text generation request.\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"id": "25b97dfe",
"metadata": {},
"source": [
"### 0. Configuration\n",
"Set up your connection parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "38a39e44",
"metadata": {},
"outputs": [],
"source": [
"HOST = \"localhost\" # Replace with your host\n",
"PORT = 5001 # Replace with your port"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d1d097ab",
"metadata": {},
"outputs": [],
"source": [
"### 1. Set Up the Client\n",
"\n",
"Begin by importing the necessary components from Llama Stack’s client library:"
@ -41,7 +73,7 @@
"from llama_stack_client import LlamaStackClient\n",
"from llama_stack_client.types import SystemMessage, UserMessage\n",
"\n",
"client = LlamaStackClient(base_url='http://localhost:5000')"
"client = LlamaStackClient(base_url='http://{HOST}:{PORT}')"
]
},
{
@ -129,7 +161,7 @@
"from llama_stack_client.types import UserMessage\n",
"from termcolor import cprint\n",
"\n",
"client = LlamaStackClient(base_url='http://localhost:5000')\n",
"client = LlamaStackClient(base_url='http://{HOST}:{PORT}')\n",
"\n",
"async def chat_loop():\n",
" while True:\n",
@ -214,7 +246,7 @@
"from termcolor import cprint\n",
"\n",
"async def run_main(stream: bool = True):\n",
" client = LlamaStackClient(base_url='http://localhost:5000')\n",
" client = LlamaStackClient(base_url='http://{HOST}:{PORT}')\n",
"\n",
" message = UserMessage(\n",
" content='hello world, write me a 2 sentence poem about the moon', role='user'\n",
@ -241,7 +273,11 @@
]
}
],
"metadata": {},
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}