phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-30 07:39:38 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

remove llama configure

Browse source
This commit is contained in:
Kai Wu 2024-11-05 17:22:44 -08:00
parent ec16d2cabb
commit c5663fea72
3 changed files with 48 additions and 34 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/zero_to_hero_guide/02_Image_Chat101.ipynb
View file

@ -170,16 +170,6 @@
"\n",
"The next one in the series will teach you one of the favorite applications of Large Language Models: [Tool Calling](./03_Tool_Calling101.ipynb). Enjoy!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "277adb5d-a9cc-40ec-a961-2d194f88a00b",
"metadata": {},
"outputs": [],
"source": [
"#fin"
]
}
],
"metadata": {

67
docs/zero_to_hero_guide/05_Safety101.ipynb
View file

@ -6,14 +6,14 @@
"source": [
"## Safety API 101\n",
"\n",
"This document talks about the Safety APIs in Llama Stack.\n",
"This document talks about the Safety APIs in Llama Stack. Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html).\n",
"\n",
"As outlined in our [Responsible Use Guide](https://www.llama.com/docs/how-to-guides/responsible-use-guide-resources/), LLM apps should deploy appropriate system level safeguards to mitigate safety and security risks of LLM system, similar to the following diagram:\n",
"![Figure 1: Safety System](../_static/safety_system.webp)\n",
"\n",
"To that goal, Llama Stack uses **Prompt Guard** and **Llama Guard 3** to secure our system. Here are the quick introduction about them.\n",
"\n",
"Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html)."
"<div>\n",
"<img src=\"../_static/safety_system.webp\" alt=\"Figure 1: Safety System\" width=\"1000\"/>\n",
"</div>\n",
"To that goal, Llama Stack uses **Prompt Guard** and **Llama Guard 3** to secure our system. Here are the quick introduction about them.\n"
]
},
{
@ -41,30 +41,55 @@
"source": [
"### Configure Safety\n",
"\n",
"```bash\n",
"$ llama stack configure ~/.conda/envsllamastack-my-local-stack/my-local-stack-build.yaml\n",
"We can first take a look at our build yaml file for my-local-stack:\n",
"\n",
"....\n",
"> Configuring provider `(meta-reference)`\n",
"Do you want to configure llama_guard_shield? (y/n): y\n",
"Entering sub-configuration for llama_guard_shield:\n",
"Enter value for model (existing: Llama-Guard-3-1B) (required):\n",
"Enter value for excluded_categories (existing: []) (required):\n",
"Enter value for enable_prompt_guard (existing: False) (optional): True\n",
"```bash\n",
"cat /home/$USER/.llama/builds/conda/my-local-stack-run.yaml\n",
"\n",
"version: '2'\n",
"built_at: '2024-10-23T12:20:07.467045'\n",
"image_name: my-local-stack\n",
"docker_image: null\n",
"conda_env: my-local-stack\n",
"apis:\n",
"- inference\n",
"- safety\n",
"- agents\n",
"- memory\n",
"- telemetry\n",
"providers:\n",
" inference:\n",
" - provider_id: meta-reference\n",
" provider_type: meta-reference\n",
" config:\n",
" model: Llama3.1-8B-Instruct\n",
" torch_seed: 42\n",
" max_seq_len: 8192\n",
" max_batch_size: 1\n",
" create_distributed_process_group: true\n",
" checkpoint_dir: null\n",
" safety:\n",
" - provider_id: meta-reference\n",
" provider_type: meta-reference\n",
" config:\n",
" llama_guard_shield:\n",
" model: Llama-Guard-3-1B\n",
" excluded_categories: []\n",
" enable_prompt_guard: true\n",
"....\n",
"```\n",
"As you can see, we did basic configuration above and configured:\n",
"As you can see, we have the safety feature configured in the yaml:\n",
"- Llama Guard safety shield with model `Llama-Guard-3-1B`\n",
"- Prompt Guard safety shield, which by default will be `Prompt-Guard-86M` model.\n",
"\n",
"You will also need to manually change the yaml to support `Llama3.2-3B-Instruct` running along with `Llama-Guard-3-1B`. The yaml file for inference section should be like this:\n",
"However, you will also need to manually change the yaml to support `Llama3.1-8B-Instruct` running along with `Llama-Guard-3-1B`. The yaml file for inference section should be like this:\n",
"\n",
"```bash\n",
"inference:\n",
" - provider_id: meta0\n",
" - provider_id: meta-reference\n",
" provider_type: meta-reference\n",
" config:\n",
" model: Llama3.2-3B-Instruct\n",
" model: Llama3.1-8B-Instruct\n",
" torch_seed: null\n",
" max_seq_len: 4096\n",
" max_batch_size: 1\n",
@ -81,7 +106,7 @@
" checkpoint_dir: null\n",
"```\n",
"\n",
"Now, you can start the server by `llama stack run my-local-stack --port 5000`\n",
"Now, you can start the server by `llama stack run my-local-stack --port 5000`. \n",
"\n",
"After the server started, you can test safety example using the follow code:"
]
@ -95,7 +120,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
@ -105,7 +130,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 9,
"metadata": {},
"outputs": [
{

5
docs/zero_to_hero_guide/06_Agents101.ipynb
View file

@ -6,7 +6,7 @@
"source": [
"## Agentic API 101\n",
"\n",
"This document talks about the Agentic APIs in Llama Stack.\n",
"This document talks about the Agentic APIs in Llama Stack. Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html).\n",
"\n",
"Starting Llama 3.1 you can build agentic applications capable of:\n",
"\n",
@ -22,8 +22,7 @@
"- ability to execute tools, including a code execution environment, and loop using the model's multi-step reasoning process\n",
"\n",
"All of these components are now offered by a single Llama Stack Distribution. Llama Stack defines and standardizes these components and many others that are needed to make building Generative AI applications smoother. Various implementations of these APIs are then assembled together via a **Llama Stack Distribution**.\n",
"\n",
"Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html)."
"\n"
]
},
{