mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-10-16 06:53:47 +00:00
remove llama configure
This commit is contained in:
parent
ec16d2cabb
commit
c5663fea72
3 changed files with 48 additions and 34 deletions
|
@ -6,14 +6,14 @@
|
|||
"source": [
|
||||
"## Safety API 101\n",
|
||||
"\n",
|
||||
"This document talks about the Safety APIs in Llama Stack.\n",
|
||||
"This document talks about the Safety APIs in Llama Stack. Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html).\n",
|
||||
"\n",
|
||||
"As outlined in our [Responsible Use Guide](https://www.llama.com/docs/how-to-guides/responsible-use-guide-resources/), LLM apps should deploy appropriate system level safeguards to mitigate safety and security risks of LLM system, similar to the following diagram:\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"To that goal, Llama Stack uses **Prompt Guard** and **Llama Guard 3** to secure our system. Here are the quick introduction about them.\n",
|
||||
"\n",
|
||||
"Before you begin, please ensure Llama Stack is installed and set up by following the [Getting Started Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/index.html)."
|
||||
"<div>\n",
|
||||
"<img src=\"../_static/safety_system.webp\" alt=\"Figure 1: Safety System\" width=\"1000\"/>\n",
|
||||
"</div>\n",
|
||||
"To that goal, Llama Stack uses **Prompt Guard** and **Llama Guard 3** to secure our system. Here are the quick introduction about them.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
@ -41,30 +41,55 @@
|
|||
"source": [
|
||||
"### Configure Safety\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"$ llama stack configure ~/.conda/envsllamastack-my-local-stack/my-local-stack-build.yaml\n",
|
||||
"We can first take a look at our build yaml file for my-local-stack:\n",
|
||||
"\n",
|
||||
"....\n",
|
||||
"> Configuring provider `(meta-reference)`\n",
|
||||
"Do you want to configure llama_guard_shield? (y/n): y\n",
|
||||
"Entering sub-configuration for llama_guard_shield:\n",
|
||||
"Enter value for model (existing: Llama-Guard-3-1B) (required):\n",
|
||||
"Enter value for excluded_categories (existing: []) (required):\n",
|
||||
"Enter value for enable_prompt_guard (existing: False) (optional): True\n",
|
||||
"```bash\n",
|
||||
"cat /home/$USER/.llama/builds/conda/my-local-stack-run.yaml\n",
|
||||
"\n",
|
||||
"version: '2'\n",
|
||||
"built_at: '2024-10-23T12:20:07.467045'\n",
|
||||
"image_name: my-local-stack\n",
|
||||
"docker_image: null\n",
|
||||
"conda_env: my-local-stack\n",
|
||||
"apis:\n",
|
||||
"- inference\n",
|
||||
"- safety\n",
|
||||
"- agents\n",
|
||||
"- memory\n",
|
||||
"- telemetry\n",
|
||||
"providers:\n",
|
||||
" inference:\n",
|
||||
" - provider_id: meta-reference\n",
|
||||
" provider_type: meta-reference\n",
|
||||
" config:\n",
|
||||
" model: Llama3.1-8B-Instruct\n",
|
||||
" torch_seed: 42\n",
|
||||
" max_seq_len: 8192\n",
|
||||
" max_batch_size: 1\n",
|
||||
" create_distributed_process_group: true\n",
|
||||
" checkpoint_dir: null\n",
|
||||
" safety:\n",
|
||||
" - provider_id: meta-reference\n",
|
||||
" provider_type: meta-reference\n",
|
||||
" config:\n",
|
||||
" llama_guard_shield:\n",
|
||||
" model: Llama-Guard-3-1B\n",
|
||||
" excluded_categories: []\n",
|
||||
" enable_prompt_guard: true\n",
|
||||
"....\n",
|
||||
"```\n",
|
||||
"As you can see, we did basic configuration above and configured:\n",
|
||||
"As you can see, we have the safety feature configured in the yaml:\n",
|
||||
"- Llama Guard safety shield with model `Llama-Guard-3-1B`\n",
|
||||
"- Prompt Guard safety shield, which by default will be `Prompt-Guard-86M` model.\n",
|
||||
"\n",
|
||||
"You will also need to manually change the yaml to support `Llama3.2-3B-Instruct` running along with `Llama-Guard-3-1B`. The yaml file for inference section should be like this:\n",
|
||||
"However, you will also need to manually change the yaml to support `Llama3.1-8B-Instruct` running along with `Llama-Guard-3-1B`. The yaml file for inference section should be like this:\n",
|
||||
"\n",
|
||||
"```bash\n",
|
||||
"inference:\n",
|
||||
" - provider_id: meta0\n",
|
||||
" - provider_id: meta-reference\n",
|
||||
" provider_type: meta-reference\n",
|
||||
" config:\n",
|
||||
" model: Llama3.2-3B-Instruct\n",
|
||||
" model: Llama3.1-8B-Instruct\n",
|
||||
" torch_seed: null\n",
|
||||
" max_seq_len: 4096\n",
|
||||
" max_batch_size: 1\n",
|
||||
|
@ -81,7 +106,7 @@
|
|||
" checkpoint_dir: null\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Now, you can start the server by `llama stack run my-local-stack --port 5000`\n",
|
||||
"Now, you can start the server by `llama stack run my-local-stack --port 5000`. \n",
|
||||
"\n",
|
||||
"After the server started, you can test safety example using the follow code:"
|
||||
]
|
||||
|
@ -95,7 +120,7 @@
|
|||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
|
@ -105,7 +130,7 @@
|
|||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue