fix: rewording

Signed-off-by: Jeff MAURY <jmaury@redhat.com>
This commit is contained in:
Jeff MAURY 2025-03-26 15:51:05 +01:00 committed by Philippe Martin
parent dd86427ce3
commit aa68e98b7a
5 changed files with 11 additions and 13 deletions

View file

@ -26,7 +26,7 @@ The `llamastack/distribution-podman-ai-lab` distribution consists of the followi
| vector_io | `inline::faiss`, `remote::chromadb`, `remote::pgvector` | | vector_io | `inline::faiss`, `remote::chromadb`, `remote::pgvector` |
You should use this distribution if you have a regular desktop machine without very powerful GPUs. Of course, if you have powerful GPUs, you can still continue using this distribution since Ollama supports GPU acceleration. You should use this distribution if you have a regular desktop machine without very powerful GPUs. Of course, if you have powerful GPUs, you can still continue using this distribution since Podman AI Lab supports GPU acceleration.
### Environment Variables ### Environment Variables
@ -34,12 +34,12 @@ The following environment variables can be configured:
- `LLAMA_STACK_PORT`: Port for the Llama Stack distribution server (default: `5001`) - `LLAMA_STACK_PORT`: Port for the Llama Stack distribution server (default: `5001`)
- `PODMAN_AI_LAB_URL`: URL of the Podman AI Lab server (default: `http://127.0.0.1:10434`) - `PODMAN_AI_LAB_URL`: URL of the Podman AI Lab server (default: `http://127.0.0.1:10434`)
- `SAFETY_MODEL`: Safety model loaded into the Ollama server (default: `meta-llama/Llama-Guard-3-1B`) - `SAFETY_MODEL`: Safety model loaded into the Podman AI Lab server (default: `meta-llama/Llama-Guard-3-1B`)
## Setting up Podman AI Lab server ## Setting up Podman AI Lab server
Please check the [Podman AI Lab Documentation](https://github.com/containers/podman-desktop-extension-ai-lab) on how to install and run Ollama. After installing Ollama, you need to run `ollama serve` to start the server. Please check the [Podman AI Lab Documentation](https://github.com/containers/podman-desktop-extension-ai-lab) on how to install and run Podman AI Lab.
If you are using Llama Stack Safety / Shield APIs, you will also need to pull and run the safety model. If you are using Llama Stack Safety / Shield APIs, you will also need to pull and run the safety model.
@ -47,7 +47,6 @@ If you are using Llama Stack Safety / Shield APIs, you will also need to pull an
```bash ```bash
export SAFETY_MODEL="meta-llama/Llama-Guard-3-1B" export SAFETY_MODEL="meta-llama/Llama-Guard-3-1B"
# ollama names this model differently, and we must use the ollama name when loading the model
export PODMAN_AI_LAB_SAFETY_MODEL="llama-guard3:1b" export PODMAN_AI_LAB_SAFETY_MODEL="llama-guard3:1b"
``` ```
@ -81,7 +80,7 @@ docker run \
-it \ -it \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \ -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-v ~/.llama:/root/.llama \ -v ~/.llama:/root/.llama \
-v ./llama_stack/templates/ollama/run-with-safety.yaml:/root/my-run.yaml \ -v ./llama_stack/templates/podman-ai-lab/run-with-safety.yaml:/root/my-run.yaml \
llamastack/distribution-podman-ai-lab \ llamastack/distribution-podman-ai-lab \
--yaml-config /root/my-run.yaml \ --yaml-config /root/my-run.yaml \
--port $LLAMA_STACK_PORT \ --port $LLAMA_STACK_PORT \

View file

@ -14,7 +14,7 @@ The `llamastack/distribution-{{ name }}` distribution consists of the following
{{ providers_table }} {{ providers_table }}
You should use this distribution if you have a regular desktop machine without very powerful GPUs. Of course, if you have powerful GPUs, you can still continue using this distribution since Ollama supports GPU acceleration. You should use this distribution if you have a regular desktop machine without very powerful GPUs. Of course, if you have powerful GPUs, you can still continue using this distribution since Podman AI Lab supports GPU acceleration.
{% if run_config_env_vars %} {% if run_config_env_vars %}
### Environment Variables ### Environment Variables
@ -29,7 +29,7 @@ The following environment variables can be configured:
## Setting up Podman AI Lab server ## Setting up Podman AI Lab server
Please check the [Podman AI Lab Documentation](https://github.com/containers/podman-desktop-extension-ai-lab) on how to install and run Ollama. After installing Ollama, you need to run `ollama serve` to start the server. Please check the [Podman AI Lab Documentation](https://github.com/containers/podman-desktop-extension-ai-lab) on how to install and run Podman AI Lab.
If you are using Llama Stack Safety / Shield APIs, you will also need to pull and run the safety model. If you are using Llama Stack Safety / Shield APIs, you will also need to pull and run the safety model.
@ -37,7 +37,6 @@ If you are using Llama Stack Safety / Shield APIs, you will also need to pull an
```bash ```bash
export SAFETY_MODEL="meta-llama/Llama-Guard-3-1B" export SAFETY_MODEL="meta-llama/Llama-Guard-3-1B"
# ollama names this model differently, and we must use the ollama name when loading the model
export PODMAN_AI_LAB_SAFETY_MODEL="llama-guard3:1b" export PODMAN_AI_LAB_SAFETY_MODEL="llama-guard3:1b"
``` ```
@ -71,7 +70,7 @@ docker run \
-it \ -it \
-p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \ -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
-v ~/.llama:/root/.llama \ -v ~/.llama:/root/.llama \
-v ./llama_stack/templates/ollama/run-with-safety.yaml:/root/my-run.yaml \ -v ./llama_stack/templates/podman-ai-lab/run-with-safety.yaml:/root/my-run.yaml \
llamastack/distribution-{{ name }} \ llamastack/distribution-{{ name }} \
--yaml-config /root/my-run.yaml \ --yaml-config /root/my-run.yaml \
--port $LLAMA_STACK_PORT \ --port $LLAMA_STACK_PORT \

View file

@ -50,7 +50,7 @@ def get_distribution_template() -> DistributionTemplate:
safety_model = ModelInput( safety_model = ModelInput(
model_id="${env.SAFETY_MODEL}", model_id="${env.SAFETY_MODEL}",
provider_id="ollama", provider_id="podman-ai-lab",
) )
default_tool_groups = [ default_tool_groups = [
ToolGroupInput( ToolGroupInput(
@ -131,7 +131,7 @@ def get_distribution_template() -> DistributionTemplate:
), ),
"SAFETY_MODEL": ( "SAFETY_MODEL": (
"meta-llama/Llama-Guard-3-1B", "meta-llama/Llama-Guard-3-1B",
"Safety model loaded into the Ollama server", "Safety model loaded into the Podman AI Lab server",
), ),
}, },
) )

View file

@ -1,7 +1,7 @@
# Report for Podman AI Lab distribution # Report for Podman AI Lab distribution
## Supported Models ## Supported Models
| Model Descriptor | ollama | | Model Descriptor | Podman AI Lab |
|:---|:---| |:---|:---|
| Llama-3-8B-Instruct | ❌ | | Llama-3-8B-Instruct | ❌ |
| Llama-3-70B-Instruct | ❌ | | Llama-3-70B-Instruct | ❌ |

View file

@ -109,7 +109,7 @@ metadata_store:
models: models:
- metadata: {} - metadata: {}
model_id: ${env.SAFETY_MODEL} model_id: ${env.SAFETY_MODEL}
provider_id: ollama provider_id: podman-ai-lab
model_type: llm model_type: llm
shields: shields:
- shield_id: ${env.SAFETY_MODEL} - shield_id: ${env.SAFETY_MODEL}