forked from phoenix-oss/llama-stack-mirror

feat: added nvidia as safety provider (#1248 )

# What does this PR do?
Adds nvidia as a safety provider by interfacing with the nemo guardrails
microservice.
This enables checking user’s input or the LLM’s output against input and
output guardrails by using the `/v1/guardrails/checks` endpoint of the[
guardrails
API.](https://developer.nvidia.com/docs/nemo-microservices/guardrails/source/guides/checks-guide.html)

## Test Plan
Deploy nemo guardrails service following the documentation:
https://developer.nvidia.com/docs/nemo-microservices/guardrails/source/getting-started/deploy-docker.html

### Standalone:
```bash
(venv) local-cdgamarose@a1u1g-rome-0153:~/llama-stack$ pytest -v -s llama_stack/providers/tests/safety/test_safety.py --providers inference=nvidia,safety=nvidia --safety-shield meta/llama-3.1-8b-instruct

=================================================================================== test session starts ===================================================================================
platform linux -- Python 3.10.12, pytest-8.3.4, pluggy-1.5.0 -- /localhome/local-cdgamarose/llama-stack/venv/bin/python3
cachedir: .pytest_cache
metadata: {'Python': '3.10.12', 'Platform': 'Linux-5.15.0-122-generic-x86_64-with-glibc2.35', 'Packages': {'pytest': '8.3.4', 'pluggy': '1.5.0'}, 'Plugins': {'metadata': '3.1.1', 'asyncio': '0.25.3', 'anyio': '4.8.0', 'html': '4.1.1'}}
rootdir: /localhome/local-cdgamarose/llama-stack
configfile: pyproject.toml
plugins: metadata-3.1.1, asyncio-0.25.3, anyio-4.8.0, html-4.1.1
asyncio: mode=strict, asyncio_default_fixture_loop_scope=None
collected 2 items

llama_stack/providers/tests/safety/test_safety.py::TestSafety::test_shield_list[--inference=nvidia:safety=nvidia] Initializing NVIDIASafetyAdapter(http://0.0.0.0:7331)...
PASSED
llama_stack/providers/tests/safety/test_safety.py::TestSafety::test_run_shield[--inference=nvidia:safety=nvidia] PASSED

============================================================================== 2 passed, 2 warnings in 4.78s ==============================================================================

```
### Distribution:
```
llama stack run llama_stack/templates/nvidia/run-with-safety.yaml
curl -v -X 'POST' "http://localhost:8321/v1/safety/run-shield" -H 'accept: application/json' -H 'Content-Type: application/json' -d '{"shield_id": "meta/llama-3.1-8b-instruct", "messages":[{"role": "user", "content": "you are stupid"}]}'
{"violation":{"violation_level":"error","user_message":"Sorry I cannot do this.","metadata":{"self check input":{"status":"blocked"}}}}
```

[//]: # (## Documentation)

---------

Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>

2025-03-17 14:39:23 -07:00

2.7 KiB

Raw Blame History

NVIDIA Distribution

The llamastack/distribution-nvidia distribution consists of the following provider configurations.

API	Provider(s)
agents	`inline::meta-reference`
datasetio	`inline::localfs`
eval	`inline::meta-reference`
inference	`remote::nvidia`
safety	`remote::nvidia`
scoring	`inline::basic`
telemetry	`inline::meta-reference`
tool_runtime	`inline::rag-runtime`
vector_io	`inline::faiss`

Environment Variables

The following environment variables can be configured:

NVIDIA_API_KEY: NVIDIA API Key (default: ``)
GUARDRAILS_SERVICE_URL: URL for the NeMo Guardrails Service (default: http://0.0.0.0:7331)
INFERENCE_MODEL: Inference model (default: Llama3.1-8B-Instruct)
SAFETY_MODEL: Name of the model to use for safety (default: meta/llama-3.1-8b-instruct)

Models

The following models are available by default:

meta/llama3-8b-instruct (aliases: meta-llama/Llama-3-8B-Instruct)
meta/llama3-70b-instruct (aliases: meta-llama/Llama-3-70B-Instruct)
meta/llama-3.1-8b-instruct (aliases: meta-llama/Llama-3.1-8B-Instruct)
meta/llama-3.1-70b-instruct (aliases: meta-llama/Llama-3.1-70B-Instruct)
meta/llama-3.1-405b-instruct (aliases: meta-llama/Llama-3.1-405B-Instruct-FP8)
meta/llama-3.2-1b-instruct (aliases: meta-llama/Llama-3.2-1B-Instruct)
meta/llama-3.2-3b-instruct (aliases: meta-llama/Llama-3.2-3B-Instruct)
meta/llama-3.2-11b-vision-instruct (aliases: meta-llama/Llama-3.2-11B-Vision-Instruct)
meta/llama-3.2-90b-vision-instruct (aliases: meta-llama/Llama-3.2-90B-Vision-Instruct)
nvidia/llama-3.2-nv-embedqa-1b-v2
nvidia/nv-embedqa-e5-v5
nvidia/nv-embedqa-mistral-7b-v2
snowflake/arctic-embed-l

Prerequisite: API Keys

Make sure you have access to a NVIDIA API Key. You can get one by visiting https://build.nvidia.com/.

Running Llama Stack with NVIDIA

You can do this via Conda (build code) or Docker which has a pre-built image.

Via Docker

This method allows you to get started quickly without having to build the distribution code.

LLAMA_STACK_PORT=5001
docker run \
  -it \
  -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
  -v ./run.yaml:/root/my-run.yaml \
  llamastack/distribution-nvidia \
  --yaml-config /root/my-run.yaml \
  --port $LLAMA_STACK_PORT \
  --env NVIDIA_API_KEY=$NVIDIA_API_KEY

Via Conda

llama stack build --template nvidia --image-type conda
llama stack run ./run.yaml \
  --port 5001 \
  --env NVIDIA_API_KEY=$NVIDIA_API_KEY
  --env INFERENCE_MODEL=$INFERENCE_MODEL

2.7 KiB Raw Blame History