---
title: watsonx Distribution
description: Use watsonx for running LLM inference in Llama Stack
sidebar_label: watsonx
sidebar_position: 2
---

# watsonx Distribution

The `llamastack/distribution-watsonx` distribution consists of the following provider configurations.

## Provider Configuration

| API | Provider(s) |
|-----|-------------|
| agents | `inline::meta-reference` |
| datasetio | `remote::huggingface`, `inline::localfs` |
| eval | `inline::meta-reference` |
| inference | `remote::watsonx`, `inline::sentence-transformers` |
| safety | `inline::llama-guard` |
| scoring | `inline::basic`, `inline::llm-as-judge`, `inline::braintrust` |
| telemetry | `inline::meta-reference` |
| tool_runtime | `remote::brave-search`, `remote::tavily-search`, `inline::rag-runtime`, `remote::model-context-protocol` |
| vector_io | `inline::faiss` |

## Environment Variables

The following environment variables can be configured:

- `LLAMASTACK_PORT`: Port for the Llama Stack distribution server (default: `5001`)
- `WATSONX_API_KEY`: watsonx API Key (default: ``)
- `WATSONX_PROJECT_ID`: watsonx Project ID (default: ``)

## Available Models

The following models are available by default:

- `meta-llama/llama-3-3-70b-instruct` (aliases: `meta-llama/Llama-3.3-70B-Instruct`)
- `meta-llama/llama-2-13b-chat` (aliases: `meta-llama/Llama-2-13b`)
- `meta-llama/llama-3-1-70b-instruct` (aliases: `meta-llama/Llama-3.1-70B-Instruct`)
- `meta-llama/llama-3-1-8b-instruct` (aliases: `meta-llama/Llama-3.1-8B-Instruct`)
- `meta-llama/llama-3-2-11b-vision-instruct` (aliases: `meta-llama/Llama-3.2-11B-Vision-Instruct`)
- `meta-llama/llama-3-2-1b-instruct` (aliases: `meta-llama/Llama-3.2-1B-Instruct`)
- `meta-llama/llama-3-2-3b-instruct` (aliases: `meta-llama/Llama-3.2-3B-Instruct`)
- `meta-llama/llama-3-2-90b-vision-instruct` (aliases: `meta-llama/Llama-3.2-90B-Vision-Instruct`)
- `meta-llama/llama-guard-3-11b-vision` (aliases: `meta-llama/Llama-Guard-3-11B-Vision`)

## Prerequisites

### API Keys

Make sure you have access to a watsonx API Key. You can get one by referring to the [watsonx.ai documentation](https://www.ibm.com/docs/en/masv-and-l/maximo-manage/continuous-delivery?topic=setup-create-watsonx-api-key).

## Running Llama Stack with watsonx

You can do this via venv or Docker which has a pre-built image.

### Via Docker

This method allows you to get started quickly without having to build the distribution code.

```bash
LLAMA_STACK_PORT=5001
docker run \
  -it \
  -p $LLAMA_STACK_PORT:$LLAMA_STACK_PORT \
  -v ./run.yaml:/root/my-run.yaml \
  llamastack/distribution-watsonx \
  --config /root/my-run.yaml \
  --port $LLAMA_STACK_PORT \
  --env WATSONX_API_KEY=$WATSONX_API_KEY \
  --env WATSONX_PROJECT_ID=$WATSONX_PROJECT_ID \
  --env WATSONX_BASE_URL=$WATSONX_BASE_URL
```

### Via venv

If you've set up your local development environment, you can also build the image using your local virtual environment:

```bash
llama stack build --distro watsonx --image-type venv
llama stack run ./run.yaml \
  --port 5001 \
  --env WATSONX_API_KEY=$WATSONX_API_KEY \
  --env WATSONX_PROJECT_ID=$WATSONX_PROJECT_ID \
  --env WATSONX_BASE_URL=$WATSONX_BASE_URL
```

## Use Cases

The watsonx distribution is ideal for:

- **Enterprise deployments** with IBM infrastructure
- **Production workloads** requiring enterprise support
- **Regulated industries** needing compliance and governance features
- **Hybrid cloud** deployments with IBM watsonx integration

## Getting Started

1. **Set up watsonx credentials** - Obtain API keys and project IDs from IBM watsonx
2. **Configure environment variables** - Set WATSONX_API_KEY and WATSONX_PROJECT_ID
3. **Run the distribution** - Use Docker or venv to start the Llama Stack server
4. **Test the connection** - Use llama-stack-client to verify the setup

## Related Guides

- **[Remote-Hosted Overview](./index)** - Overview of remote-hosted distributions
- **[Available Distributions](../list-of-distributions)** - Compare with other distributions
- **[Configuration Reference](../configuration)** - Understanding configuration options
- **[Building Custom Distributions](../building-distro)** - Create your own distribution
