---
title: Creating External Providers
description: Comprehensive guide for developing custom Llama Stack providers
sidebar_label: Creation Guide
sidebar_position: 2
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Creating External Providers

This guide walks you through creating custom Llama Stack providers that live outside the main codebase.

## Configuration Methods

There are two ways to configure external providers in Llama Stack:

<Tabs>
<TabItem value="module" label="Module Method (Recommended)">

To enable external providers, add `module` into your build yaml, allowing Llama Stack to install the required package corresponding to the external provider.

An example entry in your build.yaml should look like:

```yaml
- provider_type: remote::ramalama
  module: ramalama_stack
```

This method automatically handles package installation and provider discovery.

</TabItem>
<TabItem value="directory" label="Directory Method (Deprecated)">

:::warning[Deprecated Method]
This method is in the process of being deprecated in favor of the `module` method.
:::

You can configure the `external_providers_dir` in your Llama Stack configuration:

```yaml
external_providers_dir: ~/.llama/providers.d/
```

The external provider directory should contain your external provider specifications.

</TabItem>
</Tabs>

## Directory Structure

When using the directory method, the external providers directory should follow this structure:

```
providers.d/
  remote/
    inference/
      custom_ollama.yaml
      vllm.yaml
    vector_io/
      qdrant.yaml
    safety/
      llama-guard.yaml
  inline/
    inference/
      custom_ollama.yaml
      vllm.yaml
    vector_io/
      qdrant.yaml
    safety/
      llama-guard.yaml
```

Each YAML file in these directories defines a provider specification for that particular API.

## Provider Types

Llama Stack supports two types of external providers:

### 🌐 Remote Providers

Providers that communicate with external services (e.g., cloud APIs)

#### Remote Provider Specification

Here's an example for a custom Ollama provider:

```yaml
adapter:
  adapter_type: custom_ollama
  pip_packages:
  - ollama
  - aiohttp
  config_class: llama_stack_ollama_provider.config.OllamaImplConfig
  module: llama_stack_ollama_provider
api_dependencies: []
optional_api_dependencies: []
```

#### Adapter Configuration

The `adapter` section defines how to load and configure the provider:

- **`adapter_type`**: A unique identifier for this adapter
- **`pip_packages`**: List of Python packages required by the provider
- **`config_class`**: The full path to the configuration class
- **`module`**: The Python module containing the provider implementation

### 🏠 Inline Providers

Providers that run locally within the Llama Stack process.

#### Inline Provider Specification

Here's an example for a custom vector store provider:

```yaml
module: llama_stack_vector_provider
config_class: llama_stack_vector_provider.config.VectorStoreConfig
pip_packages:
  - faiss-cpu
  - numpy
api_dependencies:
  - inference
optional_api_dependencies:
  - vector_io
provider_data_validator: llama_stack_vector_provider.validator.VectorStoreValidator
container_image: custom-vector-store:latest  # optional
```

#### Inline Provider Fields

- **`module`**: The Python module containing the provider implementation
- **`config_class`**: The full path to the configuration class
- **`pip_packages`**: List of Python packages required by the provider
- **`api_dependencies`**: List of Llama Stack APIs that this provider depends on
- **`optional_api_dependencies`**: List of optional Llama Stack APIs that this provider can use
- **`provider_data_validator`**: Optional validator for provider data
- **`container_image`**: Optional container image to use instead of pip packages

## Required Implementation

### All Providers

All providers must contain a `get_provider_spec` function in their `provider` module. This is a standardized structure that Llama Stack expects and is necessary for getting things such as the config class.

The `get_provider_spec` method returns a structure identical to the `adapter`. An example function may look like:

```python
from llama_stack.providers.datatypes import (
    ProviderSpec,
    Api,
    AdapterSpec,
    remote_provider_spec,
)


def get_provider_spec() -> ProviderSpec:
    return remote_provider_spec(
        api=Api.inference,
        adapter=AdapterSpec(
            adapter_type="ramalama",
            pip_packages=["ramalama>=0.8.5", "pymilvus"],
            config_class="ramalama_stack.config.RamalamaImplConfig",
            module="ramalama_stack",
        ),
    )
```

### Remote Providers

Remote providers must expose a `get_adapter_impl()` function in their module that takes two arguments:

1. **`config`**: An instance of the provider's config class
2. **`deps`**: A dictionary of API dependencies

This function must return an instance of the provider's adapter class that implements the required protocol for the API.

**Example:**
```python
async def get_adapter_impl(
    config: OllamaImplConfig, deps: Dict[Api, Any]
) -> OllamaInferenceAdapter:
    return OllamaInferenceAdapter(config)
```

### Inline Providers

Inline providers must expose a `get_provider_impl()` function in their module that takes two arguments:

1. **`config`**: An instance of the provider's config class
2. **`deps`**: A dictionary of API dependencies

**Example:**
```python
async def get_provider_impl(
    config: VectorStoreConfig, deps: Dict[Api, Any]
) -> VectorStoreImpl:
    impl = VectorStoreImpl(config, deps[Api.inference])
    await impl.initialize()
    return impl
```

## Dependencies

The provider package must be installed on the system. For example:

```bash
$ uv pip show llama-stack-ollama-provider
Name: llama-stack-ollama-provider
Version: 0.1.0
Location: /path/to/venv/lib/python3.10/site-packages
```

## Best Practices

### 📦 Package Naming
Use the prefix `llama-stack-provider-` for your provider packages to make them easily identifiable.

### 🏷️ Version Management
Keep your provider package versioned and compatible with the Llama Stack version you're using.

### ⚡ Dependencies
Only include the minimum required dependencies in your provider package.

### 📚 Documentation
Include clear documentation in your provider package about:
- Installation requirements
- Configuration options
- Usage examples
- Any limitations or known issues

### 🧪 Testing
Include tests in your provider package to ensure it works correctly with Llama Stack.

You can refer to the [integration tests guide](https://github.com/meta-llama/llama-stack/blob/main/tests/integration/README.md) for more information. Execute the test for the Provider type you are developing.

## Troubleshooting

If your external provider isn't being loaded:

1. **Check Module Path**: Verify that `module` points to a published pip package with a top level `provider` module including `get_provider_spec`.

2. **Check Directory Path**: Verify that the `external_providers_dir` path is correct and accessible.

3. **Validate YAML**: Ensure that the YAML files are properly formatted.

4. **Check Dependencies**: Ensure all required Python packages are installed.

5. **Review Logs**: Check the Llama Stack server logs for any error messages - turn on debug logging to get more information using `LLAMA_STACK_LOGGING=all=debug`.

6. **Verify Installation**: Verify that the provider package is installed in your Python environment if using `external_providers_dir`.

## Complete Examples

### Example 1: Using `external_providers_dir` (Custom Ollama Provider)

Here's a complete example of creating and using a custom Ollama provider:

#### 1. Create the Provider Package

```bash
mkdir -p llama-stack-provider-ollama
cd llama-stack-provider-ollama
git init
uv init
```

#### 2. Configure Package

Edit `pyproject.toml`:

```toml
[project]
name = "llama-stack-provider-ollama"
version = "0.1.0"
description = "Ollama provider for Llama Stack"
requires-python = ">=3.12"
dependencies = ["llama-stack", "pydantic", "ollama", "aiohttp"]
```

#### 3. Create Provider Specification

```yaml
# ~/.llama/providers.d/remote/inference/custom_ollama.yaml
adapter:
  adapter_type: custom_ollama
  pip_packages: ["ollama", "aiohttp"]
  config_class: llama_stack_provider_ollama.config.OllamaImplConfig
  module: llama_stack_provider_ollama
api_dependencies: []
optional_api_dependencies: []
```

#### 4. Install the Provider

```bash
uv pip install -e .
```

#### 5. Configure Llama Stack

```yaml
external_providers_dir: ~/.llama/providers.d/
```

The provider will now be available in Llama Stack with the type `remote::custom_ollama`.

### Example 2: Using `module` (ramalama-stack)

[ramalama-stack](https://github.com/containers/ramalama-stack) is a recognized external provider that supports installation via module.

To install Llama Stack with this external provider a user can provide the following build.yaml:

```yaml
version: 2
distribution_spec:
  description: Use (an external) Ramalama server for running LLM inference
  container_image: null
  providers:
    inference:
    - provider_type: remote::ramalama
      module: ramalama_stack==0.3.0a0
image_type: venv
image_name: null
external_providers_dir: null
additional_pip_packages:
- aiosqlite
- sqlalchemy[asyncio]
```

No other steps are required other than `llama stack build` and `llama stack run`. The build process will use `module` to install all of the provider dependencies, retrieve the spec, etc.

The provider will now be available in Llama Stack with the type `remote::ramalama`.

## Next Steps

1. **📖 Study Examples**: Review the [Known External Providers](./external-providers-list) for real-world examples
2. **🔧 Start Building**: Create your first provider using the templates above
3. **🧪 Test Integration**: Use the integration test framework to validate your provider
4. **📦 Publish**: Share your provider with the community via PyPI
5. **📢 Share**: Add your provider to our [community list](./external-providers-list)

## Related Resources

- **[External Providers Overview](./index)** - Understanding external providers
- **[Known External Providers](./external-providers-list)** - Community provider directory
- **[API Protocols](/docs/api/)** - Provider interface specifications
- **[Integration Tests](https://github.com/meta-llama/llama-stack/blob/main/tests/integration/README.md)** - Testing your providers
