mirror of https://github.com/meta-llama/llama-stack.git synced 2025-06-28 19:04:19 +00:00

docs: Add note about distro_codegen.py and provider dependencies (#1175 )

# What does this PR do?

This expands upon the existing distro_codegen.py text in the new API
provider documentation to include a note about not including
provider-specific dependencies in the code path that builds the
distribution's template.

Our distro_codegen pre-commit hook will catch this case anyway, but this
attempts to inform provider authors ahead of time about that.

## Test Plan

I built the docs website locally via the following:
```
pip install docs/requirements.txt
sphinx-build -M html docs/source docs_output
```
Then, I opened that newly generated
`docs_output/html/contributing/new_api_provider.html` in my browser and
confirmed everything rendered correctly.

Signed-off-by: Ben Browning <bbrownin@redhat.com>

2025-02-20 09:23:46 -08:00

2.5 KiB

Raw Blame History

Adding a New API Provider

This guide will walk you through the process of adding a new API provider to Llama Stack.

Begin by reviewing the core concepts of Llama Stack and choose the API your provider belongs to (Inference, Safety, VectorIO, etc.)
Determine the provider type ({repopath}Remote::llama_stack/providers/remote or {repopath}Inline::llama_stack/providers/inline). Remote providers make requests to external services, while inline providers execute implementation locally.
Add your provider to the appropriate {repopath}Registry::llama_stack/providers/registry/. Specify pip dependencies necessary.
Update any distribution {repopath}Templates::llama_stack/templates/ build.yaml and run.yaml files if they should include your provider by default. Run {repopath}llama_stack/scripts/distro_codegen.py if necessary. Note that distro_codegen.py will fail if the new provider causes any distribution template to attempt to import provider-specific dependencies. This usually means the distribution's get_distribution_template() code path should only import any necessary Config or model alias definitions from each provider and not the provider's actual implementation.

Here are some example PRs to help you get started:

Testing the Provider

1. Integration Testing

Create integration tests that use real provider instances and configurations
For remote services, test actual API interactions
Avoid mocking at the provider level since adapter layers tend to be thin
Reference examples in {repopath}tests/client-sdk

2. Unit Testing (Optional)

Add unit tests for provider-specific functionality
See examples in {repopath}llama_stack/providers/tests/inference/test_text_inference.py

3. End-to-End Testing

Start a Llama Stack server with your new provider
Test using client requests
Verify compatibility with existing client scripts in the llama-stack-apps repository
Document which scripts are compatible with your provider

Submitting Your PR

Ensure all tests pass
Include a comprehensive test plan in your PR summary
Document any known limitations or considerations
Submit your pull request for review

2.5 KiB Raw Blame History