phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions
llama-stack/docs/source/contributing/new_api_provider.md
Nathan Weinberg 0f14378135
fix: broken "core concepts" link in docs website (#940) 
# What does this PR do?
The `core concepts` link on [this
page](https://llama-stack.readthedocs.io/en/latest/contributing/new_api_provider.html)
is currently broken - this PR fixes that link

## Test Plan
Ran local docs build as described
[here](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md#building-the-documentation)

## Sources
N/A

## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the
other checks if that's the case).
- [x] Ran pre-commit to handle lint / formatting issues.
- [x] Read the [contributor
guideline](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md),
      Pull Request section?
- [x] Updated relevant documentation.
- [x] Wrote necessary unit or integration tests.

Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
2025-02-03 13:46:34 -08:00

2.1 KiB
Raw Blame History

Adding a New API Provider

This guide will walk you through the process of adding a new API provider to Llama Stack.

  • Begin by reviewing the core concepts of Llama Stack and choose the API your provider belongs to (Inference, Safety, VectorIO, etc.)
  • Determine the provider type ({repopath}Remote::llama_stack/providers/remote or {repopath}Inline::llama_stack/providers/inline). Remote providers make requests to external services, while inline providers execute implementation locally.
  • Add your provider to the appropriate {repopath}Registry::llama_stack/providers/registry/. Specify pip dependencies necessary.
  • Update any distribution {repopath}Templates::llama_stack/templates/ build.yaml and run.yaml files if they should include your provider by default. Run {repopath}llama_stack/scripts/distro_codegen.py if necessary.

Here are some example PRs to help you get started:

Testing the Provider

1. Integration Testing

  • Create integration tests that use real provider instances and configurations
  • For remote services, test actual API interactions
  • Avoid mocking at the provider level since adapter layers tend to be thin
  • Reference examples in {repopath}tests/client-sdk

2. Unit Testing (Optional)

  • Add unit tests for provider-specific functionality
  • See examples in {repopath}llama_stack/providers/tests/inference/test_text_inference.py

3. End-to-End Testing

  1. Start a Llama Stack server with your new provider
  2. Test using client requests
  3. Verify compatibility with existing client scripts in the llama-stack-apps repository
  4. Document which scripts are compatible with your provider

Submitting Your PR

  1. Ensure all tests pass
  2. Include a comprehensive test plan in your PR summary
  3. Document any known limitations or considerations
  4. Submit your pull request for review