phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-06-27 18:50:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/docs/source/contributing/new_api_provider.md
Jeffrey Lind 5fc2ee6f77
Fix URLs to Llama Stack Read the Docs Webpages (#547) 
# What does this PR do?

Many of the URLs pointing to the Llama Stack's Read The Docs webpages
were broken, presumably due to recent refactor of the documentation.
This PR fixes all effected URLs throughout the repository.
2024-11-29 10:11:50 -06:00

2.2 KiB
Raw Blame History

Adding a New API Provider

This guide contains references to walk you through adding a new API provider.

  1. First, decide which API your provider falls into (e.g. Inference, Safety, Agents, Memory).

  2. Decide whether your provider is a remote provider, or inline implmentation. A remote provider is a provider that makes a remote request to an service. An inline provider is a provider where implementation is executed locally. Checkout the examples, and follow the structure to add your own API provider. Please find the following code pointers:

    • {repopath}Remote Providers::llama_stack/providers/remote
    • {repopath}Inline Providers::llama_stack/providers/inline

  3. Build a Llama Stack distribution with your API provider.

  4. Test your code!

Testing your newly added API providers

  1. Start with an integration test for your provider. That means we will instantiate the real provider, pass it real configuration and if it is a remote service, we will actually hit the remote service. We strongly discourage mocking for these tests at the provider level. Llama Stack is first and foremost about integration so we need to make sure stuff works end-to-end. See {repopath}llama_stack/providers/tests/inference/test_text_inference.py for an example.

  2. In addition, if you want to unit test functionality within your provider, feel free to do so. You can find some tests in tests/ but they aren't well supported so far.

  3. Test with a client-server Llama Stack setup. (a) Start a Llama Stack server with your own distribution which includes the new provider. (b) Send a client request to the server. See llama_stack/apis/<api>/client.py for how this is done. These client scripts can serve as lightweight tests.

You can find more complex client scripts llama-stack-apps repo. Note down which scripts works and do not work with your distribution.

Submit your PR

After you have fully tested your newly added API provider, submit a PR with the attached test plan. You must have a Test Plan in the summary section of your PR.