phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

Doc updates

Browse source
This commit is contained in:
Ashwin Bharambe 2025-01-23 20:43:10 -08:00
parent 9351a4b2d7
commit 2118f37350
5 changed files with 75 additions and 129 deletions
Download patch file Download diff file Expand all files Collapse all files

85
CONTRIBUTING.md
View file

@ -12,6 +12,57 @@ We actively welcome your pull requests.
5. Make sure your code lints. 5. Make sure your code lints.
6. If you haven't already, complete the Contributor License Agreement ("CLA"). 6. If you haven't already, complete the Contributor License Agreement ("CLA").
## Contributor License Agreement ("CLA")
In order to accept your pull request, we need you to submit a CLA. You only need
to do this once to work on any of Meta's open source projects.
Complete your CLA here: <https://code.facebook.com/cla>
## Issues
We use GitHub issues to track public bugs. Please ensure your description is
clear and has sufficient instructions to be able to reproduce the issue.
Meta has a [bounty program](http://facebook.com/whitehat/info) for the safe
disclosure of security bugs. In those cases, please go through the process
outlined on that page and do not file a public issue.
## Pre-commit Hooks
We use [pre-commit](https://pre-commit.com/) to run linting and formatting checks on your code. You can install the pre-commit hooks by running:
```bash
$ cd llama-stack
$ conda activate <your-environment>
$ pip install pre-commit
$ pre-commit install
```
After that, pre-commit hooks will run automatically before each commit.
## Coding Style
* 2 spaces for indentation rather than tabs
* 80 character line length
* ...
## Common Tasks
Some tips about common tasks you work on while contributing to Llama Stack:
### Using `llama stack build`
Building a stack image (conda / docker) will use the production version of the `llama-stack`, `llama-models` and `llama-stack-client` packages. If you are developing with a llama-stack repository checked out and need your code to be reflected in the stack image, set `LLAMA_STACK_DIR` and `LLAMA_MODELS_DIR` to the appropriate checked out directories when running any of the `llama` CLI commands.
Example:
```bash
$ cd work/
$ git clone https://github.com/meta-llama/llama-stack.git
$ git clone https://github.com/meta-llama/llama-models.git
$ cd llama-stack
$ LLAMA_STACK_DIR=$(pwd) LLAMA_MODELS_DIR=../llama-models llama stack build --template <...>
```
### Updating Provider Configurations ### Updating Provider Configurations
@ -31,40 +82,6 @@ make html
sphinx-autobuild source build/html sphinx-autobuild source build/html
``` ```
## Pre-commit Hooks
We use [pre-commit](https://pre-commit.com/) to run linting and formatting checks on your code. You can install the pre-commit hooks by running:
```bash
$ cd llama-stack
$ conda activate <your-environment>
$ pip install pre-commit
$ pre-commit install
```
After that, pre-commit hooks will run automatically before each commit.
## Contributor License Agreement ("CLA")
In order to accept your pull request, we need you to submit a CLA. You only need
to do this once to work on any of Meta's open source projects.
Complete your CLA here: <https://code.facebook.com/cla>
## Issues
We use GitHub issues to track public bugs. Please ensure your description is
clear and has sufficient instructions to be able to reproduce the issue.
Meta has a [bounty program](http://facebook.com/whitehat/info) for the safe
disclosure of security bugs. In those cases, please go through the process
outlined on that page and do not file a public issue.
## Coding Style
* 2 spaces for indentation rather than tabs
* 80 character line length
* ...
## Tips
* If you are developing with a llama-stack repository checked out and need your distribution to reflect changes from there, set `LLAMA_STACK_DIR` to that dir when running any of the `llama` CLI commands.
## License ## License
By contributing to Llama, you agree that your contributions will be licensed By contributing to Llama, you agree that your contributions will be licensed

14
docs/source/building_applications/telemetry.md
View file

@ -2,9 +2,7 @@
The Llama Stack telemetry system provides comprehensive tracing, metrics, and logging capabilities. It supports multiple sink types including OpenTelemetry, SQLite, and Console output. The Llama Stack telemetry system provides comprehensive tracing, metrics, and logging capabilities. It supports multiple sink types including OpenTelemetry, SQLite, and Console output.
#### Key Concepts ### Events
#### Events
The telemetry system supports three main types of events: The telemetry system supports three main types of events:
- **Unstructured Log Events**: Free-form log messages with severity levels - **Unstructured Log Events**: Free-form log messages with severity levels
@ -30,16 +28,16 @@ structured_log_event = SpanStartPayload(
) )
``` ```
#### Spans and Traces ### Spans and Traces
- **Spans**: Represent operations with timing and hierarchical relationships - **Spans**: Represent operations with timing and hierarchical relationships
- **Traces**: Collection of related spans forming a complete request flow - **Traces**: Collection of related spans forming a complete request flow
#### Sinks ### Sinks
- **OpenTelemetry**: Send events to an OpenTelemetry Collector. This is useful for visualizing traces in a tool like Jaeger. - **OpenTelemetry**: Send events to an OpenTelemetry Collector. This is useful for visualizing traces in a tool like Jaeger.
- **SQLite**: Store events in a local SQLite database. This is needed if you want to query the events later through the Llama Stack API. - **SQLite**: Store events in a local SQLite database. This is needed if you want to query the events later through the Llama Stack API.
- **Console**: Print events to the console. - **Console**: Print events to the console.
#### Providers ### Providers
#### Meta-Reference Provider #### Meta-Reference Provider
Currently, only the meta-reference provider is implemented. It can be configured to send events to three sink types: Currently, only the meta-reference provider is implemented. It can be configured to send events to three sink types:
@ -60,7 +58,7 @@ Here's an example that sends telemetry signals to all three sink types. Your con
sqlite_db_path: "/path/to/telemetry.db" sqlite_db_path: "/path/to/telemetry.db"
``` ```
#### Jaeger to visualize traces ### Jaeger to visualize traces
The `otel` sink works with any service compatible with the OpenTelemetry collector. Let's use Jaeger to visualize this data. The `otel` sink works with any service compatible with the OpenTelemetry collector. Let's use Jaeger to visualize this data.
@ -74,6 +72,6 @@ $ docker run --rm --name jaeger \
Once the Jaeger instance is running, you can visualize traces by navigating to http://localhost:16686/. Once the Jaeger instance is running, you can visualize traces by navigating to http://localhost:16686/.
#### Querying Traces Stored in SQLIte ### Querying Traces Stored in SQLite
The `sqlite` sink allows you to query traces without an external system. Here are some example queries. Refer to the notebook at [Llama Stack Building AI Applications](https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.ipynb) for more examples on how to query traces and spaces. The `sqlite` sink allows you to query traces without an external system. Here are some example queries. Refer to the notebook at [Llama Stack Building AI Applications](https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.ipynb) for more examples on how to query traces and spaces.

69
docs/source/contributing/index.md
View file

@ -1,73 +1,14 @@
# Contributing to Llama Stack # Contributing to Llama Stack
If you are interested in contributing to Llama Stack, this guide will cover some of the key topics that might help you get started. Start with the [Contributing Guide](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md) for some general tips. This section covers a few key topics in more detail.
Also, check out our [Contributing Guide](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md) for more details on how to contribute to Llama Stack.
## Adding a New API Provider
This guide will walk you through the process of adding a new API provider to Llama Stack.
### Getting Started
1. **Choose Your API Category**
- Determine which API category your provider belongs to (Inference, Safety, Agents, VectorIO)
- Review the core concepts of Llama Stack in the [concepts guide](../concepts/index.md)
2. **Determine Provider Type**
- **Remote Provider**: Makes requests to external services
- **Inline Provider**: Executes implementation locally
Reference existing implementations:
- {repopath}`Remote Providers::llama_stack/providers/remote`
- {repopath}`Inline Providers::llama_stack/providers/inline`
Example PRs:
- [Grok Inference Implementation](https://github.com/meta-llama/llama-stack/pull/609)
- [Nvidia Inference Implementation](https://github.com/meta-llama/llama-stack/pull/355)
- [Model context protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/665)
3. **Register Your Provider**
- Add your provider to the appropriate {repopath}`Registry::llama_stack/providers/registry/`
- Specify any required pip dependencies
4. **Integration**
- Update the run.yaml file to include your provider
- To make your provider a default option or create a new distribution, look at the teamplates in {repopath}`llama_stack/templates/` and run {repopath}`llama_stack/scripts/distro_codegen.py`
- Example PRs:
- [Adding Model Context Protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/816)
### Testing Guidelines
#### 1. Integration Testing
- Create integration tests that use real provider instances and configurations
- For remote services, test actual API interactions
- Avoid mocking at the provider level
- Reference examples in {repopath}`tests/client-sdk`
#### 2. Unit Testing (Optional)
- Add unit tests for provider-specific functionality
- See examples in {repopath}`llama_stack/providers/tests/inference/test_text_inference.py`
#### 3. End-to-End Testing
1. Start a Llama Stack server with your new provider
2. Test using client requests
3. Verify compatibility with existing client scripts in the [llama-stack-apps](https://github.com/meta-llama/llama-stack-apps/tree/main) repository
4. Document which scripts are compatible with your provider
### Submitting Your PR
1. Ensure all tests pass
2. Include a comprehensive test plan in your PR summary
3. Document any known limitations or considerations
4. Submit your pull request for review
- [Adding a New API Provider](new_api_provider.md) describes adding new API providers to the Stack.
- [Testing Llama Stack](testing.md) provides details about the testing framework and how to test providers and distributions.
```{toctree} ```{toctree}
:maxdepth: 1 :maxdepth: 1
:hidden:
new_api_provider new_api_provider
memory_api testing
``` ```

30
docs/source/contributing/new_api_provider.md
View file

@ -2,41 +2,25 @@
This guide will walk you through the process of adding a new API provider to Llama Stack. This guide will walk you through the process of adding a new API provider to Llama Stack.
## Getting Started
1. **Choose Your API Category** - Begin by reviewing the [core concepts](../concepts/) of Llama Stack and choose the API your provider belongs to (Inference, Safety, VectorIO, etc.)
- Determine which API category your provider belongs to (Inference, Safety, Agents, VectorIO) - Determine the provider type ({repopath}`Remote::llama_stack/providers/remote` or {repopath}`Inline::llama_stack/providers/inline`). Remote providers make requests to external services, while inline providers execute implementation locally.
- Review the core concepts of Llama Stack in the [concepts guide](../concepts/index.md) - Add your provider to the appropriate {repopath}`Registry::llama_stack/providers/registry/`. Specify pip dependencies necessary.
- Update any distribution {repopath}`Templates::llama_stack/templates/` build.yaml and run.yaml files if they should include your provider by default. Run {repopath}`llama_stack/scripts/distro_codegen.py` if necessary.
2. **Determine Provider Type**
- **Remote Provider**: Makes requests to external services
- **Inline Provider**: Executes implementation locally
Reference existing implementations: Here are some example PRs to help you get started:
- {repopath}`Remote Providers::llama_stack/providers/remote`
- {repopath}`Inline Providers::llama_stack/providers/inline`
Example PRs:
- [Grok Inference Implementation](https://github.com/meta-llama/llama-stack/pull/609) - [Grok Inference Implementation](https://github.com/meta-llama/llama-stack/pull/609)
- [Nvidia Inference Implementation](https://github.com/meta-llama/llama-stack/pull/355) - [Nvidia Inference Implementation](https://github.com/meta-llama/llama-stack/pull/355)
- [Model context protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/665) - [Model context protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/665)
3. **Register Your Provider**
- Add your provider to the appropriate {repopath}`Registry::llama_stack/providers/registry/`
- Specify any required pip dependencies
4. **Integration** ## Testing the Provider
- Update the run.yaml file to include your provider
- To make your provider a default option or create a new distribution, look at the teamplates in {repopath}`llama_stack/templates/` and run {repopath}`llama_stack/scripts/distro_codegen.py`
- Example PRs:
- [Adding Model Context Protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/816)
## Testing Guidelines
### 1. Integration Testing ### 1. Integration Testing
- Create integration tests that use real provider instances and configurations - Create integration tests that use real provider instances and configurations
- For remote services, test actual API interactions - For remote services, test actual API interactions
- Avoid mocking at the provider level - Avoid mocking at the provider level since adapter layers tend to be thin
- Reference examples in {repopath}`tests/client-sdk` - Reference examples in {repopath}`tests/client-sdk`
### 2. Unit Testing (Optional) ### 2. Unit Testing (Optional)

6
docs/source/contributing/testing.md Normal file
View file

@ -0,0 +1,6 @@
# Testing Llama Stack
Tests are of three different kinds:
- Unit tests
- Provider focused integration tests
- Client SDK tests