diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4713f564a..e42d6db75 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,6 +12,57 @@ We actively welcome your pull requests. 5. Make sure your code lints. 6. If you haven't already, complete the Contributor License Agreement ("CLA"). +## Contributor License Agreement ("CLA") +In order to accept your pull request, we need you to submit a CLA. You only need +to do this once to work on any of Meta's open source projects. + +Complete your CLA here: + +## Issues +We use GitHub issues to track public bugs. Please ensure your description is +clear and has sufficient instructions to be able to reproduce the issue. + +Meta has a [bounty program](http://facebook.com/whitehat/info) for the safe +disclosure of security bugs. In those cases, please go through the process +outlined on that page and do not file a public issue. + + +## Pre-commit Hooks + +We use [pre-commit](https://pre-commit.com/) to run linting and formatting checks on your code. You can install the pre-commit hooks by running: + +```bash +$ cd llama-stack +$ conda activate +$ pip install pre-commit +$ pre-commit install +``` + +After that, pre-commit hooks will run automatically before each commit. + + +## Coding Style +* 2 spaces for indentation rather than tabs +* 80 character line length +* ... + +## Common Tasks + +Some tips about common tasks you work on while contributing to Llama Stack: + +### Using `llama stack build` + +Building a stack image (conda / docker) will use the production version of the `llama-stack`, `llama-models` and `llama-stack-client` packages. If you are developing with a llama-stack repository checked out and need your code to be reflected in the stack image, set `LLAMA_STACK_DIR` and `LLAMA_MODELS_DIR` to the appropriate checked out directories when running any of the `llama` CLI commands. + +Example: +```bash +$ cd work/ +$ git clone https://github.com/meta-llama/llama-stack.git +$ git clone https://github.com/meta-llama/llama-models.git +$ cd llama-stack +$ LLAMA_STACK_DIR=$(pwd) LLAMA_MODELS_DIR=../llama-models llama stack build --template <...> +``` + ### Updating Provider Configurations @@ -31,40 +82,6 @@ make html sphinx-autobuild source build/html ``` -## Pre-commit Hooks - -We use [pre-commit](https://pre-commit.com/) to run linting and formatting checks on your code. You can install the pre-commit hooks by running: - -```bash -$ cd llama-stack -$ conda activate -$ pip install pre-commit -$ pre-commit install -``` - -After that, pre-commit hooks will run automatically before each commit. - -## Contributor License Agreement ("CLA") -In order to accept your pull request, we need you to submit a CLA. You only need -to do this once to work on any of Meta's open source projects. - -Complete your CLA here: - -## Issues -We use GitHub issues to track public bugs. Please ensure your description is -clear and has sufficient instructions to be able to reproduce the issue. - -Meta has a [bounty program](http://facebook.com/whitehat/info) for the safe -disclosure of security bugs. In those cases, please go through the process -outlined on that page and do not file a public issue. - -## Coding Style -* 2 spaces for indentation rather than tabs -* 80 character line length -* ... - -## Tips -* If you are developing with a llama-stack repository checked out and need your distribution to reflect changes from there, set `LLAMA_STACK_DIR` to that dir when running any of the `llama` CLI commands. ## License By contributing to Llama, you agree that your contributions will be licensed diff --git a/docs/source/building_applications/telemetry.md b/docs/source/building_applications/telemetry.md index 25b637821..4b4397d1e 100644 --- a/docs/source/building_applications/telemetry.md +++ b/docs/source/building_applications/telemetry.md @@ -2,9 +2,7 @@ The Llama Stack telemetry system provides comprehensive tracing, metrics, and logging capabilities. It supports multiple sink types including OpenTelemetry, SQLite, and Console output. -#### Key Concepts - -#### Events +### Events The telemetry system supports three main types of events: - **Unstructured Log Events**: Free-form log messages with severity levels @@ -30,16 +28,16 @@ structured_log_event = SpanStartPayload( ) ``` -#### Spans and Traces +### Spans and Traces - **Spans**: Represent operations with timing and hierarchical relationships - **Traces**: Collection of related spans forming a complete request flow -#### Sinks +### Sinks - **OpenTelemetry**: Send events to an OpenTelemetry Collector. This is useful for visualizing traces in a tool like Jaeger. - **SQLite**: Store events in a local SQLite database. This is needed if you want to query the events later through the Llama Stack API. - **Console**: Print events to the console. -#### Providers +### Providers #### Meta-Reference Provider Currently, only the meta-reference provider is implemented. It can be configured to send events to three sink types: @@ -60,7 +58,7 @@ Here's an example that sends telemetry signals to all three sink types. Your con sqlite_db_path: "/path/to/telemetry.db" ``` -#### Jaeger to visualize traces +### Jaeger to visualize traces The `otel` sink works with any service compatible with the OpenTelemetry collector. Let's use Jaeger to visualize this data. @@ -74,6 +72,6 @@ $ docker run --rm --name jaeger \ Once the Jaeger instance is running, you can visualize traces by navigating to http://localhost:16686/. -#### Querying Traces Stored in SQLIte +### Querying Traces Stored in SQLite The `sqlite` sink allows you to query traces without an external system. Here are some example queries. Refer to the notebook at [Llama Stack Building AI Applications](https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.ipynb) for more examples on how to query traces and spaces. diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index 6ae76d23f..8f89ea9f2 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -1,73 +1,14 @@ # Contributing to Llama Stack -If you are interested in contributing to Llama Stack, this guide will cover some of the key topics that might help you get started. - -Also, check out our [Contributing Guide](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md) for more details on how to contribute to Llama Stack. - - - -## Adding a New API Provider - -This guide will walk you through the process of adding a new API provider to Llama Stack. - -### Getting Started - -1. **Choose Your API Category** - - Determine which API category your provider belongs to (Inference, Safety, Agents, VectorIO) - - Review the core concepts of Llama Stack in the [concepts guide](../concepts/index.md) - -2. **Determine Provider Type** - - **Remote Provider**: Makes requests to external services - - **Inline Provider**: Executes implementation locally - - Reference existing implementations: - - {repopath}`Remote Providers::llama_stack/providers/remote` - - {repopath}`Inline Providers::llama_stack/providers/inline` - - Example PRs: - - [Grok Inference Implementation](https://github.com/meta-llama/llama-stack/pull/609) - - [Nvidia Inference Implementation](https://github.com/meta-llama/llama-stack/pull/355) - - [Model context protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/665) - -3. **Register Your Provider** - - Add your provider to the appropriate {repopath}`Registry::llama_stack/providers/registry/` - - Specify any required pip dependencies - -4. **Integration** - - Update the run.yaml file to include your provider - - To make your provider a default option or create a new distribution, look at the teamplates in {repopath}`llama_stack/templates/` and run {repopath}`llama_stack/scripts/distro_codegen.py` - - Example PRs: - - [Adding Model Context Protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/816) - -### Testing Guidelines - -#### 1. Integration Testing -- Create integration tests that use real provider instances and configurations -- For remote services, test actual API interactions -- Avoid mocking at the provider level -- Reference examples in {repopath}`tests/client-sdk` - -#### 2. Unit Testing (Optional) -- Add unit tests for provider-specific functionality -- See examples in {repopath}`llama_stack/providers/tests/inference/test_text_inference.py` - -#### 3. End-to-End Testing -1. Start a Llama Stack server with your new provider -2. Test using client requests -3. Verify compatibility with existing client scripts in the [llama-stack-apps](https://github.com/meta-llama/llama-stack-apps/tree/main) repository -4. Document which scripts are compatible with your provider - -### Submitting Your PR - -1. Ensure all tests pass -2. Include a comprehensive test plan in your PR summary -3. Document any known limitations or considerations -4. Submit your pull request for review +Start with the [Contributing Guide](https://github.com/meta-llama/llama-stack/blob/main/CONTRIBUTING.md) for some general tips. This section covers a few key topics in more detail. +- [Adding a New API Provider](new_api_provider.md) describes adding new API providers to the Stack. +- [Testing Llama Stack](testing.md) provides details about the testing framework and how to test providers and distributions. ```{toctree} :maxdepth: 1 +:hidden: new_api_provider -memory_api +testing ``` diff --git a/docs/source/contributing/new_api_provider.md b/docs/source/contributing/new_api_provider.md index 439021685..1dd836a16 100644 --- a/docs/source/contributing/new_api_provider.md +++ b/docs/source/contributing/new_api_provider.md @@ -2,41 +2,25 @@ This guide will walk you through the process of adding a new API provider to Llama Stack. -## Getting Started -1. **Choose Your API Category** - - Determine which API category your provider belongs to (Inference, Safety, Agents, VectorIO) - - Review the core concepts of Llama Stack in the [concepts guide](../concepts/index.md) +- Begin by reviewing the [core concepts](../concepts/) of Llama Stack and choose the API your provider belongs to (Inference, Safety, VectorIO, etc.) +- Determine the provider type ({repopath}`Remote::llama_stack/providers/remote` or {repopath}`Inline::llama_stack/providers/inline`). Remote providers make requests to external services, while inline providers execute implementation locally. +- Add your provider to the appropriate {repopath}`Registry::llama_stack/providers/registry/`. Specify pip dependencies necessary. +- Update any distribution {repopath}`Templates::llama_stack/templates/` build.yaml and run.yaml files if they should include your provider by default. Run {repopath}`llama_stack/scripts/distro_codegen.py` if necessary. -2. **Determine Provider Type** - - **Remote Provider**: Makes requests to external services - - **Inline Provider**: Executes implementation locally - Reference existing implementations: - - {repopath}`Remote Providers::llama_stack/providers/remote` - - {repopath}`Inline Providers::llama_stack/providers/inline` - - Example PRs: +Here are some example PRs to help you get started: - [Grok Inference Implementation](https://github.com/meta-llama/llama-stack/pull/609) - [Nvidia Inference Implementation](https://github.com/meta-llama/llama-stack/pull/355) - [Model context protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/665) -3. **Register Your Provider** - - Add your provider to the appropriate {repopath}`Registry::llama_stack/providers/registry/` - - Specify any required pip dependencies -4. **Integration** - - Update the run.yaml file to include your provider - - To make your provider a default option or create a new distribution, look at the teamplates in {repopath}`llama_stack/templates/` and run {repopath}`llama_stack/scripts/distro_codegen.py` - - Example PRs: - - [Adding Model Context Protocol Tool Runtime](https://github.com/meta-llama/llama-stack/pull/816) - -## Testing Guidelines +## Testing the Provider ### 1. Integration Testing - Create integration tests that use real provider instances and configurations - For remote services, test actual API interactions -- Avoid mocking at the provider level +- Avoid mocking at the provider level since adapter layers tend to be thin - Reference examples in {repopath}`tests/client-sdk` ### 2. Unit Testing (Optional) diff --git a/docs/source/contributing/testing.md b/docs/source/contributing/testing.md new file mode 100644 index 000000000..47bf9dea7 --- /dev/null +++ b/docs/source/contributing/testing.md @@ -0,0 +1,6 @@ +# Testing Llama Stack + +Tests are of three different kinds: +- Unit tests +- Provider focused integration tests +- Client SDK tests