Updates to ReadTheDocs (#859)

Move evals section to AI Agents section 
drop from top level and other minor fixes

docs/source/building_applications/agent_execution_loop.md

@@ -1,4 +1,4 @@
-# Agent Execution Loop
+## Agent Execution Loop
 
 Agents are the heart of complex AI applications. They combine inference, memory, safety, and tool usage into coherent workflows. At its core, an agent follows a sophisticated execution loop that enables multi-step reasoning, tool usage, and safety checks.
 

docs/source/building_applications/evals.md

@@ -1,8 +1,8 @@
-# Benchmark Evaluations
+# Evals
 
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/10CHyykee9j2OigaIcRv47BKG9mrNm0tJ?usp=sharing)
 
-Llama Stack provides the building blocks needed to run benchmark and application evaluations. This guide will walk you through how to use these components to run open benchmark evaluations. Visit our [Evaluation Concepts](../concepts/evaluation_concepts.md) guide for more details on how evaluations work in Llama Stack, and our [Evaluation Reference](../references/evals_reference/index.md) guide for a comprehensive reference on the APIs. Check out our [Colab notebook](https://colab.research.google.com/drive/10CHyykee9j2OigaIcRv47BKG9mrNm0tJ?usp=sharing) on working examples on how you can use Llama Stack for running benchmark evaluations.
+Llama Stack provides the building blocks needed to run benchmark and application evaluations. This guide will walk you through how to use these components to run open benchmark evaluations. Visit our [Evaluation Concepts](../concepts/evaluation_concepts.md) guide for more details on how evaluations work in Llama Stack, and our [Evaluation Reference](../references/evals_reference/index.md) guide for a comprehensive reference on the APIs.
 
 ### 1. Open Benchmark Model Evaluation
 

docs/source/building_applications/index.md

@@ -6,12 +6,14 @@ The best way to get started is to look at this notebook which walks through the
 
 **Notebook**: [Building AI Applications](docs/notebooks/Llama_Stack_Building_AI_Applications.ipynb)
 
-## Agentic Concepts
+Here are some key topics that will help you build effective agents:
+
 - **[Agent Execution Loop](agent_execution_loop)**
 - **[RAG](rag)**
 - **[Safety](safety)**
 - **[Tools](tools)**
 - **[Telemetry](telemetry)**
+- **[Evals](evals)**
 
 
 ```{toctree}
@@ -23,4 +25,5 @@ rag
 safety
 tools
 telemetry
+evals
 ```

docs/source/building_applications/telemetry.md

@@ -1,11 +1,10 @@
-# Telemetry
-
+## Telemetry
 
 The Llama Stack telemetry system provides comprehensive tracing, metrics, and logging capabilities. It supports multiple sink types including OpenTelemetry, SQLite, and Console output.
 
-## Key Concepts
+#### Key Concepts
 
-### Events
+#### Events
 The telemetry system supports three main types of events:
 
 - **Unstructured Log Events**: Free-form log messages with severity levels
@@ -31,24 +30,24 @@ structured_log_event = SpanStartPayload(
 )
 ```
 
-### Spans and Traces
+#### Spans and Traces
 - **Spans**: Represent operations with timing and hierarchical relationships
 - **Traces**: Collection of related spans forming a complete request flow
 
-### Sinks
+#### Sinks
 - **OpenTelemetry**: Send events to an OpenTelemetry Collector. This is useful for visualizing traces in a tool like Jaeger.
 - **SQLite**: Store events in a local SQLite database. This is needed if you want to query the events later through the Llama Stack API.
 - **Console**: Print events to the console.
 
-## Providers
+#### Providers
 
-### Meta-Reference Provider
+#### Meta-Reference Provider
 Currently, only the meta-reference provider is implemented. It can be configured to send events to three sink types:
 1) OpenTelemetry Collector
 2) SQLite
 3) Console
 
-## Configuration
+#### Configuration
 
 Here's an example that sends telemetry signals to all three sink types. Your configuration might use only one.
 ```yaml
@@ -61,7 +60,7 @@ Here's an example that sends telemetry signals to all three sink types. Your con
       sqlite_db_path: "/path/to/telemetry.db"
 ```
 
-## Jaeger to visualize traces
+#### Jaeger to visualize traces
 
 The `otel` sink works with any service compatible with the OpenTelemetry collector. Let's use Jaeger to visualize this data.
 
@@ -75,6 +74,6 @@ $ docker run --rm --name jaeger \
 
 Once the Jaeger instance is running, you can visualize traces by navigating to http://localhost:16686/.
 
-## Querying Traces Stored in SQLIte
+#### Querying Traces Stored in SQLIte
 
 The `sqlite` sink allows you to query traces without an external system. Here are some example queries. Refer to the notebook at [Llama Stack Building AI Applications](https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.ipynb) for more examples on how to query traces and spaces.

docs/source/contributing/memory_api.md

@@ -1,53 +0,0 @@
-# Memory API Providers
-
-This guide gives you references to switch between different memory API providers.
-
-##### pgvector
-1. Start running the pgvector server:
-
-```
-$ docker run --network host --name mypostgres -it -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword -e POSTGRES_USER=postgres -e POSTGRES_DB=postgres pgvector/pgvector:pg16
-```
-
-2. Edit the `run.yaml` file to point to the pgvector server.
-```
-memory:
-  - provider_id: pgvector
-    provider_type: remote::pgvector
-    config:
-      host: 127.0.0.1
-      port: 5432
-      db: postgres
-      user: postgres
-      password: mysecretpassword
-```
-
-> [!NOTE]
-> If you get a `RuntimeError: Vector extension is not installed.`. You will need to run `CREATE EXTENSION IF NOT EXISTS vector;` to include the vector extension. E.g.
-
-```
-docker exec -it mypostgres ./bin/psql -U postgres
-postgres=# CREATE EXTENSION IF NOT EXISTS vector;
-postgres=# SELECT extname from pg_extension;
- extname
-```
-
-3. Run `docker compose up` with the updated `run.yaml` file.
-
-##### chromadb
-1. Start running chromadb server
-```
-docker run -it --network host --name chromadb -p 6000:6000 -v ./chroma_vdb:/chroma/chroma -e IS_PERSISTENT=TRUE chromadb/chroma:latest
-```
-
-2. Edit the `run.yaml` file to point to the chromadb server.
-```
-memory:
-  - provider_id: remote::chromadb
-    provider_type: remote::chromadb
-    config:
-      host: localhost
-      port: 6000
-```
-
-3. Run `docker compose up` with the updated `run.yaml` file.

docs/source/index.md

@@ -15,7 +15,7 @@ Our goal is to provide pre-packaged implementations (aka "distributions") which
 
 - New to Llama Stack? Start with the [Introduction](introduction/index) to understand our motivation and vision.
 - Ready to build? Check out the [Quick Start](getting_started/index) to get started.
-- Need specific providers? Browse [Distributions](distributions/index) to see all the options available.
+- Need specific providers? Browse [Distributions](distributions/selection) to see all the options available.
 - Want to contribute? See the [Contributing](contributing/index) guide.
 
 ## Available SDKs
@@ -60,7 +60,6 @@ concepts/index
 distributions/index
 distributions/selection
 building_applications/index
-benchmark_evaluations/index
 playground/index
 contributing/index
 references/index