diff --git a/README.md b/README.md index 1198b416a..8103cdadc 100644 --- a/README.md +++ b/README.md @@ -62,12 +62,12 @@ A Distribution is where APIs and Providers are assembled together to provide a c | **Distribution** | **Llama Stack Docker** | Start This Distribution | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** | |:----------------: |:------------------------------------------: |:-----------------------: |:------------------: |:------------------: |:------------------: |:------------------: |:------------------: | -| Meta Reference | [llamastack/distribution-meta-reference-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/meta-reference-gpu.html) | meta-reference | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| Meta Reference Quantized | [llamastack/distribution-meta-reference-quantized-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-quantized-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/meta-reference-quantized-gpu.html) | meta-reference-quantized | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| Ollama | [llamastack/distribution-ollama](https://hub.docker.com/repository/docker/llamastack/distribution-ollama/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/ollama.html) | remote::ollama | meta-reference | remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| TGI | [llamastack/distribution-tgi](https://hub.docker.com/repository/docker/llamastack/distribution-tgi/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/tgi.html) | remote::tgi | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| Together | [llamastack/distribution-together](https://hub.docker.com/repository/docker/llamastack/distribution-together/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/hosted_distro/together.html) | remote::together | meta-reference | remote::weaviate | meta-reference | meta-reference | -| Fireworks | [llamastack/distribution-fireworks](https://hub.docker.com/repository/docker/llamastack/distribution-fireworks/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/hosted_distro/fireworks.html) | remote::fireworks | meta-reference | remote::weaviate | meta-reference | meta-reference | +| Meta Reference | [llamastack/distribution-meta-reference-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/meta-reference-gpu.html) | meta-reference | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| Meta Reference Quantized | [llamastack/distribution-meta-reference-quantized-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-quantized-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/meta-reference-quantized-gpu.html) | meta-reference-quantized | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| Ollama | [llamastack/distribution-ollama](https://hub.docker.com/repository/docker/llamastack/distribution-ollama/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/ollama.html) | remote::ollama | meta-reference | remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| TGI | [llamastack/distribution-tgi](https://hub.docker.com/repository/docker/llamastack/distribution-tgi/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/tgi.html) | remote::tgi | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| Together | [llamastack/distribution-together](https://hub.docker.com/repository/docker/llamastack/distribution-together/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/together.html) | remote::together | meta-reference | remote::weaviate | meta-reference | meta-reference | +| Fireworks | [llamastack/distribution-fireworks](https://hub.docker.com/repository/docker/llamastack/distribution-fireworks/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/fireworks.html) | remote::fireworks | meta-reference | remote::weaviate | meta-reference | meta-reference | ## Installation You have two ways to install this repository: diff --git a/docs/source/getting_started/distributions/index.md b/docs/source/getting_started/distributions/index.md index 8df4fd113..087e08ce4 100644 --- a/docs/source/getting_started/distributions/index.md +++ b/docs/source/getting_started/distributions/index.md @@ -4,15 +4,15 @@ A Distribution is where APIs and Providers are assembled together to provide a c We offer three types of distributions: -1. [Deployable Distribution](./deployable_distro/index.md): If you want to run Llama Stack inference on your local machine. -2. [Hosted Distribution](./hosted_distro/index.md): If you want to connect to a remote hosted inference provider. +1. [Deployable Distribution](./self_hosted_distro/index.md): If you want to run Llama Stack inference on your local machine. +2. [Hosted Distribution](./remote_hosted_distro/index.md): If you want to connect to a remote hosted inference provider. 3. [On-device Distribution](./ondevice_distro/index.md): If you want to run Llama Stack inference on your iOS / Android device. ```{toctree} :maxdepth: 1 :hidden: -deployable_distro/index +self_hosted_distro/index hosted_distro/index ondevice_distro/index ``` diff --git a/docs/source/getting_started/distributions/hosted_distro/fireworks.md b/docs/source/getting_started/distributions/remote_hosted_distro/fireworks.md similarity index 100% rename from docs/source/getting_started/distributions/hosted_distro/fireworks.md rename to docs/source/getting_started/distributions/remote_hosted_distro/fireworks.md diff --git a/docs/source/getting_started/distributions/hosted_distro/index.md b/docs/source/getting_started/distributions/remote_hosted_distro/index.md similarity index 55% rename from docs/source/getting_started/distributions/hosted_distro/index.md rename to docs/source/getting_started/distributions/remote_hosted_distro/index.md index 91b790ca8..59585109a 100644 --- a/docs/source/getting_started/distributions/hosted_distro/index.md +++ b/docs/source/getting_started/distributions/remote_hosted_distro/index.md @@ -1,11 +1,11 @@ -# Hosted Distribution +# Remote Hosted Distribution -Hosted distributions are distributions connecting to remote hosted services through Llama Stack server. Inference is done through remote providers. These are useful if you have an API key for a remote inference provider like Fireworks, Together, etc. +Remote Hosted distributions are distributions connecting to remote hosted services through Llama Stack server. Inference is done through remote providers. These are useful if you have an API key for a remote inference provider like Fireworks, Together, etc. | **Distribution** | **Llama Stack Docker** | Start This Distribution | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** | |:----------------: |:------------------------------------------: |:-----------------------: |:------------------: |:------------------: |:------------------: |:------------------: |:------------------: | -| Together | [llamastack/distribution-together](https://hub.docker.com/repository/docker/llamastack/distribution-together/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/hosted_distro/together.html) | remote::together | meta-reference | remote::weaviate | meta-reference | meta-reference | -| Fireworks | [llamastack/distribution-fireworks](https://hub.docker.com/repository/docker/llamastack/distribution-fireworks/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/hosted_distro/fireworks.html) | remote::fireworks | meta-reference | remote::weaviate | meta-reference | meta-reference | +| Together | [llamastack/distribution-together](https://hub.docker.com/repository/docker/llamastack/distribution-together/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/together.html) | remote::together | meta-reference | remote::weaviate | meta-reference | meta-reference | +| Fireworks | [llamastack/distribution-fireworks](https://hub.docker.com/repository/docker/llamastack/distribution-fireworks/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/fireworks.html) | remote::fireworks | meta-reference | remote::weaviate | meta-reference | meta-reference | ```{toctree} :maxdepth: 1 diff --git a/docs/source/getting_started/distributions/hosted_distro/together.md b/docs/source/getting_started/distributions/remote_hosted_distro/together.md similarity index 100% rename from docs/source/getting_started/distributions/hosted_distro/together.md rename to docs/source/getting_started/distributions/remote_hosted_distro/together.md diff --git a/docs/source/getting_started/distributions/deployable_distro/dell-tgi.md b/docs/source/getting_started/distributions/self_hosted_distro/dell-tgi.md similarity index 100% rename from docs/source/getting_started/distributions/deployable_distro/dell-tgi.md rename to docs/source/getting_started/distributions/self_hosted_distro/dell-tgi.md diff --git a/docs/source/getting_started/distributions/deployable_distro/index.md b/docs/source/getting_started/distributions/self_hosted_distro/index.md similarity index 61% rename from docs/source/getting_started/distributions/deployable_distro/index.md rename to docs/source/getting_started/distributions/self_hosted_distro/index.md index b67401bdb..ec2a25f26 100644 --- a/docs/source/getting_started/distributions/deployable_distro/index.md +++ b/docs/source/getting_started/distributions/self_hosted_distro/index.md @@ -1,13 +1,13 @@ -# Deployable Distribution +# Self-Hosted Distribution We offer deployable distributions where you can host your own Llama Stack server using local inference. | **Distribution** | **Llama Stack Docker** | Start This Distribution | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** | |:----------------: |:------------------------------------------: |:-----------------------: |:------------------: |:------------------: |:------------------: |:------------------: |:------------------: | -| Meta Reference | [llamastack/distribution-meta-reference-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/meta-reference-gpu.html) | meta-reference | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| Meta Reference Quantized | [llamastack/distribution-meta-reference-quantized-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-quantized-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/meta-reference-quantized-gpu.html) | meta-reference-quantized | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| Ollama | [llamastack/distribution-ollama](https://hub.docker.com/repository/docker/llamastack/distribution-ollama/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/ollama.html) | remote::ollama | meta-reference | remote::pgvector; remote::chromadb | meta-reference | meta-reference | -| TGI | [llamastack/distribution-tgi](https://hub.docker.com/repository/docker/llamastack/distribution-tgi/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/tgi.html) | remote::tgi | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| Meta Reference | [llamastack/distribution-meta-reference-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/meta-reference-gpu.html) | meta-reference | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| Meta Reference Quantized | [llamastack/distribution-meta-reference-quantized-gpu](https://hub.docker.com/repository/docker/llamastack/distribution-meta-reference-quantized-gpu/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/meta-reference-quantized-gpu.html) | meta-reference-quantized | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| Ollama | [llamastack/distribution-ollama](https://hub.docker.com/repository/docker/llamastack/distribution-ollama/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/ollama.html) | remote::ollama | meta-reference | remote::pgvector; remote::chromadb | meta-reference | meta-reference | +| TGI | [llamastack/distribution-tgi](https://hub.docker.com/repository/docker/llamastack/distribution-tgi/general) | [Guide](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/tgi.html) | remote::tgi | meta-reference | meta-reference; remote::pgvector; remote::chromadb | meta-reference | meta-reference | ```{toctree} :maxdepth: 1 diff --git a/docs/source/getting_started/distributions/deployable_distro/meta-reference-gpu.md b/docs/source/getting_started/distributions/self_hosted_distro/meta-reference-gpu.md similarity index 100% rename from docs/source/getting_started/distributions/deployable_distro/meta-reference-gpu.md rename to docs/source/getting_started/distributions/self_hosted_distro/meta-reference-gpu.md diff --git a/docs/source/getting_started/distributions/deployable_distro/meta-reference-quantized-gpu.md b/docs/source/getting_started/distributions/self_hosted_distro/meta-reference-quantized-gpu.md similarity index 100% rename from docs/source/getting_started/distributions/deployable_distro/meta-reference-quantized-gpu.md rename to docs/source/getting_started/distributions/self_hosted_distro/meta-reference-quantized-gpu.md diff --git a/docs/source/getting_started/distributions/deployable_distro/ollama.md b/docs/source/getting_started/distributions/self_hosted_distro/ollama.md similarity index 100% rename from docs/source/getting_started/distributions/deployable_distro/ollama.md rename to docs/source/getting_started/distributions/self_hosted_distro/ollama.md diff --git a/docs/source/getting_started/distributions/deployable_distro/tgi.md b/docs/source/getting_started/distributions/self_hosted_distro/tgi.md similarity index 100% rename from docs/source/getting_started/distributions/deployable_distro/tgi.md rename to docs/source/getting_started/distributions/self_hosted_distro/tgi.md diff --git a/docs/source/getting_started/index.md b/docs/source/getting_started/index.md index 523491d18..de1b02db3 100644 --- a/docs/source/getting_started/index.md +++ b/docs/source/getting_started/index.md @@ -34,16 +34,16 @@ Running inference on the underlying Llama model is one of the most critical requ - **Do you have access to a machine with powerful GPUs?** If so, we suggest: - - [`distribution-meta-reference-gpu`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/meta-reference-gpu.html) + - [`distribution-meta-reference-gpu`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/meta-reference-gpu.html) - [`distribution-tgi`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/tgi.html) - **Are you running on a "regular" desktop machine?** If so, we suggest: - - [`distribution-ollama`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/deployable_distro/ollama.html) + - [`distribution-ollama`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/self_hosted_distro/ollama.html) - **Do you have an API key for a remote inference provider like Fireworks, Together, etc.?** If so, we suggest: - - [`distribution-together`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/hosted_distro/together.html) - - [`distribution-fireworks`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/hosted_distro/fireworks.html) + - [`distribution-together`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/together.html) + - [`distribution-fireworks`](https://llama-stack.readthedocs.io/en/latest/getting_started/distributions/remote_hosted_distro/fireworks.html) ### Quick Start Commands