d73955a41e
# What does this PR do? Finding these issues while moving to github pages. ## Test Plan uv run --group docs sphinx-autobuild docs/source docs/build/html --write-all
1.6 KiB
Distributions
While there is a lot of flexibility to mix-and-match providers, often users will work with a specific set of providers (hardware support, contractual obligations, etc.) We therefore need to provide a convenient shorthand for such collections. We call this shorthand a Llama Stack Distribution or a Distro. One can think of it as specific pre-packaged versions of the Llama Stack. Here are some examples:
Remotely Hosted Distro: These are the simplest to consume from a user perspective. You can simply obtain the API key for these providers, point to a URL and have all Llama Stack APIs working out of the box. Currently, Fireworks and Together provide such easy-to-consume Llama Stack distributions.
Locally Hosted Distro: You may want to run Llama Stack on your own hardware. Typically though, you still need to use Inference via an external service. You can use providers like HuggingFace TGI, Fireworks, Together, etc. for this purpose. Or you may have access to GPUs and can run a vLLM or NVIDIA NIM instance. If you "just" have a regular desktop machine, you can use Ollama for inference. To provide convenient quick access to these options, we provide a number of such pre-configured locally-hosted Distros.
On-device Distro: To run Llama Stack directly on an edge device (mobile phone or a tablet), we provide Distros for iOS and Android