mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-07-05 05:35:22 +00:00
4d2bd2d39e
* verify dockers * together distro verified * readme * fireworks distro * fireworks compose up * fireworks verified
28 lines
1.3 KiB
Markdown
28 lines
1.3 KiB
Markdown
# Meta Reference Distribution
|
|
|
|
The `llamastack/distribution-meta-reference-gpu` distribution consists of the following provider configurations.
|
|
|
|
|
|
| **API** | **Inference** | **Agents** | **Memory** | **Safety** | **Telemetry** |
|
|
|----------------- |--------------- |---------------- |-------------------------------------------------- |---------------- |---------------- |
|
|
| **Provider(s)** | meta-reference | meta-reference | meta-reference, remote::pgvector, remote::chroma | meta-reference | meta-reference |
|
|
|
|
|
|
### Start the Distribution (Single Node GPU)
|
|
|
|
> [!NOTE]
|
|
> This assumes you have access to GPU to start a local server with access to your GPU.
|
|
|
|
|
|
> [!NOTE]
|
|
> `~/.llama` should be the path containing downloaded weights of Llama models.
|
|
|
|
|
|
To download and start running a pre-built docker container, you may use the following commands:
|
|
|
|
```
|
|
docker run -it -p 5000:5000 -v ~/.llama:/root/.llama -v ./run.yaml:/root/my-run.yaml --gpus=all distribution-meta-reference-gpu --yaml_config /root/my-run.yaml
|
|
```
|
|
|
|
### Alternative (Build and start distribution locally via conda)
|
|
- You may checkout the [Getting Started](../../docs/getting_started.md) for more details on building locally via conda and starting up a meta-reference distribution.
|