phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

More doc cleanup

Browse source
This commit is contained in:
Ashwin Bharambe 2024-11-22 14:37:22 -08:00
parent 900b0556e7
commit c2c53d0272
6 changed files with 34 additions and 121 deletions
Download patch file Download diff file Expand all files Collapse all files

62
docs/source/distributions/index.md
View file

@ -1,4 +1,13 @@
# Starting a Llama Stack
```{toctree}
:maxdepth: 3
:hidden:
self_hosted_distro/index
remote_hosted_distro/index
building_distro
ondevice_distro/index
```
As mentioned in the [Concepts](../concepts/index), Llama Stack Distributions are specific pre-packaged versions of the Llama Stack. These templates make it easy to get started quickly.
@ -19,56 +28,9 @@ If so, we suggest:
- [distribution-ollama](self_hosted_distro/ollama)
- **Do you have an API key for a remote inference provider like Fireworks, Together, etc.?** If so, we suggest:
- [distribution-together](#remote-hosted-distributions)
- [distribution-fireworks](#remote-hosted-distributions)
- [distribution-together](remote_hosted_distro/index)
- [distribution-fireworks](remote_hosted_distro/index)
- **Do you want to run Llama Stack inference on your iOS / Android device** If so, we suggest:
- [iOS](ondevice_distro/ios_sdk)
- [Android](ondevice_distro/android_sdk) (coming soon)
## Remote-Hosted Distributions
Remote-Hosted distributions are available endpoints serving Llama Stack API that you can directly connect to.
| Distribution | Endpoint | Inference | Agents | Memory | Safety | Telemetry |
|-------------|----------|-----------|---------|---------|---------|------------|
| Together | [https://llama-stack.together.ai](https://llama-stack.together.ai) | remote::together | meta-reference | remote::weaviate | meta-reference | meta-reference |
| Fireworks | [https://llamastack-preview.fireworks.ai](https://llamastack-preview.fireworks.ai) | remote::fireworks | meta-reference | remote::weaviate | meta-reference | meta-reference |
You can use `llama-stack-client` to interact with these endpoints. For example, to list the available models served by the Fireworks endpoint:
```bash
$ pip install llama-stack-client
$ llama-stack-client configure --endpoint https://llamastack-preview.fireworks.ai
$ llama-stack-client models list
```
## On-Device Distributions
On-device distributions are Llama Stack distributions that run locally on your iOS / Android device.
## Building Your Own Distribution
<TODO> talk about llama stack build --image-type conda, etc.
### Prerequisites
```bash
$ git clone git@github.com:meta-llama/llama-stack.git
```
### Troubleshooting
- If you encounter any issues, search through our [GitHub Issues](https://github.com/meta-llama/llama-stack/issues), or file an new issue.
- Use `--port <PORT>` flag to use a different port number. For docker run, update the `-p <PORT>:<PORT>` flag.
```{toctree}
:maxdepth: 3
remote_hosted_distro/index
ondevice_distro/index
```
- Android (coming soon)