phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-13 04:22:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fireworks distro

Browse source
This commit is contained in:
Xi Yan 2024-10-21 18:11:52 -07:00
parent 69d7011ed6
commit c52ba17d7b
5 changed files with 107 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

9
distributions/together/README.md
View file

@ -23,13 +23,14 @@ compose.yaml run.yaml
$ docker compose up
```
Replace <ENTER_YOUR_TGI_HOSTED_ENDPOINT> in `run.yaml` file with your TGI endpoint.
Make sure in you `run.yaml` file, you inference provider is pointing to the correct Together URL server endpoint. E.g.
```
inference:
- provider_id: tgi0
provider_type: remote::tgi
- provider_id: together
provider_type: remote::together
config:
url: <ENTER_YOUR_TGI_HOSTED_ENDPOINT>
url: https://api.together.xyz/v1
api_key: <optional api key>
```
### (Alternative) TGI server + llama stack run (Single Node GPU)