phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

Fix bedrock inference impl

Browse source
This commit is contained in:
Ashwin Bharambe 2024-12-16 14:22:34 -08:00
parent eb37fba9da
commit c2f7905fa4
5 changed files with 47 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

7
docs/source/distributions/self_hosted_distro/bedrock.md
View file

@ -28,6 +28,13 @@ The following environment variables can be configured:
- `LLAMASTACK_PORT`: Port for the Llama Stack distribution server (default: `5001`)
### Models
The following models are available by default:
- `meta-llama/Llama-3.1-8B-Instruct (meta.llama3-1-8b-instruct-v1:0)`
- `meta-llama/Llama-3.1-70B-Instruct (meta.llama3-1-70b-instruct-v1:0)`
- `meta-llama/Llama-3.1-405B-Instruct-FP8 (meta.llama3-1-405b-instruct-v1:0)`
### Prerequisite: API Keys