phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-05 10:13:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

portkey fix md

Browse source
This commit is contained in:
siddharthsambharia-portkey 2024-12-20 19:30:17 +05:30
parent 71de927b29
commit 2251643c9a
1 changed files with 11 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

17
llama_stack/templates/portkey/doc_template.md
View file

@ -1,4 +1,4 @@
# Cerebras Distribution # Portkey Distribution
The `llamastack/distribution-{{ name }}` distribution consists of the following provider configurations. The `llamastack/distribution-{{ name }}` distribution consists of the following provider configurations.
@ -27,10 +27,10 @@ The following models are available by default:
### Prerequisite: API Keys ### Prerequisite: API Keys
Make sure you have access to a Cerebras API Key. You can get one by visiting [cloud.cerebras.ai](https://cloud.cerebras.ai/). Make sure you have access to a Portkey API Key and Virtual Key or Config ID. You can get these by visiting [app.portkey.ai](https://app.portkey.ai/).
## Running Llama Stack with Cerebras ## Running Llama Stack with Portkey
You can do this via Conda (build code) or Docker which has a pre-built image. You can do this via Conda (build code) or Docker which has a pre-built image.
@ -47,14 +47,19 @@ docker run \
llamastack/distribution-{{ name }} \ llamastack/distribution-{{ name }} \
--yaml-config /root/my-run.yaml \ --yaml-config /root/my-run.yaml \
--port $LLAMA_STACK_PORT \ --port $LLAMA_STACK_PORT \
--env CEREBRAS_API_KEY=$CEREBRAS_API_KEY --env PORTKEY_API_KEY=$PORTKEY_API_KEY
--env PORTKEY_VIRTUAL_KEY=$PORTKEY_VIRTUAL_KEY
--env PORTKEY_CONFIG_ID=$PORTKEY_CONFIG_ID
``` ```
### Via Conda ### Via Conda
```bash ```bash
llama stack build --template cerebras --image-type conda llama stack build --template portkey --image-type conda
llama stack run ./run.yaml \ llama stack run ./run.yaml \
--port 5001 \ --port 5001 \
--env CEREBRAS_API_KEY=$CEREBRAS_API_KEY --env PORTKEY_API_KEY=$PORTKEY_API_KEY
--env PORTKEY_VIRTUAL_KEY=$PORTKEY_VIRTUAL_KEY
--env PORTKEY_CONFIG_ID=$PORTKEY_CONFIG_ID
``` ```