phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-08-07 02:58:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

add NVIDIA_BASE_URL and NVIDIA_API_KEY to control hosted vs local endpoints

Browse source 
use --env NVIDIA_BASE_URL=http://localhost:8000 to connect to a local NIM running at localhost:8000

 use --env NVIDIA_API_KEY=blah when connecting to hosted NIM, e.g. NVIDIA_BASE_URL=https://integrate.api.nvidia.com
This commit is contained in:
Matthew Farrellee 2025-01-29 11:52:43 -05:00
parent 9f709387e2
commit 115bf2a23f
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
llama_stack/templates/nvidia/run.yaml
View file

@ -15,8 +15,8 @@ providers:
- provider_id: nvidia - provider_id: nvidia
provider_type: remote::nvidia provider_type: remote::nvidia
config: config:
url: https://integrate.api.nvidia.com url: ${env.NVIDIA_BASE_URL:https://integrate.api.nvidia.com}
api_key: ${env.NVIDIA_API_KEY} api_key: ${env.NVIDIA_API_KEY:}
vector_io: vector_io:
- provider_id: faiss - provider_id: faiss
provider_type: inline::faiss provider_type: inline::faiss