phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-06 10:37:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: nvidia provider no longer crashes sans API key (#4317)

Browse source 
# What does this PR do?
previously the nvidia provider would throw an exception if a hosted
instance was being used but no API key was set

modify this behavior to instead log an error informing users that a key
is needed to use a hosted NIM but still allow the server to boot

Closes #4295

## Test Plan
Run `uv run llama stack run --providers inference=remote::nvidia` with
`NVIDIA_API_KEY` unset - server now boots with logged error, where it
previously crashed
```
INFO     2025-12-04 14:16:26,156 llama_stack.providers.remote.inference.nvidia.nvidia:47 inference::nvidia: Initializing
         NVIDIAInferenceAdapter(https://integrate.api.nvidia.com/v1)...                                                 
ERROR    2025-12-04 14:16:26,157 llama_stack.providers.remote.inference.nvidia.nvidia:51 inference::nvidia: API key is  
         required for hosted NVIDIA NIM. Either provide an API key or use a self-hosted NIM.                            
INFO     2025-12-04 14:16:26,239 uvicorn.error:84 uncategorized: Started server process [251651]                        
INFO     2025-12-04 14:16:26,240 uvicorn.error:48 uncategorized: Waiting for application startup.                       
INFO     2025-12-04 14:16:26,244 llama_stack.core.server.server:168 core::server: Starting up Llama Stack server        
         (version: 0.4.0.dev0)                                                                                          
INFO     2025-12-04 14:16:26,245 llama_stack.core.stack:495 core: starting registry refresh task                        
INFO     2025-12-04 14:16:26,246 uvicorn.error:62 uncategorized: Application startup complete.                          
INFO     2025-12-04 14:16:26,246 uvicorn.error:216 uncategorized: Uvicorn running on http://['::', '0.0.0.0']:8321      
         (Press CTRL+C to quit)
```

Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
This commit is contained in:
Nathan Weinberg 2025-12-04 14:38:16 -05:00 • committed by GitHub
parent 686065fe27
commit 8bbcfc4f56
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/llama_stack/providers/remote/inference/nvidia/nvidia.py
View file

@ -48,7 +48,7 @@ class NVIDIAInferenceAdapter(OpenAIMixin):
if _is_nvidia_hosted(self.config): if _is_nvidia_hosted(self.config):
if not self.config.auth_credential: if not self.config.auth_credential:
raise RuntimeError( logger.error(
"API key is required for hosted NVIDIA NIM. Either provide an API key or use a self-hosted NIM." "API key is required for hosted NVIDIA NIM. Either provide an API key or use a self-hosted NIM."
) )