phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-14 11:52:37 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Improve VLLM model discovery error handling

Browse source 
• Add comprehensive error handling in check_model_availability method
• Provide helpful error messages with actionable solutions for 404 errors
• Warn when API token is set but model discovery is disabled
This commit is contained in:
Akram Ben Aissi 2025-10-06 12:56:05 +02:00
parent e9214f9004
commit e28bc93635
15 changed files with 69 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

2
llama_stack/distributions/postgres-demo/run.yaml
View file

@ -16,7 +16,7 @@ providers:
max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
api_token: ${env.VLLM_API_TOKEN:=fake}
tls_verify: ${env.VLLM_TLS_VERIFY:=true}
allow_listing_models: ${env.VLLM_ALLOW_LISTING_MODELS:=true}
enable_model_discovery: ${env.VLLM_ENABLE_MODEL_DISCOVERY:=true}
- provider_id: sentence-transformers
provider_type: inline::sentence-transformers
vector_io: