phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-18 18:28:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: add refresh_models support to inference adapters (default: false)

Browse source 
inference adapters can now configure `refresh_models: bool` to control periodic model listing from their providers

BREAKING CHANGE: together inference adapter default changed. previously always refreshed, now follows config.
This commit is contained in:
Matthew Farrellee 2025-10-07 07:23:07 -04:00
parent 509ac4a659
commit bc47900ec0
31 changed files with 33 additions and 67 deletions
Download patch file Download diff file Expand all files Collapse all files

4
llama_stack/providers/remote/inference/vllm/vllm.py
View file

@ -53,10 +53,6 @@ class VLLMInferenceAdapter(OpenAIMixin):
"You must provide a URL in run.yaml (or via the VLLM_URL environment variable) to use vLLM."
)
async def should_refresh_models(self) -> bool:
# Strictly respecting the refresh_models directive
return self.config.refresh_models
async def health(self) -> HealthResponse:
"""
Performs a health check by verifying connectivity to the remote vLLM server.