llama-stack-mirror/llama_stack/providers/registry
Akram Ben Aissi 5e74bc7fcf Add dynamic authentication token forwarding support for vLLM provider
This enables per-request authentication tokens for vLLM providers, supporting use cases like RAG operations where different requests may need different authentication tokens. The implementation follows the same pattern as other providers like Together AI, Fireworks, and Passthrough.

- Add LiteLLMOpenAIMixin that manages the vllm_api_token properly

Usage:

- Static: VLLM_API_TOKEN env var or config.api_token
- Dynamic: X-LlamaStack-Provider-Data header with vllm_api_token
All existing functionality is preserved while adding new dynamic capabilities.

Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
2025-09-15 13:01:12 +01:00
..
__init__.py API Updates (#73) 2024-09-17 19:51:35 -07:00
agents.py fix: only load mcp when enabled in tool_group (#2621) 2025-07-04 20:27:05 +05:30
batches.py chore: remove openai dependency from providers (#3398) 2025-09-11 10:19:59 +02:00
datasetio.py fix(deps): bump datasets versions for all providers (#3382) 2025-09-08 15:13:42 -07:00
eval.py docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
files.py feat: Add S3 Files Provider (#3202) 2025-08-22 10:38:59 -04:00
inference.py Add dynamic authentication token forwarding support for vLLM provider 2025-09-15 13:01:12 +01:00
post_training.py fix(deps): bump datasets versions for all providers (#3382) 2025-09-08 15:13:42 -07:00
safety.py docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
scoring.py chore: remove openai dependency from providers (#3398) 2025-09-11 10:19:59 +02:00
telemetry.py docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
tool_runtime.py feat: Updating Rag Tool to use Files API and Vector Stores API (#3344) 2025-09-06 07:26:34 -06:00
vector_io.py feat: implement keyword, vector and hybrid search inside vector stores for PGVector provider (#3064) 2025-08-29 16:30:12 +02:00