phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-31 02:13:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/llama_stack/providers/remote/inference
History
Ben Browning ac5dc8fae2 Add prompt_logprobs and guided_choice to OpenAI completions 
This adds the vLLM-specific extra_body parameters of prompt_logprobs
and guided_choice to our openai_completion inference endpoint. The
plan here would be to expand this to support all common optional
parameters of any of the OpenAI providers, allowing each provider to
use or ignore these parameters based on whether their server supports them.

Signed-off-by: Ben Browning <bbrownin@redhat.com>
2025-04-09 15:47:02 -04:00
..
anthropic feat(providers): Groq now uses LiteLLM openai-compat (#1303) 2025-02-27 13:16:50 -08:00
bedrock Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
cerebras Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
cerebras_openai_compat test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
databricks Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
fireworks test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
fireworks_openai_compat test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
gemini feat(providers): Groq now uses LiteLLM openai-compat (#1303) 2025-02-27 13:16:50 -08:00
groq test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
groq_openai_compat test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
nvidia Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
ollama Add prompt_logprobs and guided_choice to OpenAI completions 2025-04-09 15:47:02 -04:00
openai feat(providers): Groq now uses LiteLLM openai-compat (#1303) 2025-02-27 13:16:50 -08:00
passthrough Add prompt_logprobs and guided_choice to OpenAI completions 2025-04-09 15:47:02 -04:00
runpod Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
sambanova Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
sambanova_openai_compat test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
tgi Add unsupported OpenAI mixin to all remaining inference providers 2025-04-09 15:47:02 -04:00
together Add prompt_logprobs and guided_choice to OpenAI completions 2025-04-09 15:47:02 -04:00
together_openai_compat test: verification on provider's OAI endpoints (#1893) 2025-04-07 23:06:28 -07:00
vllm Add prompt_logprobs and guided_choice to OpenAI completions 2025-04-09 15:47:02 -04:00
__init__.py impls -> inline, adapters -> remote (#381) 2024-11-06 14:54:05 -08:00