llama-stack/llama_stack/providers/remote/inference
ehhuang 047303e339
feat: introduce APIs for retrieving chat completion requests (#2145)
# What does this PR do?
This PR introduces APIs to retrieve past chat completion requests, which
will be used in the LS UI.

Our current `Telemetry` is ill-suited for this purpose as it's untyped
so we'd need to filter by obscure attribute names, making it brittle.

Since these APIs are 'provided by stack' and don't need to be
implemented by inference providers, we introduce a new InferenceProvider
class, containing the existing inference protocol, which is implemented
by inference providers.

The APIs are OpenAI-compliant, with an additional `input_messages`
field.


## Test Plan
This PR just adds the API and marks them provided_by_stack. S
tart stack server -> doesn't crash
2025-05-18 21:43:19 -07:00
..
anthropic chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
bedrock chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
cerebras chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
cerebras_openai_compat feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
databricks chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
fireworks chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
fireworks_openai_compat feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
gemini chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
groq chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
groq_openai_compat feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
llama_openai_compat feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
nvidia chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
ollama feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
openai feat: use openai-python for openai inference provider (#2193) 2025-05-16 12:57:56 -07:00
passthrough chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
runpod chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
sambanova feat(providers): sambanova updated to use LiteLLM openai-compat (#1596) 2025-05-06 16:50:22 -07:00
sambanova_openai_compat feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
tgi chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
together fix: revert "feat(provider): adding llama4 support in together inference provider (#2123)" (#2124) 2025-05-08 15:18:16 -07:00
together_openai_compat feat: introduce APIs for retrieving chat completion requests (#2145) 2025-05-18 21:43:19 -07:00
vllm fix: multiple tool calls in remote-vllm chat_completion (#2161) 2025-05-15 11:23:29 -07:00
watsonx chore: enable pyupgrade fixes (#1806) 2025-05-01 14:23:50 -07:00
__init__.py impls -> inline, adapters -> remote (#381) 2024-11-06 14:54:05 -08:00