mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-07-20 03:40:05 +00:00
ade075152e
Inline _inference_ providers haven't proved to be very useful -- they
are rarely used. And for good reason -- it is almost never a good idea
to include a complex (distributed) inference engine bundled into a
distributed stateful front-end server serving many other things.
Responsibility should be split properly.
See Discord discussion:
1395849853
31 lines
No EOL
1.4 KiB
Markdown
31 lines
No EOL
1.4 KiB
Markdown
# Inference Providers
|
|
|
|
This section contains documentation for all available providers for the **inference** API.
|
|
|
|
- [inline::meta-reference](inline_meta-reference.md)
|
|
- [inline::sentence-transformers](inline_sentence-transformers.md)
|
|
- [remote::anthropic](remote_anthropic.md)
|
|
- [remote::bedrock](remote_bedrock.md)
|
|
- [remote::cerebras](remote_cerebras.md)
|
|
- [remote::cerebras-openai-compat](remote_cerebras-openai-compat.md)
|
|
- [remote::databricks](remote_databricks.md)
|
|
- [remote::fireworks](remote_fireworks.md)
|
|
- [remote::fireworks-openai-compat](remote_fireworks-openai-compat.md)
|
|
- [remote::gemini](remote_gemini.md)
|
|
- [remote::groq](remote_groq.md)
|
|
- [remote::groq-openai-compat](remote_groq-openai-compat.md)
|
|
- [remote::hf::endpoint](remote_hf_endpoint.md)
|
|
- [remote::hf::serverless](remote_hf_serverless.md)
|
|
- [remote::llama-openai-compat](remote_llama-openai-compat.md)
|
|
- [remote::nvidia](remote_nvidia.md)
|
|
- [remote::ollama](remote_ollama.md)
|
|
- [remote::openai](remote_openai.md)
|
|
- [remote::passthrough](remote_passthrough.md)
|
|
- [remote::runpod](remote_runpod.md)
|
|
- [remote::sambanova](remote_sambanova.md)
|
|
- [remote::sambanova-openai-compat](remote_sambanova-openai-compat.md)
|
|
- [remote::tgi](remote_tgi.md)
|
|
- [remote::together](remote_together.md)
|
|
- [remote::together-openai-compat](remote_together-openai-compat.md)
|
|
- [remote::vllm](remote_vllm.md)
|
|
- [remote::watsonx](remote_watsonx.md) |