phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-26 12:08:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Docs: update full list of providers with matched APIs

Browse source 
- add model_type in example
- change "Memory" to "VectorIO" as column name
- update providers table
- update images from dockerhub
- update index.md

Signed-off-by: Wen Zhou <wenzhou@redhat.com>
(cherry picked from commit bb6c96c11c204fb195d81fa15431728c409847fc)
This commit is contained in:
Wen Zhou 2025-06-16 19:08:48 +02:00
parent d165000bbc
commit c9b0cc6439
4 changed files with 131 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

45
docs/source/providers/index.md
View file

@ -1,9 +1,10 @@
# Providers Overview
The goal of Llama Stack is to build an ecosystem where users can easily swap out different implementations for the same API. Examples for these include:
- LLM inference providers (e.g., Ollama, Fireworks, Together, AWS Bedrock, Groq, Cerebras, SambaNova, vLLM, etc.),
- Vector databases (e.g., ChromaDB, Weaviate, Qdrant, Milvus, FAISS, PGVector, SQLite-Vec, etc.),
- Safety providers (e.g., Meta's Llama Guard, AWS Bedrock Guardrails, etc.)
- LLM inference providers (e.g., Meta Reference, Ollama, Fireworks, Together, AWS Bedrock, Groq, Cerebras, SambaNova, vLLM, OpenAI, Anthropic, Gemini, WatsonX, etc.),
- Vector databases (e.g., FAISS, SQLite-Vec, ChromaDB, Weaviate, Qdrant, Milvus, PGVector, etc.),
- Safety providers (e.g., Meta's Llama Guard, Prompt Guard, Code Scanner, AWS Bedrock Guardrails, etc.),
- Tool Runtime providers (e.g., RAG Runtime, Brave Search, etc.)
Providers come in two flavors:
- **Remote**: the provider runs as a separate service external to the Llama Stack codebase. Llama Stack contains a small amount of adapter code.
@ -11,6 +12,44 @@ Providers come in two flavors:
Importantly, Llama Stack always strives to provide at least one fully inline provider for each API so you can iterate on a fully featured environment locally.
## Available Providers
Here is a comprehensive list of all available API providers in Llama Stack:
| API Provider Builder | Environments | Agents | Inference | VectorIO | Safety | Telemetry | Post Training | Eval | DatasetIO | Tool Runtime |
|:----------------------:|:------------------:|:------:|:---------:|:--------:|:------:|:---------:|:-------------:|:----:|:---------:|:------------:|
| Meta Reference | Single Node | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| SambaNova | Hosted | | ✅ | | ✅ | | | | | |
| Cerebras | Hosted | | ✅ | | | | | | | |
| Fireworks | Hosted | ✅ | ✅ | ✅ | | | | | | |
| AWS Bedrock | Hosted | | ✅ | | ✅ | | | | | |
| Together | Hosted | ✅ | ✅ | | ✅ | | | | | |
| Groq | Hosted | | ✅ | | | | | | | |
| Ollama | Single Node | | ✅ | | | | | | | |
| TGI | Hosted/Single Node | | ✅ | | | | | | | |
| NVIDIA NIM | Hosted/Single Node | | ✅ | | | | | | | |
| Chroma | Single Node | | | ✅ | | | | | | |
| PG Vector | Single Node | | | ✅ | | | | | | |
| PyTorch ExecuTorch | On-device iOS | ✅ | ✅ | | | | | | | |
| vLLM | Single Node | | ✅ | | | | | | | |
| OpenAI | Hosted | | ✅ | | | | | | | |
| Anthropic | Hosted | | ✅ | | | | | | | |
| Gemini | Hosted | | ✅ | | | | | | | |
| WatsonX | Hosted | | ✅ | | | | | | | |
| HuggingFace | Single Node | | | | | | ✅ | | ✅ | |
| TorchTune | Single Node | | | | | | ✅ | | | |
| NVIDIA NEMO | Hosted | | ✅ | ✅ | | | ✅ | ✅ | ✅ | |
| FAISS | Single Node | | | ✅ | | | | | | |
| SQLite-Vec | Single Node | | | ✅ | | | | | | |
| Qdrant | Hosted/Single Node | | | ✅ | | | | | | |
| Weaviate | Hosted | | | ✅ | | | | | | |
| Milvus | Hosted/Single Node | | | ✅ | | | | | | |
| Prompt Guard | Single Node | | | | ✅ | | | | | |
| Llama Guard | Single Node | | | | ✅ | | | | | |
| Code Scanner | Single Node | | | | ✅ | | | | | |
| Brave Search | Hosted | | | | | | | | | ✅ |
| RAG Runtime | Single Node | | | | | | | | | ✅ |
## External Providers
Llama Stack supports external providers that live outside of the main codebase. This allows you to create and maintain your own providers independently.