llama-stack/docs/source/getting_started/distributions/self_hosted_distro/index.md
2024-11-07 19:16:38 -08:00

3.4 KiB

Self-Hosted Distribution

We offer deployable distributions where you can host your own Llama Stack server using local inference.

Distribution Llama Stack Docker Start This Distribution Inference Agents Memory Safety Telemetry
Meta Reference llamastack/distribution-meta-reference-gpu Guide meta-reference meta-reference meta-reference; remote::pgvector; remote::chromadb meta-reference meta-reference
Meta Reference Quantized llamastack/distribution-meta-reference-quantized-gpu Guide meta-reference-quantized meta-reference meta-reference; remote::pgvector; remote::chromadb meta-reference meta-reference
Ollama llamastack/distribution-ollama Guide remote::ollama meta-reference remote::pgvector; remote::chromadb meta-reference meta-reference
TGI llamastack/distribution-tgi Guide remote::tgi meta-reference meta-reference; remote::pgvector; remote::chromadb meta-reference meta-reference
Together llamastack/distribution-together Guide remote::together meta-reference remote::weaviate meta-reference meta-reference
Fireworks llamastack/distribution-fireworks Guide remote::fireworks meta-reference remote::weaviate meta-reference meta-reference
Bedrock llamastack/distribution-bedrock Guide remote::bedrock meta-reference remote::weaviate meta-reference meta-reference
:maxdepth: 1

meta-reference-gpu
meta-reference-quantized-gpu
ollama
tgi
dell-tgi
together
fireworks
bedrock