phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions
llama-stack/docs/source/distributions/self_hosted_distro/passthrough.md
Xi Yan 9617468d13
fix: passthrough provider template + fix (#1612) 
# What does this PR do?

- Fix issue w/ passthrough provider


[//]: # (If resolving an issue, uncomment and update the line below)
[//]: # (Closes #[issue-number])

## Test Plan
llama stack run

[//]: # (## Documentation)
2025-03-13 09:44:26 -07:00

1.3 KiB
Raw Blame History

orphan
true

Passthrough Distribution

:maxdepth: 2
:hidden:

self

The llamastack/distribution-passthrough distribution consists of the following provider configurations.

API Provider(s)
agents inline::meta-reference
datasetio remote::huggingface, inline::localfs
eval inline::meta-reference
inference remote::passthrough, inline::sentence-transformers
safety inline::llama-guard
scoring inline::basic, inline::llm-as-judge, inline::braintrust
telemetry inline::meta-reference
tool_runtime remote::brave-search, remote::tavily-search, remote::wolfram-alpha, inline::code-interpreter, inline::rag-runtime, remote::model-context-protocol
vector_io inline::faiss, remote::chromadb, remote::pgvector

Environment Variables

The following environment variables can be configured:

  • LLAMA_STACK_PORT: Port for the Llama Stack distribution server (default: 5001)
  • PASSTHROUGH_API_KEY: Passthrough API Key (default: ``)
  • PASSTHROUGH_URL: Passthrough URL (default: ``)

Models

The following models are available by default:

  • llama3.1-8b-instruct
  • llama3.2-11b-vision-instruct