mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-07-29 07:14:20 +00:00
19c90d9bfc
Some checks failed
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 2s
Vector IO Integration Tests / test-matrix (3.12, inline::sqlite-vec) (push) Failing after 4s
Vector IO Integration Tests / test-matrix (3.12, remote::pgvector) (push) Failing after 5s
Vector IO Integration Tests / test-matrix (3.13, remote::chromadb) (push) Failing after 6s
Test Llama Stack Build / build-ubi9-container-distribution (push) Failing after 4s
Integration Tests / discover-tests (push) Successful in 10s
Test Llama Stack Build / generate-matrix (push) Successful in 7s
Coverage Badge / unit-tests (push) Failing after 13s
Vector IO Integration Tests / test-matrix (3.13, inline::faiss) (push) Failing after 12s
Unit Tests / unit-tests (3.12) (push) Failing after 7s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Failing after 9s
Test Llama Stack Build / build-custom-container-distribution (push) Failing after 10s
Vector IO Integration Tests / test-matrix (3.12, inline::milvus) (push) Failing after 15s
Vector IO Integration Tests / test-matrix (3.13, inline::sqlite-vec) (push) Failing after 15s
Integration Tests / test-matrix (push) Failing after 6s
Test Llama Stack Build / build (push) Failing after 7s
Python Package Build Test / build (3.12) (push) Failing after 15s
Test Llama Stack Build / build-single-provider (push) Failing after 15s
Vector IO Integration Tests / test-matrix (3.12, inline::faiss) (push) Failing after 21s
Vector IO Integration Tests / test-matrix (3.13, inline::milvus) (push) Failing after 19s
Vector IO Integration Tests / test-matrix (3.13, remote::pgvector) (push) Failing after 19s
Vector IO Integration Tests / test-matrix (3.12, remote::chromadb) (push) Failing after 21s
Test External API and Providers / test-external (venv) (push) Failing after 16s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 24s
Unit Tests / unit-tests (3.13) (push) Failing after 16s
Python Package Build Test / build (3.13) (push) Failing after 42s
Update ReadTheDocs / update-readthedocs (push) Failing after 40s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 51s
Pre-commit / pre-commit (push) Successful in 1m58s
# What does this PR do? <!-- Provide a short summary of what this PR does and why. Link to relevant issues if applicable. --> Updates provider template from outdated `ollama` to `starter` <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> Closes: #2839 ## Test Plan <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* -->
1.2 KiB
1.2 KiB
Using Llama Stack as a Library
Setup Llama Stack without a Server
If you are planning to use an external service for Inference (even Ollama or TGI counts as external), it is often easier to use Llama Stack as a library. This avoids the overhead of setting up a server.
# setup
uv pip install llama-stack
llama stack build --template starter --image-type venv
from llama_stack.distribution.library_client import LlamaStackAsLibraryClient
client = LlamaStackAsLibraryClient(
"starter",
# provider_data is optional, but if you need to pass in any provider specific data, you can do so here.
provider_data={"tavily_search_api_key": os.environ["TAVILY_SEARCH_API_KEY"]},
)
client.initialize()
This will parse your config and set up any inline implementations and remote clients needed for your implementation.
Then, you can access the APIs like models and inference on the client and call their methods directly:
response = client.models.list()
If you've created a custom distribution, you can also use the run.yaml configuration file directly:
client = LlamaStackAsLibraryClient(config_path)
client.initialize()