2076 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Ben Browning	46da232e83	Make Files API an optional dependency of faiss provider ... Because we haven't updated every template to include a Files API provider yet, this adjusts the Files API to be an optional dependency instead of a required one of the Faiss provider. Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:55 -04:00
Ben Browning	8a5ea57253	Responses file_search wire up additional params ... This adds passing of max_num_results from the file_search tool call into the knowledge_search tool, as well as logs warnings if the filters or ranking_options params are used since those are not wired up yet. And, it adds the API surface for filters and ranking options so we don't have to generate clients again as we add that. Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:55 -04:00
Ben Browning	788d34d8b4	Move vector file attach code to OpenAIVectorStoreMixin ... This moves the vector store file attach code to the OpenAIVectorStoreMixin. It also centralizes the mime type and pdf parsing logic into the existing functions in vector_store.py by making a small refactor there to allow us to use the same code path. Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:55 -04:00
Ben Browning	055885bd5a	Add pdf support to file_search for Responses API ... This adds basic PDF support (using our existing `parse_pdf` function) to the file_search tool and corresponding Vector Files API. When a PDF file is uploaded and attached to a vector store, we parse the pdf and then chunk its content as normal. This is not the best solution long-term, but it does match what we've been doing so far for PDF files in the memory tool. Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:55 -04:00
Ben Browning	57eccf023d	Test responses queries of empty vector stores ... Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:55 -04:00
Ben Browning	de84ee0748	Implement attaching files to vector stores ... This adds the ability to attach files to vector stores (client.vector_stores.files.create) for the OpenAI Vector Stores Files API. The initial implementation is only for Faiss, and tested via the existing test_responses.py::test_response_non_streaming_file_search. Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:54 -04:00
Ben Browning	8ede67b809	More work on file_search verification test ... This gets the file_search verification test working against ollama, fireworks, and api.openai.com. We don't have the entirety of the vector store API implemented in Llama Stack yet, so this still has a bit of a hack to swap between using only OpenAI-compatible APIs versus using the LlamaStackClient to insert content into our vector stores. Outside of actually inserting file contents, the rest of the test works the same and uses only the OpenAI client for all of these providers. How to run the tests: Ollama (sometimes flakes with small model): ``` ollama run llama3.2:3b INFERENCE_MODEL="meta-llama/Llama-3.2-3B-Instruct" \ llama stack run ./llama_stack/templates/ollama/run.yaml \ --image-type venv \ --env OLLAMA_URL="http://0.0.0.0:11434" pytest -sv \ 'tests/verifications/openai_api/test_responses.py::test_response_non_streaming_file_search' \ --base-url=http://localhost:8321/v1/openai/v1 \ --model meta-llama/Llama-3.2-3B-Instruct ``` Fireworks via Llama Stack: ``` llama stack run llama_stack/templates/fireworks/run.yaml pytest -sv \ 'tests/verifications/openai_api/test_responses.py::test_response_non_streaming_file_search' \ --base-url=http://localhost:8321/v1/openai/v1 \ --model meta-llama/Llama-3.3-70B-Instruct ``` OpenAI directly: ``` pytest -sv \ 'tests/verifications/openai_api/test_responses.py::test_response_non_streaming_file_search' \ --base-url=https://api.openai.com/v1 \ --model gpt-4o ``` Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:04 -04:00
Ben Browning	fa34468308	feat: File search tool for Responses API ... This is an initial working prototype of wiring up the `file_search` builtin tool for the Responses API to our existing rag knowledge search tool. I stubbed in a new test (that uses a hardcoded url hybrid of the OpenAI and Llama Stack clients for now, only until we finish landing the vector store APIs and insertion support). Note that this is currently under tests/verification only because it sometimes flakes with tool calling of the small Llama-3.2-3B model we run in CI (and that I use as an example below). We'd want to make the test a bit more robust in some way if we moved this over to tests/integration and ran it in CI. ``` ollama run llama3.2:3b INFERENCE_MODEL="meta-llama/Llama-3.2-3B-Instruct" \ llama stack run ./llama_stack/templates/ollama/run.yaml \ --image-type venv \ --env OLLAMA_URL="http://0.0.0.0:11434" pytest -sv 'tests/verifications/openai_api/test_responses.py::test_response_non_streaming_file_search' \ --base-url=http://localhost:8321/v1/openai/v1 \ --model meta-llama/Llama-3.2-3B-Instruct ``` Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-13 09:36:04 -04:00
grs	e2e15ebb6c	feat(auth): allow token to be provided for use against jwks endpoint (#2394) ... Though the jwks endpoint does not usually require authentication, it does in a kubernetes cluster. While the cluster can be configured to allow anonymous access to that endpoint, this avoids the need to do so.	2025-06-13 10:13:41 +02:00
Hardik Shah	ddaee42650	test: Update integration-tests.yml (#2443) ... Added `vector_io` to the CI integration tests.	2025-06-13 10:04:08 +02:00
Hardik Shah	fef670b024	feat: update openai tests to work with both clients (#2442) ... https://github.com/meta-llama/llama-stack-client-python/pull/238 updated llama-stack-client to also support Open AI endpoints for embeddings, files, vector-stores. This updates the test to test all configs -- openai sdk, llama stack sdk and library-as-client.	2025-06-12 16:30:23 -07:00
Hardik Shah	0bc1747ed8	feat: update search for vector_stores (#2441) ... Updated the `search` functionality return response to match openai. ## Test Plan ``` pytest -sv --stack-config=http://localhost:8321 tests/integration/vector_io/test_openai_vector_stores.py --embedding-model all-MiniLM-L6-v2 ```	2025-06-12 15:34:22 -07:00
Ibrahim Haroon	35c2817d0a	fix(weaviate): handle case where distance is 0 by setting score to infinity (#2415) ... # What does this PR do? Fixes provider weaviate `query_vector` function for when the distance between the query embedding and an embedding within the vector db is 0 (identical vectors). Catches `ZeroDivisionError` and then sets `score` to infinity, which represent maximum similarity. <!-- If resolving an issue, uncomment and update the line below --> Closes [#2381] ## Test Plan Checkout this PR Execute this code and there will no longer be a `ZeroDivisionError` exception ``` from llama_stack_client import LlamaStackClient base_url = "http://localhost:8321" client = LlamaStackClient(base_url=base_url) models = client.models.list() embedding_model = ( em := next(m for m in models if m.model_type == "embedding") ).identifier embedding_dimension = 384 _ = client.vector_dbs.register( vector_db_id="foo_db", embedding_model=embedding_model, embedding_dimension=embedding_dimension, provider_id="weaviate", ) chunk = { "content": "foo", "mime_type": "text/plain", "metadata": { "document_id": "foo-id" } } client.vector_io.insert(vector_db_id="foo_db", chunks=[chunk]) client.vector_io.query(vector_db_id="foo_db", query="foo") ```	2025-06-12 11:23:59 -04:00
Sébastien Han	eb04731750	ci: fix external provider test (#2438) ... # What does this PR do? The test wasn't using the correct virtual environment. Also augment the console width for logs. Signed-off-by: Sébastien Han <seb@redhat.com>	2025-06-12 16:14:32 +02:00
Hardik Shah	de37a04c3e	fix: set appropriate defaults for params (#2434) ... Setting defaults to be `| None` else they get marked as required params in open-api spec.	2025-06-11 17:30:34 -07:00
Hardik Shah	d55100d9b7	feat: OpenAIVectorIOMixin for vector_stores common logic (#2427) ... Extracts common OpenAI vector-store code into its own mixin so that all providers can share the same core logic. This also makes it easy for Llama Stack to support both vector-stores and Llama Stack APIs in the interim so that both share the same underlying vector-dbs. Each provider contains storage specific logic to `create / edit / delete / list` vector dbs while the plumbing logic is standardized in the common code. Ensured that this works well with both faiss and sqllite-vec. ### Test Plan ``` llama stack run starter pytest -sv --stack-config http://localhost:8321 tests/integration/vector_io/test_openai_vector_stores.py --embedding-model all-MiniLM-L6-v2 ```	2025-06-11 15:40:57 -07:00
Rohan Awhad	4e37b49cdc	fix: #1867 InferenceRouter has no attribute formatter (#2422) ... # What does this PR do? Closes #1867 [Steps to reproduce the bug](https://github.com/meta-llama/llama-stack/issues/1867#issuecomment-2956819381) The change was designed to minimize code changes. Open to option of skipping `metrics` field entirely when `telemetry` is disabled. ## Test Plan 1. Build llama-stack remote-vllm container ```bash llama stack build --template remote-vllm --image-type container ``` 2. Create a small run.yaml ```yaml version: '2' image_name: remote-vllm apis: - inference providers: inference: - provider_id: vllm-inference provider_type: remote::vllm config: url: ${env.VLLM_URL:http://localhost:8000/v1} max_tokens: ${env.VLLM_MAX_TOKENS:4096} api_token: ${env.VLLM_API_TOKEN:fake} tls_verify: ${env.VLLM_TLS_VERIFY:true} metadata_store: type: sqlite db_path: ${env.SQLITE_STORE_DIR:~/.llama/distributions/remote-vllm}/registry.db inference_store: type: sqlite db_path: ${env.SQLITE_STORE_DIR:~/.llama/distributions/remote-vllm}/inference_store.db models: - metadata: {} model_id: ${env.INFERENCE_MODEL} provider_id: vllm-inference model_type: llm shields: [] vector_dbs: [] datasets: [] scoring_fns: [] benchmarks: [] server: port: 8321 ``` 3. Run the llama-stack server ```bash export VLLM_URL="http://localhost:8000/v1" export INFERENCE_MODEL="meta-llama/Llama-3.2-3B-Instruct" llama stack run run.yaml ``` 4. Then perform a curl ```bash curl -X 'POST' \ 'http://localhost:8321/v1/inference/completion' \ -H 'accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "model_id": "meta-llama/Llama-3.2-3B-Instruct", "content": "string", "sampling_params": { "strategy": { "type": "greedy" }, "max_tokens": 10, "repetition_penalty": 1, "stop": [ "string" ] }, "stream": false, "logprobs": { "top_k": 0 } }' ``` 5. You should receive a 200 response with metric values set to 0, similar to one below: ``` { "metrics": [ { "metric": "prompt_tokens", "value": 0, "unit": null }, { "metric": "completion_tokens", "value": 0, "unit": null }, { "metric": "total_tokens", "value": 0, "unit": null } ], [...] } ``` Co-authored-by: Rohan Awhad <rawhad@redhat.com>	2025-06-11 18:14:41 +02:00
Hardik Shah	5ac43268e8	feat: Add OpenAI compat /v1/vector_store APIs (#2423) ... Adding OpenAI compat `/v1/vector-store` apis. This PR implements the `faiss` provider with followup PRs coming up for other providers. Added routes to create, update, delete, list vector stores. Also added route to search a vector store Inserting into vector stores is missing and will be a follow up diff. ### Test Plan - Added new integration test for testing the faiss provider ``` pytest -sv --stack-config http://localhost:8321 tests/integration/vector_io/test_openai_vector_stores.py --embedding-model all-MiniLM-L6-v2 ```	2025-06-10 13:07:39 -07:00
Ben Browning	ee57e58f29	fix: loosen tool call checks in inference store (#2420) ... # What does this PR do? This loosens up the tool call function name and arguments checks in `tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls` because the small models we use in CI cannot reliably get the tool call function name or arguments exactly right. Closes #2345 ## Test Plan I ran this flaking test in a loop, let it run many dozens of times, and didn't observe any flakes after the changes. Previously it flaked quite regularly. ``` while uv run pytest -s -v \ 'tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[llama_stack_client-txt=3B-False]' \ --stack-config=http://localhost:8321 \ --text-model="meta-llama/Llama-3.2-3B-Instruct" \ --embedding-model=all-MiniLM-L6-v2; do; sleep 0.1; done ``` Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-10 14:45:55 +02:00
Yuan Tang	5639ad7466	docs: Add recent releases (#2424) ... # What does this PR do? This adds recent release notes. Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>	2025-06-10 08:43:02 +05:30
Yuan Tang	f6718b2408	fix(security): Upgrade requests to 2.32.4. Fixes CVE-2024-47081 (#2425) ... # What does this PR do? This address https://github.com/advisories/GHSA-9hjg-9r4m-mvj7. Diff was generated via: ``` uv sync --upgrade-package requests uv export --frozen --no-hashes --no-emit-project --no-default-groups --output-file=requirements.txt ``` Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>	2025-06-10 08:33:28 +05:30
Ibrahim Haroon	28ca00d0d9	fix(pgvector): handle case where distance is 0 by setting score to infinity (#2416) ... # What does this PR do? Fixes provider pgvector `query_vector` function for when the distance between the query embedding and an embedding within the vector db is 0 (identical vectors). Catches `ZeroDivisionError` and then sets `score` to infinity, which represent maximum similarity. <!-- If resolving an issue, uncomment and update the line below --> Closes [#2381] ## Test Plan Checkout this PR Execute this code and there will no longer be a `ZeroDivisionError` exception ``` from llama_stack_client import LlamaStackClient base_url = "http://localhost:8321" client = LlamaStackClient(base_url=base_url) models = client.models.list() embedding_model = ( em := next(m for m in models if m.model_type == "embedding") ).identifier embedding_dimension = 384 _ = client.vector_dbs.register( vector_db_id="foo_db", embedding_model=embedding_model, embedding_dimension=embedding_dimension, provider_id="pgvector", ) chunk = { "content": "foo", "mime_type": "text/plain", "metadata": { "document_id": "foo-id" } } client.vector_io.insert(vector_db_id="foo_db", chunks=[chunk]) client.vector_io.query(vector_db_id="foo_db", query="foo") ```	2025-06-07 16:31:30 -04:00
Ibrahim Haroon	a34cef925b	fix(faiss): handle case where distance is 0 by setting d to minimum positive… (#2387) ... # What does this PR do? Adds try-catch to faiss `query_vector` function for when the distance between the query embedding and an embedding within the vector db is 0 (identical vectors). Catches `ZeroDivisionError` and then appends `(1.0 / sys.float_info.min)` to `scores` to represent maximum similarity. <!-- If resolving an issue, uncomment and update the line below --> Closes [#2381] ## Test Plan Checkout this PR Execute this code and there will no longer be a `ZeroDivisionError` exception ``` from llama_stack_client import LlamaStackClient base_url = "http://localhost:8321" client = LlamaStackClient(base_url=base_url) models = client.models.list() embedding_model = ( em := next(m for m in models if m.model_type == "embedding") ).identifier embedding_dimension = 384 _ = client.vector_dbs.register( vector_db_id="foo_db", embedding_model=embedding_model, embedding_dimension=embedding_dimension, provider_id="faiss", ) chunk = { "content": "foo", "mime_type": "text/plain", "metadata": { "document_id": "foo-id" } } client.vector_io.insert(vector_db_id="foo_db", chunks=[chunk]) client.vector_io.query(vector_db_id="foo_db", query="foo") ``` ### Running unit tests `uv run pytest tests/unit/rag/test_rag_query.py -v` --------- Signed-off-by: Ben Browning <bbrownin@redhat.com> Co-authored-by: Ben Browning <bbrownin@redhat.com>	2025-06-07 16:09:46 -04:00
Sumit Jaiswal	33ecefd284	feat: To add health status check for remote VLLM (#2303) ... # What does this PR do? <!-- Provide a short summary of what this PR does and why. Link to relevant issues if applicable. --> To add health status check for remote VLLM <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> ## Test Plan <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* --> PR includes the unit test to test the added health check implementation feature.	2025-06-06 15:33:12 -04:00
Alexey Rybak	32c651e3a7	chore: update CODEOWNERS (#2414)	2025-06-06 20:35:15 +02:00
Hardik Shah	1f48577a02	fix: ChromaDB provider (#2413) ... fixes the remote::chromaDB provider for vector_io by updating the method definition appropriately. Fixed impl to use score_threshold properly. ### Test Plan ``` # Start Chroma Docker docker run --rm \ --name chromadb \ -p 8800:8000 \ -v ~/chroma:/chroma/chroma \ -e IS_PERSISTENT=TRUE \ -e ANONYMIZED_TELEMETRY=FALSE \ chromadb/chroma:latest # run pytest CHROMADB_URL="http://localhost:8800" pytest -sv tests/integration/vector_io/test_vector_io.py --stack-config vector_io=remote::chromadb,inference=fireworks --embedding-model nomic-ai/nomic-embed-text-v1.5 ```	2025-06-06 11:25:58 -07:00
Sébastien Han	0d0b8d2be1	ci: use ollama container image with loaded models (#2410) ... # What does this PR do? Instead of downloading the models each time we now have a single Ollama container that is baked with the models pulled and ready to use. This will remove the CI flakiness on model pulling. Signed-off-by: Sébastien Han <seb@redhat.com>	2025-06-06 12:08:20 +02:00
github-actions[bot]	692709cd45	build: Bump version to 0.2.10	2025-06-05 22:56:39 +00:00
Hardik Shah	102516f33c	fix: Pin fastapi to avoid picking up spurious versions in test pypi (#2409) ... as titled	2025-06-05 15:33:30 -07:00
ehhuang	446893f791	feat: add deps dynamically based on metastore config (#2405) ... # What does this PR do? ## Test Plan changed metastore in one of the templates, rerun distro gen, observe change in build.yaml	2025-06-05 14:07:25 -07:00
ehhuang	92b59a3377	test: skip files integrations tests for library client (#2407) ... # What does this PR do? ## Test Plan LLAMA_STACK_CONFIG=fireworks pytest -s -v tests/integration/files/test_files.py::test_openai_client_basic_operations	2025-06-05 13:42:10 -07:00
ehhuang	ee6feaa2d5	chore: remove dead code (#2403) ... # What does this PR do? ## Test Plan	2025-06-05 21:17:54 +02:00
Hardik Shah	04592b9590	fix: update pyproject to include recursive LS deps (#2404) ... trying to run `llama` cli after installing wheel fails with this error ``` Traceback (most recent call last): File "/tmp/tmp.wdZath9U6j/.venv/bin/llama", line 4, in <module> from llama_stack.cli.llama import main File "/tmp/tmp.wdZath9U6j/.venv/lib/python3.10/site-packages/llama_stack/__init__.py", line 7, in <module> from llama_stack.distribution.library_client import ( # noqa: F401 ModuleNotFoundError: No module named 'llama_stack.distribution.library_client' ``` This PR fixes it by ensurring that all sub-directories of `llama_stack` are also included. Also, fixes the missing `fastapi` dependency issue.	2025-06-05 11:46:48 -07:00
Sébastien Han	4fb228a1d8	ci: run integration test on more python version (#2400) ... # What does this PR do? Expand the test matrix to include Python 3.10, 3.11, and 3.12 to ensure the project runs correctly on these versions. This will give us confidence to begin considering an increase to the project's minimum supported Python version. Signed-off-by: Sébastien Han <seb@redhat.com>	2025-06-05 20:40:21 +02:00
Ashwin Bharambe	3251b44d8a	refactor: unify stream and non-stream impls for responses (#2388) ... The non-streaming version is just a small layer on top of the streaming version - just pluck off the final `response.completed` event and return that as the response! This PR also includes a couple other changes which I ended up making while working on it on a flight: - changes to `ollama` so it does not pull embedding models unconditionally - a small fix to library client to make the stream and non-stream cases a bit more symmetric	2025-06-05 17:48:09 +02:00
Jose Angel Morena Simon	ef885d2147	fix(server): Add missing OpenTelemetry dependencies to resolve telemetry import errors (#2391) ... This PR fixes a runtime import error caused by missing OpenTelemetry dependencies during `llama stack run`. Specifically, the following imports fail if `opentelemetry-sdk` and `opentelemetry-exporter-otlp-proto-http` are not present in the environment: ```python from opentelemetry import metrics, trace from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter ``` See [llama\_stack/providers/inline/telemetry/meta\_reference/telemetry.py#L10-L19](https://github.com/meta-llama/llama-stack/blob/main/llama_stack/providers/inline/telemetry/meta_reference/telemetry.py#L10-L19) This PR resolves the issue by adding both packages to the `SERVER_DEPENDENCIES` list: ```python "opentelemetry-sdk", "opentelemetry-exporter-otlp-proto-http", ``` ### Reproduction Steps ```bash llama stack build --config llama.yaml --image-type venv --image-name fun-with-lamas llama stack run ~/.llama/distributions/fun-with-lamas/fun-with-lamas-run.yaml ``` Results in: ``` ModuleNotFoundError: No module named 'opentelemetry' ``` or ``` ModuleNotFoundError: No module named 'opentelemetry.exporter' ``` Signed-off-by: Jose Angel Morena <jmorenas@redhat.com> Co-authored-by: raghotham <rsm@meta.com>	2025-06-05 09:34:46 +02:00
Nathan Weinberg	179d72615b	docs: update contributing guidance around uv python versions (#2398) ... As discussed with @leseb here: https://github.com/containers/ramalama-stack/pull/81#discussion_r2125961014 Signed-off-by: Nathan Weinberg <nweinber@redhat.com>	2025-06-04 23:12:03 -07:00
ehhuang	a58c0639d5	chore: update postgres_demo distro config (#2396) ... # What does this PR do? ## Test Plan	2025-06-04 17:41:27 -07:00
Sébastien Han	c8c742ba45	fix: vllm starter name (#2392) ... Signed-off-by: Sébastien Han <seb@redhat.com>	2025-06-04 16:21:36 +02:00
grs	0de9536717	fix: remove debug print accidentally merged (#2393) ... I accidentally left a debug print in a PR that was merged. This removes that.	2025-06-04 15:14:14 +02:00
Ben Browning	e9d9f01b8b	docs: Add OpenAI API compatibility page (#2316) ... # What does this PR do? This adds some initial content documenting our OpenAI compatible APIs - Responses, Chat Completions, Completions, and Models - along with instructions on how to use them via OpenAI or Llama Stack clients and some simple examples for each. It's not a lot of content, but it's a start so that users have some idea how to get going as we continue to work on these APIs. ## Test Plan I generated the docs site locally and verified things render properly. I also ran each code example to ensure it works as expected. And, I asked my AI code assistant to do a quick spell-check and review of the docs and it didn't flag any obvious errors. --------- Signed-off-by: Ben Browning <bbrownin@redhat.com> Co-authored-by: Francisco Arceo <farceo@redhat.com>	2025-06-04 06:51:52 -04:00
Ashwin Bharambe	ed69c1b3cc	feat(responses): add more streaming response types (#2375)	2025-06-03 15:48:41 -07:00
ehhuang	d96f6ec763	chore(ui): use proxy server for backend API calls; simplified k8s deployment (#2350) ... # What does this PR do? - no more CORS middleware needed ## Test Plan ### Local test llama stack run starter --image-type conda npm run dev verify UI works in browser ### Deploy to k8s temporarily change ui-k8s.yaml.template to load from PR commit <img width="604" alt="image" src="https://github.com/user-attachments/assets/87fa2e52-1e93-4e32-9e0f-5b283b7a37b3" /> sh ./apply.sh $ kubectl get services go to external_ip:8322 and play around with UI <img width="1690" alt="image" src="https://github.com/user-attachments/assets/5b7ec827-4302-4435-a9eb-df423676d873" />	2025-06-03 14:57:10 -07:00
grs	7c1998db25	feat: fine grained access control policy (#2264) ... This allows a set of rules to be defined for determining access to resources. The rules are (loosely) based on the cedar policy format. A rule defines a list of action either to permit or to forbid. It may specify a principal or a resource that must match for the rule to take effect. It may also specify a condition, either a 'when' or an 'unless', with additional constraints as to where the rule applies. A list of rules is held for each type to be protected and tried in order to find a match. If a match is found, the request is permitted or forbidden depening on the type of rule. If no match is found, the request is denied. If no rules are specified for a given type, a rule that allows any action as long as the resource attributes match the user attributes is added (i.e. the previous behaviour is the default. Some examples in yaml: ``` model: - permit: principal: user-1 actions: [create, read, delete] comment: user-1 has full access to all models - permit: principal: user-2 actions: [read] resource: model-1 comment: user-2 has read access to model-1 only - permit: actions: [read] when: user_in: resource.namespaces comment: any user has read access to models with matching attributes vector_db: - forbid: actions: [create, read, delete] unless: user_in: role::admin comment: only user with admin role can use vector_db resources ``` --------- Signed-off-by: Gordon Sim <gsim@redhat.com>	2025-06-03 14:51:12 -07:00
Ben Browning	8bee2954be	feat: Structured output for Responses API (#2324) ... # What does this PR do? This adds the missing `text` parameter to the Responses API that is how users control structured outputs. All we do with that parameter is map it to the corresponding chat completion response_format. ## Test Plan The new unit tests exercise the various permutations allowed for this property, while a couple of new verification tests actually use it for real to verify the model outputs are following the format as expected. Unit tests: `python -m pytest -s -v tests/unit/providers/agents/meta_reference/test_openai_responses.py` Verification tests: ``` llama stack run llama_stack/templates/together/run.yaml pytest -s -vv 'tests/verifications/openai_api/test_responses.py' \ --base-url=http://localhost:8321/v1/openai/v1 \ --model meta-llama/Llama-4-Scout-17B-16E-Instruct ``` Note that the verification tests can only be run with a real Llama Stack server (as opposed to using the library client via `--provider=stack:together`) because the Llama Stack python client is not yet updated to accept this text field. Signed-off-by: Ben Browning <bbrownin@redhat.com>	2025-06-03 14:43:00 -07:00
Ignas Baranauskas	c70ca8344f	fix: resolve template name to config path in llama stack run (#2361) ... # What does this PR do? <!-- Provide a short summary of what this PR does and why. Link to relevant issues if applicable. --> This PR fixes a bug where running a known template by name using: `llama stack run ollama` would fail with the following error: `ValueError: Config file ollama does not exist` <!-- If resolving an issue, uncomment and update the line below --> Closes #2291 ## Test Plan <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* --> `llama stack run ollama` should work	2025-06-03 14:39:12 -07:00
Ashwin Bharambe	cba55808ab	feat(distro): add more providers to starter distro, prefix conflicting models (#2362) ... The name changes to the verifications file are unfortunate, but maybe we don't need that @ehhuang ? Edit: deleted the verifications template now	2025-06-03 12:10:46 -07:00
Ashwin Bharambe	b380cb463f	feat: add postgres deps to starter distro (#2360) ... Once we have this, we can use the starter distro for the Kubernetes cluster demos.	2025-06-03 11:04:23 -07:00
Jorge	e743257d1d	docs: Add missing dependencies in quickstart demo command (#2347) ... Adds missing required dependencies to run the demo command in the Quickstart doc Signed-off-by: Jorge Garcia Oncins <jgarciao@redhat.com>	2025-06-03 18:01:36 +02:00
ehhuang	3c9a10d2fe	feat: reference implementation for files API (#2330) ... # What does this PR do? TSIA Added Files provider to the fireworks template. Might want to add to all templates as a follow-up. ## Test Plan llama-stack pytest tests/unit/files/test_files.py llama-stack llama stack build --template fireworks --image-type conda --run LLAMA_STACK_CONFIG=http://localhost:8321 pytest -s -v tests/integration/files/	2025-06-02 21:54:24 -07:00