Commit graph

3135 commits

Author SHA1 Message Date
ehhuang
61daef193e
Merge 6fbbb3e78b into sapling-pr-archive-ehhuang
Some checks failed
Installer CI / smoke-test-on-dev (push) Failing after 8s
Installer CI / lint (push) Failing after 9s
2025-10-16 11:33:32 -07:00
Eric Huang
6fbbb3e78b fix(telemetry): remove dependency on old telemetry config
# What does this PR do?
old telemetry config was removed in #3815

## Test Plan

❯ OTEL_SERVICE_NAME=aloha OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 uv run llama stack run starter
2025-10-16 11:33:24 -07:00
ehhuang
07ff15d917
chore: distrogen enables telemetry by default (#3828)
# What does this PR do?
leftover from #3815

## Test Plan
CI


---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with
[ReviewStack](https://reviewstack.dev/llamastack/llama-stack/pull/3828).
* #3830
* __->__ #3828
2025-10-16 11:29:51 -07:00
ehhuang
cdeb41f438
Merge 5a991b5634 into sapling-pr-archive-ehhuang 2025-10-16 11:29:11 -07:00
Eric Huang
5a991b5634 fix(telemetry): remove dependency on old telemetry config
# What does this PR do?
old telemetry config was removed in #3815

## Test Plan

❯ OTEL_SERVICE_NAME=aloha OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 uv run llama stack run starter
2025-10-16 11:29:06 -07:00
ehhuang
53ea3222ac
Merge 38976b5ac1 into sapling-pr-archive-ehhuang 2025-10-16 11:26:09 -07:00
Eric Huang
38976b5ac1 fix(telemetry): remove dependency on old telemetry config
# What does this PR do?
old telemetry config was removed in #3815

## Test Plan

❯ OTEL_SERVICE_NAME=aloha OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 uv run llama stack run starter
2025-10-16 11:26:01 -07:00
Eric Huang
c4662ac316 merge commit for archive created by Sapling 2025-10-16 11:21:19 -07:00
Eric Huang
3679612b5f chore: distrogen enables telemetry by default
# What does this PR do?


## Test Plan
Telemetry provider was added to all distributions in the latest commit
but the protocol mapping was missing, causing a KeyError when the stack
tried to validate provider compliance.
2025-10-16 11:21:13 -07:00
ehhuang
41c54b7e16
Merge b7c276ea6d into sapling-pr-archive-ehhuang 2025-10-16 10:56:14 -07:00
Eric Huang
b7c276ea6d chore: distrogen enables telemetry by default
# What does this PR do?


## Test Plan
Telemetry provider was added to all distributions in the latest commit
but the protocol mapping was missing, causing a KeyError when the stack
tried to validate provider compliance.
2025-10-16 10:56:07 -07:00
Eric Huang
70c96147ae merge commit for archive created by Sapling 2025-10-16 10:47:44 -07:00
Eric Huang
60e7d2ac60 chore: distrogen enables telemetry by default
# What does this PR do?


## Test Plan
2025-10-16 10:47:35 -07:00
Charlie Doern
f22aaef42f
chore!: remove telemetry API usage (#3815)
# What does this PR do?

remove telemetry as a providable API from the codebase. This includes
removing it from generated distributions but also the provider registry,
the router, etc

since `setup_logger` is tied pretty strictly to `Api.telemetry` being in
impls we still need an "instantiated provider" in our implementations.
However it should not be auto-routed or provided. So in
validate_and_prepare_providers (called from resolve_impls) I made it so
that if run_config.telemetry.enabled, we set up the meta-reference
"provider" internally to be used so that log_event will work when
called.

This is the neatest way I think we can remove telemetry from the
provider configs but also not need to rip apart the whole "telemetry is
a provider" logic just yet, but we can do it internally later without
disrupting users.

so telemetry is removed from the registry such that if a user puts
`telemetry:` as an API in their build/run config it will err out, but
can still be used by us internally as we go through this transition.


relates to #3806

Signed-off-by: Charlie Doern <cdoern@redhat.com>
2025-10-16 10:39:32 -07:00
slekkala1
8c5705d39e
fix: test id not being set in headers (#3827)
# What does this PR do?
When stack config is set to server in docker
STACK_CONFIG_ARG=--stack-config=http://localhost:8321, the env variable
was not getting correctly set and test id not set, causing
This is needed for test-and-cut to work 
E openai.BadRequestError: Error code: 400 - {'detail': 'Invalid value:
Test ID is required for file ID allocation'}



5286461406

## Test Plan
CI
2025-10-16 10:29:07 -07:00
Bill Murdock
c19eb9854d
docs: Document known limitations of Responses (#3776)
# What does this PR do?

Adds a subpage of the OpenAI compatibility page in the documentation.
This subpage documents known limitations of the Responses API.

<!-- If resolving an issue, uncomment and update the line below -->

Closes #3575

---------

Signed-off-by: Bill Murdock <bmurdock@redhat.com>
2025-10-16 10:26:23 -07:00
Ashwin Bharambe
185de61d8e
fix(openai_mixin): no yelling for model listing if API keys are not provided (#3826)
As indicated in the title. Our `starter` distribution enables all remote
providers _very intentionally_ because we believe it creates an easier,
more welcoming experience to new folks using the software. If we do
that, and then slam the logs with errors making them question their life
choices, it is not so good :)

Note that this fix is limited in scope. If you ever try to actually
instantiate the OpenAI client from a code path without an API key being
present, you deserve to fail hard.

## Test Plan

Run `llama stack run starter` with `OPENAI_API_KEY` set. No more wall of
text, just one message saying "listed 96 models".
2025-10-16 10:12:13 -07:00
Ashwin Bharambe
07fc8013eb
fix(tests): reduce some test noise (#3825)
a bunch of logger.info()s are good for server code to help debug in
production, but we don't want them killing our unit test output :)

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-10-16 09:52:16 -07:00
Sébastien Han
0c368492b7
chore: update agent call (#3824)
Some checks failed
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 0s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 0s
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Python Package Build Test / build (3.12) (push) Failing after 1s
Python Package Build Test / build (3.13) (push) Failing after 4s
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 6s
Unit Tests / unit-tests (3.13) (push) Failing after 6s
Unit Tests / unit-tests (3.12) (push) Failing after 7s
Test External API and Providers / test-external (venv) (push) Failing after 9s
Vector IO Integration Tests / test-matrix (push) Failing after 11s
API Conformance Tests / check-schema-compatibility (push) Successful in 17s
UI Tests / ui-tests (22) (push) Successful in 1m49s
Pre-commit / pre-commit (push) Successful in 2m51s
followup on https://github.com/llamastack/llama-stack/pull/3810

Signed-off-by: Sébastien Han <seb@redhat.com>
2025-10-16 16:04:43 +02:00
Derek Higgins
edb8afb219
chore: remove test_cases/openai/responses.json (#3823)
Its unused

Signed-off-by: Derek Higgins <derekh@redhat.com>
2025-10-16 06:59:29 -07:00
Ashwin Bharambe
f70aa99c97
fix(models)!: always prefix models with provider_id when registering (#3822)
**!!BREAKING CHANGE!!**

The lookup is also straightforward -- we always look for this identifier
and don't try to find a match for something without the provider_id
prefix.

Note that, this ideally means we need to update the `register_model()`
API also (we should kill "identifier" from there) but I am not doing
that as part of this PR.

## Test Plan

Existing unit tests
2025-10-16 06:47:39 -07:00
Eric Huang
9bcd2f5bdb merge commit for archive created by Sapling
Some checks failed
Installer CI / lint (push) Failing after 4s
Installer CI / smoke-test-on-dev (push) Failing after 5s
2025-10-15 22:15:49 -07:00
Eric Huang
0034c6189b chore: add telemetry setup to install.sh
# What does this PR do?


## Test Plan
2025-10-15 22:15:43 -07:00
Ashwin Bharambe
f205ab6f6c
fix(responses): fixes, re-record tests (#3820)
Some checks failed
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Python Package Build Test / build (3.12) (push) Failing after 2s
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 3s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 5s
Python Package Build Test / build (3.13) (push) Failing after 3s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 8s
Vector IO Integration Tests / test-matrix (push) Failing after 6s
Test External API and Providers / test-external (venv) (push) Failing after 4s
Unit Tests / unit-tests (3.12) (push) Failing after 6s
Unit Tests / unit-tests (3.13) (push) Failing after 5s
API Conformance Tests / check-schema-compatibility (push) Successful in 17s
UI Tests / ui-tests (22) (push) Successful in 55s
Pre-commit / pre-commit (push) Successful in 1m43s
Wanted to re-enable Responses CI but it seems to hang for some reason
due to some interactions with conversations_store or responses_store.

## Test Plan

```
# library client
./scripts/integration-tests.sh --stack-config ci-tests --suite responses

# server
./scripts/integration-tests.sh --stack-config server:ci-tests --suite responses
```
2025-10-15 16:37:42 -07:00
ehhuang
f8d418ad38
Merge 6e83f07d12 into sapling-pr-archive-ehhuang 2025-10-15 16:14:25 -07:00
Eric Huang
6e83f07d12 chore: add telemetry setup to install.sh
# What does this PR do?


## Test Plan
2025-10-15 16:14:13 -07:00
slekkala1
99141c29b1
feat: Add responses and safety impl extra_body (#3781)
Some checks failed
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 0s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 0s
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 2s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Python Package Build Test / build (3.13) (push) Failing after 1s
Test Llama Stack Build / generate-matrix (push) Successful in 3s
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 6s
Test Llama Stack Build / build-custom-container-distribution (push) Failing after 3s
Test Llama Stack Build / build-single-provider (push) Failing after 4s
Python Package Build Test / build (3.12) (push) Failing after 6s
Vector IO Integration Tests / test-matrix (push) Failing after 9s
Unit Tests / unit-tests (3.13) (push) Failing after 6s
Test Llama Stack Build / build-ubi9-container-distribution (push) Failing after 9s
Test External API and Providers / test-external (venv) (push) Failing after 8s
Test Llama Stack Build / build (push) Failing after 7s
Unit Tests / unit-tests (3.12) (push) Failing after 9s
API Conformance Tests / check-schema-compatibility (push) Successful in 19s
UI Tests / ui-tests (22) (push) Successful in 37s
Pre-commit / pre-commit (push) Successful in 1m33s
# What does this PR do?

Have closed the previous PR due to merge conflicts with multiple PRs
Addressed all comments from
https://github.com/llamastack/llama-stack/pull/3768 (sorry for carrying
over to this one)


## Test Plan
Added UTs and integration tests
2025-10-15 15:01:37 -07:00
Ashwin Bharambe
8e7e0ddfec
fix(responses): use conversation items when no stored messages exist (#3819)
Handle a base case when no stored messages exist because no Response
call has been made.

## Test Plan

```
./scripts/integration-tests.sh --stack-config server:ci-tests \
   --suite responses   --inference-mode record-if-missing --pattern test_conversation_responses
```
2025-10-15 14:43:44 -07:00
ehhuang
6ba9db3929
chore!: BREAKING CHANGE: remove sqlite from telemetry config (#3808)
# What does this PR do?
- Removed sqlite sink from telemetry config.
- Removed related code
- Updated doc related to telemetry

## Test Plan
CI
2025-10-15 14:24:45 -07:00
ehhuang
460097bd7b
Merge 33d27393f4 into sapling-pr-archive-ehhuang 2025-10-15 14:19:34 -07:00
Eric Huang
33d27393f4 chore!: BREAKING CHANGE: remove sqlite from telemetry config
# What does this PR do?


## Test Plan
2025-10-15 14:19:27 -07:00
Ashwin Bharambe
0a96a7faa5
fix(responses): fix subtle bugs in non-function tool calling (#3817)
We were generating "FunctionToolCall" items even for MCP (and
file-search, etc.) server-side calls. ID mismatches, etc. galore.
2025-10-15 13:57:37 -07:00
ehhuang
d709eeb33f
chore: mark recordings as generated files (#3816)
# What does this PR do?


## Test Plan
<img width="1506" height="653" alt="image"
src="https://github.com/user-attachments/assets/6c28b8e8-effe-41ab-8e31-72482c05662d"
/>
2025-10-15 11:06:42 -07:00
Sumanth Kamenani
bc8b377a7c
fix(vector-io): handle missing document_id in insert_chunks (#3521)
Fixed KeyError when chunks don't have document_id in metadata or
chunk_metadata. Updated logging to safely extract document_id using
getattr and RAG memory to handle different document_id locations. Added
test for missing document_id scenarios.

Fixes issue #3494 where /v1/vector-io/insert would crash with KeyError.
Fixed KeyError when chunks don't have document_id in metadata or
chunk_metadata. Updated logging to safely extract document_id using
getattr and RAG memory to handle different document_id locations. Added
test for missing document_id scenarios.

 # What does this PR do?

Fixes a KeyError crash in `/v1/vector-io/insert` when chunks are missing
`document_id` fields. The API
was failing even though `document_id` is optional according to the
schema.

  Closes #3494

  ## Test Plan

  **Before fix:**
  - POST to `/v1/vector-io/insert` with chunks → 500 KeyError
  - Happened regardless of where `document_id` was placed

  **After fix:**
  - Same request works fine → 200 OK
  - Tested with Postman using FAISS backend
  - Added unit test covering missing `document_id` scenarios
2025-10-15 11:02:48 -07:00
ehhuang
980e46d1f7
Merge f347df50b2 into sapling-pr-archive-ehhuang 2025-10-15 10:42:30 -07:00
Eric Huang
f347df50b2 chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:42:26 -07:00
Eric Huang
7698c336f3 merge commit for archive created by Sapling 2025-10-15 10:42:01 -07:00
Eric Huang
a067dd835e chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:41:57 -07:00
Eric Huang
d7c898aaa1 merge commit for archive created by Sapling 2025-10-15 10:40:58 -07:00
Eric Huang
7f98b911ae chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:40:53 -07:00
Eric Huang
7d64aea057 merge commit for archive created by Sapling 2025-10-15 10:40:21 -07:00
Eric Huang
018d6f0b10 chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:40:16 -07:00
Eric Huang
c0097a3f2d merge commit for archive created by Sapling 2025-10-15 10:39:22 -07:00
Eric Huang
ceb557bcf2 chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:39:17 -07:00
Eric Huang
a34d2ef005 merge commit for archive created by Sapling 2025-10-15 10:38:46 -07:00
Eric Huang
d2491ff522 chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:38:42 -07:00
ehhuang
109bb969d5
Merge ad6d48aaab into sapling-pr-archive-ehhuang 2025-10-15 10:33:40 -07:00
Eric Huang
ad6d48aaab chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:33:36 -07:00
ehhuang
a9e08d7b8e
Merge b93963949d into sapling-pr-archive-ehhuang 2025-10-15 10:33:06 -07:00
Eric Huang
b93963949d chore: mark recordings as generated files
# What does this PR do?


## Test Plan
2025-10-15 10:32:59 -07:00