phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-16 23:19:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/tests/integration/common/recordings/models-d98e7566147f9d534bc0461f2efe61e3f525c18360a07bb3dda397579e25c27b-a882f554.json
Francisco Javier Arceo 95b2948d11
Some checks failed
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 0s
Details
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 0s
Details
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
Details
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Details
Integration Tests (Replay) / generate-matrix (push) Successful in 3s
Details
API Conformance Tests / check-schema-compatibility (push) Successful in 11s
Details
Python Package Build Test / build (3.12) (push) Successful in 15s
Details
Python Package Build Test / build (3.13) (push) Successful in 20s
Details
Test External API and Providers / test-external (venv) (push) Failing after 41s
Details
Vector IO Integration Tests / test-matrix (push) Failing after 49s
Details
UI Tests / ui-tests (22) (push) Successful in 51s
Details
Unit Tests / unit-tests (3.13) (push) Failing after 1m27s
Details
Unit Tests / unit-tests (3.12) (push) Failing after 1m45s
Details
Pre-commit / pre-commit (22) (push) Failing after 2m30s
Details
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 4m22s
Details
feat: Add support for query rewrite in vector_store.search (#4171) 
# What does this PR do?

Actualize query rewrite in search API, add
`default_query_expansion_model` and `query_expansion_prompt` in
`VectorStoresConfig`.

Makes `rewrite_query` parameter functional in vector store search.
  - `rewrite_query=false` (default): Use original query
- `rewrite_query=true`: Expand query via LLM, or fail gracefully if no
LLM available

Adds 4 parameters to`VectorStoresConfig`:
- `default_query_expansion_model`: LLM model for query expansion
(optional)
- `query_expansion_prompt`: Custom prompt template (optional, uses
built-in default)
- `query_expansion_max_tokens`: Configurable token limit (default: 100)
- `query_expansion_temperature`: Configurable temperature (default: 0.3)

Enabled `run.yaml`:
```yaml
  vector_stores:
    rewrite_query_params:
      model:
        provider_id: "ollama"
        model_id: "llama3.2:3b-instruct-fp16"
      # prompt defaults to built-in
      # max_tokens defaults to 100
      # temperature defaults to 0.3
```

  Fully customized `run.yaml`:
```yaml
  vector_stores:
    default_provider_id: faiss
    default_embedding_model:
      provider_id: sentence-transformers
      model_id: nomic-ai/nomic-embed-text-v1.5
    rewrite_query_params:
      model:
        provider_id: ollama
        model_id: llama3.2:3b-instruct-fp16
      prompt: "Rewrite this search query to improve retrieval results by expanding it with relevant synonyms and related terms: {query}"
      max_tokens: 100
      temperature: 0.3
```

## Test Plan
Added test and recording

Example script as well:

```python
import asyncio
from llama_stack_client import LlamaStackClient
from io import BytesIO

def gen_file(client, text: str=""):
    file_buffer = BytesIO(text.encode('utf-8'))
    file_buffer.name = "my_file.txt"

    uploaded_file = client.files.create(
        file=file_buffer,
        purpose="assistants"
    )
    return uploaded_file

async def test_query_rewriting():
    client = LlamaStackClient(base_url="http://0.0.0.0:8321/")
    uploaded_file = gen_file(client, "banana banana apple")
    uploaded_file2 = gen_file(client, "orange orange kiwi")

    vs = client.vector_stores.create()
    xf_vs = client.vector_stores.files.create(vector_store_id=vs.id, file_id=uploaded_file.id)
    xf_vs1 = client.vector_stores.files.create(vector_store_id=vs.id, file_id=uploaded_file2.id)
    response1 = client.vector_stores.search(
                vector_store_id=vs.id,
                query="apple",
                max_num_results=3,
                rewrite_query=False
            )
    response2 = client.vector_stores.search(
                vector_store_id=vs.id,
                query="kiwi",
                max_num_results=3,
                rewrite_query=True,
            )

    print(f"\n🔵 Response 1 (rewrite_query=False):\n\033[94m{response1}\033[0m")
    print(f"\n🟢 Response 2 (rewrite_query=True):\n\033[92m{response2}\033[0m")

    for f in [uploaded_file.id, uploaded_file2.id]:
        client.files.delete(file_id=f)
    client.vector_stores.delete(vector_store_id=vs.id)

if __name__ == "__main__":
    asyncio.run(test_query_rewriting())
```

And see the screen shot of the server logs showing it worked. 
<img width="1111" height="826" alt="Screenshot 2025-11-19 at 1 16 03 PM"
src="https://github.com/user-attachments/assets/2d188b44-1fef-4df5-b465-2d6728ca49ce"
/>

Notice the log:
```bash
 Query rewritten:
         'kiwi' → 'kiwi, a small brown or green fruit native to New Zealand, or a person having a fuzzy brown outer skin similar in appearance.'
```
So `kiwi` was expanded.

---------

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu>
2025-12-10 10:06:19 -05:00

647 lines
18 KiB
JSON
Generated
Raw Blame History

{
"test_id": null,
"request": {
"method": "POST",
"url": "https://generativelanguage.googleapis.com/v1beta/openai/v1/models",
"headers": {},
"body": {},
"endpoint": "/v1/models",
"model": ""
},
"response": {
"body": [
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/embedding-gecko-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Embedding Gecko"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-pro-preview-03-25",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Pro Preview 03-25"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-preview-05-20",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Preview 05-20"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-lite-preview-06-17",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash-Lite Preview 06-17"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-pro-preview-05-06",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Pro Preview 05-06"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-pro-preview-06-05",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Pro Preview"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-pro",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Pro"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-exp",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash Experimental"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash 001"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-exp-image-generation",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash (Image Generation) Experimental"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-lite-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash-Lite 001"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-lite",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash-Lite"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-lite-preview-02-05",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash-Lite Preview 02-05"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-lite-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash-Lite Preview"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-pro-exp",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Pro Experimental"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-pro-exp-02-05",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Pro Experimental 02-05"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-exp-1206",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Experimental 1206"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-thinking-exp-01-21",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Preview 05-20"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-thinking-exp",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Preview 05-20"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-thinking-exp-1219",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Preview 05-20"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-preview-tts",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Preview TTS"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-pro-preview-tts",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Pro Preview TTS"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/learnlm-2.0-flash-experimental",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "LearnLM 2.0 Flash Experimental"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemma-3-1b-it",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemma 3 1B"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemma-3-4b-it",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemma 3 4B"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemma-3-12b-it",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemma 3 12B"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemma-3-27b-it",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemma 3 27B"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemma-3n-e4b-it",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemma 3n E4B"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemma-3n-e2b-it",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemma 3n E2B"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-flash-latest",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Flash Latest"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-flash-lite-latest",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Flash-Lite Latest"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-pro-latest",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Pro Latest"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-lite",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash-Lite"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-image-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Nano Banana"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-image",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Nano Banana"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-preview-09-2025",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Preview Sep 2025"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-lite-preview-09-2025",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash-Lite Preview Sep 2025"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-robotics-er-1.5-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Robotics-ER 1.5 Preview"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-computer-use-preview-10-2025",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Computer Use Preview 10-2025"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/embedding-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Embedding 001"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/text-embedding-004",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Text Embedding 004"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-embedding-exp-03-07",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Embedding Experimental 03-07"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-embedding-exp",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Embedding Experimental"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-embedding-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Embedding 001"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/aqa",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Model that performs Attributed Question Answering."
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/imagen-4.0-generate-preview-06-06",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Imagen 4 (Preview)"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/imagen-4.0-ultra-generate-preview-06-06",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Imagen 4 Ultra (Preview)"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/imagen-4.0-generate-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Imagen 4"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/imagen-4.0-ultra-generate-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Imagen 4 Ultra"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/imagen-4.0-fast-generate-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Imagen 4 Fast"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/veo-2.0-generate-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Veo 2"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/veo-3.0-generate-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Veo 3"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/veo-3.0-fast-generate-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Veo 3 fast"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/veo-3.1-generate-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Veo 3.1"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/veo-3.1-fast-generate-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Veo 3.1 fast"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.0-flash-live-001",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.0 Flash 001"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-live-2.5-flash-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini Live 2.5 Flash Preview"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-live-preview",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Live Preview"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-native-audio-latest",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Native Audio Latest"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/gemini-2.5-flash-native-audio-preview-09-2025",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Gemini 2.5 Flash Native Audio Preview 09-2025"
}
},
{
"__type__": "openai.types.model.Model",
"__data__": {
"id": "models/lyria-realtime-exp",
"created": null,
"object": "model",
"owned_by": "google",
"display_name": "Lyria Realtime Experimental"
}
}
],
"is_streaming": false
},
"id_normalization_mapping": {}
}
Reference in a new issue View git blame Copy permalink