phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-07-21 03:59:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: Add missing Vector Store Files API surface (#2468)
Some checks failed
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 2s
Details
Integration Tests / test-matrix (http, 3.11, tool_runtime) (push) Failing after 13s
Details
Integration Tests / test-matrix (http, 3.11, providers) (push) Failing after 13s
Details
Integration Tests / test-matrix (http, 3.11, datasets) (push) Failing after 14s
Details
Integration Tests / test-matrix (http, 3.11, inspect) (push) Failing after 14s
Details
Integration Tests / test-matrix (http, 3.12, agents) (push) Failing after 14s
Details
Integration Tests / test-matrix (http, 3.12, scoring) (push) Failing after 13s
Details
Integration Tests / test-matrix (library, 3.11, inspect) (push) Failing after 12s
Details
Integration Tests / test-matrix (library, 3.11, tool_runtime) (push) Failing after 12s
Details
Integration Tests / test-matrix (http, 3.12, post_training) (push) Failing after 15s
Details
Integration Tests / test-matrix (http, 3.12, inference) (push) Failing after 19s
Details
Integration Tests / test-matrix (http, 3.12, inspect) (push) Failing after 22s
Details
Integration Tests / test-matrix (http, 3.12, vector_io) (push) Failing after 17s
Details
Integration Tests / test-matrix (http, 3.11, post_training) (push) Failing after 23s
Details
Integration Tests / test-matrix (library, 3.11, datasets) (push) Failing after 14s
Details
Integration Tests / test-matrix (http, 3.11, vector_io) (push) Failing after 20s
Details
Integration Tests / test-matrix (library, 3.11, inference) (push) Failing after 16s
Details
Integration Tests / test-matrix (http, 3.11, agents) (push) Failing after 26s
Details
Integration Tests / test-matrix (http, 3.12, tool_runtime) (push) Failing after 19s
Details
Python Package Build Test / build (3.11) (push) Failing after 5s
Details
Integration Tests / test-matrix (library, 3.12, post_training) (push) Failing after 6s
Details
Python Package Build Test / build (3.12) (push) Failing after 3s
Details
Integration Tests / test-matrix (http, 3.12, providers) (push) Failing after 18s
Details
Integration Tests / test-matrix (library, 3.12, providers) (push) Failing after 10s
Details
Integration Tests / test-matrix (library, 3.11, post_training) (push) Failing after 17s
Details
Integration Tests / test-matrix (library, 3.11, vector_io) (push) Failing after 15s
Details
Integration Tests / test-matrix (library, 3.11, scoring) (push) Failing after 18s
Details
Integration Tests / test-matrix (library, 3.12, datasets) (push) Failing after 13s
Details
Integration Tests / test-matrix (library, 3.12, scoring) (push) Failing after 8s
Details
Python Package Build Test / build (3.13) (push) Failing after 5s
Details
Integration Tests / test-matrix (http, 3.11, scoring) (push) Failing after 24s
Details
Integration Tests / test-matrix (library, 3.11, agents) (push) Failing after 20s
Details
Integration Tests / test-matrix (library, 3.12, inspect) (push) Failing after 10s
Details
Integration Tests / test-matrix (library, 3.12, tool_runtime) (push) Failing after 7s
Details
Integration Tests / test-matrix (library, 3.11, providers) (push) Failing after 15s
Details
Integration Tests / test-matrix (http, 3.12, datasets) (push) Failing after 21s
Details
Integration Tests / test-matrix (library, 3.12, inference) (push) Failing after 12s
Details
Integration Tests / test-matrix (library, 3.12, agents) (push) Failing after 15s
Details
Integration Tests / test-matrix (http, 3.11, inference) (push) Failing after 22s
Details
Unit Tests / unit-tests (3.11) (push) Failing after 7s
Details
Update ReadTheDocs / update-readthedocs (push) Failing after 4s
Details
Unit Tests / unit-tests (3.12) (push) Failing after 7s
Details
Integration Tests / test-matrix (library, 3.12, vector_io) (push) Failing after 48s
Details
Test External Providers / test-external-providers (venv) (push) Failing after 43s
Details
Unit Tests / unit-tests (3.13) (push) Failing after 52s
Details
Pre-commit / pre-commit (push) Successful in 2m4s
Details

Browse source 
# What does this PR do?

This adds the ability to list, retrieve, update, and delete Vector Store
Files. It implements these new APIs for the faiss and sqlite-vec
providers, since those are the two that also have the rest of the vector
store files implementation.

Closes #2445 

## Test Plan

### test_openai_vector_stores Integration Tests

There are a number of new integration tests added, which I ran for each
provider as outlined below.

faiss (from ollama distro):

```
INFERENCE_MODEL="meta-llama/Llama-3.2-3B-Instruct" \
llama stack run llama_stack/templates/ollama/run.yaml

LLAMA_STACK_CONFIG=http://localhost:8321 \
pytest -sv tests/integration/vector_io/test_openai_vector_stores.py \
  --embedding-model=all-MiniLM-L6-v2
```

sqlite-vec (from starter distro):

```
llama stack run llama_stack/templates/starter/run.yaml

LLAMA_STACK_CONFIG=http://localhost:8321 \
pytest -sv tests/integration/vector_io/test_openai_vector_stores.py \
  --embedding-model=all-MiniLM-L6-v2
```

### file_search verification tests

I also ensured the file_search verification tests continue to work, both
for faiss and sqlite-vec.

faiss (ollama distro):

```
INFERENCE_MODEL="meta-llama/Llama-3.2-3B-Instruct" \
llama stack run llama_stack/templates/ollama/run.yaml

pytest -sv tests/verifications/openai_api/test_responses.py \
  -k'file_search' \
  --base-url=http://localhost:8321/v1/openai/v1 \
  --model=meta-llama/Llama-3.2-3B-Instruct
```


sqlite-vec (starter distro):

```
llama stack run llama_stack/templates/starter/run.yaml

pytest -sv tests/verifications/openai_api/test_responses.py \
  -k'file_search' \
  --base-url=http://localhost:8321/v1/openai/v1 \
  --model=together/meta-llama/Llama-3.2-3B-Instruct-Turbo
```

---------

Signed-off-by: Ben Browning <bbrownin@redhat.com>
This commit is contained in:
Ben Browning 2025-06-19 11:08:24 -04:00 • committed by GitHub
parent a2f054607d
commit f394c7f2d9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 1991 additions and 122 deletions
Download patch file Download diff file Expand all files Collapse all files

371
docs/_static/llama-stack-spec.yaml vendored
View file

@ -2264,6 +2264,61 @@ paths:
$ref: '#/components/schemas/LogEventRequest'
required: true
/v1/openai/v1/vector_stores/{vector_store_id}/files:
get:
responses:
'200':
description: >-
A VectorStoreListFilesResponse containing the list of files.
content:
application/json:
schema:
$ref: '#/components/schemas/VectorStoreListFilesResponse'
'400':
$ref: '#/components/responses/BadRequest400'
'429':
$ref: >-
#/components/responses/TooManyRequests429
'500':
$ref: >-
#/components/responses/InternalServerError500
default:
$ref: '#/components/responses/DefaultError'
tags:
- VectorIO
description: List files in a vector store.
parameters:
- name: vector_store_id
in: path
description: >-
The ID of the vector store to list files from.
required: true
schema:
type: string
- name: limit
in: query
required: false
schema:
type: integer
- name: order
in: query
required: false
schema:
type: string
- name: after
in: query
required: false
schema:
type: string
- name: before
in: query
required: false
schema:
type: string
- name: filter
in: query
required: false
schema:
$ref: '#/components/schemas/VectorStoreFileStatus'
post:
responses:
'200':
@ -2572,6 +2627,121 @@ paths:
required: true
schema:
type: string
/v1/openai/v1/vector_stores/{vector_store_id}/files/{file_id}:
get:
responses:
'200':
description: >-
A VectorStoreFileObject representing the file.
content:
application/json:
schema:
$ref: '#/components/schemas/VectorStoreFileObject'
'400':
$ref: '#/components/responses/BadRequest400'
'429':
$ref: >-
#/components/responses/TooManyRequests429
'500':
$ref: >-
#/components/responses/InternalServerError500
default:
$ref: '#/components/responses/DefaultError'
tags:
- VectorIO
description: Retrieves a vector store file.
parameters:
- name: vector_store_id
in: path
description: >-
The ID of the vector store containing the file to retrieve.
required: true
schema:
type: string
- name: file_id
in: path
description: The ID of the file to retrieve.
required: true
schema:
type: string
post:
responses:
'200':
description: >-
A VectorStoreFileObject representing the updated file.
content:
application/json:
schema:
$ref: '#/components/schemas/VectorStoreFileObject'
'400':
$ref: '#/components/responses/BadRequest400'
'429':
$ref: >-
#/components/responses/TooManyRequests429
'500':
$ref: >-
#/components/responses/InternalServerError500
default:
$ref: '#/components/responses/DefaultError'
tags:
- VectorIO
description: Updates a vector store file.
parameters:
- name: vector_store_id
in: path
description: >-
The ID of the vector store containing the file to update.
required: true
schema:
type: string
- name: file_id
in: path
description: The ID of the file to update.
required: true
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/OpenaiUpdateVectorStoreFileRequest'
required: true
delete:
responses:
'200':
description: >-
A VectorStoreFileDeleteResponse indicating the deletion status.
content:
application/json:
schema:
$ref: '#/components/schemas/VectorStoreFileDeleteResponse'
'400':
$ref: '#/components/responses/BadRequest400'
'429':
$ref: >-
#/components/responses/TooManyRequests429
'500':
$ref: >-
#/components/responses/InternalServerError500
default:
$ref: '#/components/responses/DefaultError'
tags:
- VectorIO
description: Delete a vector store file.
parameters:
- name: vector_store_id
in: path
description: >-
The ID of the vector store containing the file to delete.
required: true
schema:
type: string
- name: file_id
in: path
description: The ID of the file to delete.
required: true
schema:
type: string
/v1/openai/v1/embeddings:
post:
responses:
@ -2762,6 +2932,44 @@ paths:
required: true
schema:
type: string
/v1/openai/v1/vector_stores/{vector_store_id}/files/{file_id}/content:
get:
responses:
'200':
description: >-
A list of InterleavedContent representing the file contents.
content:
application/json:
schema:
$ref: '#/components/schemas/VectorStoreFileContentsResponse'
'400':
$ref: '#/components/responses/BadRequest400'
'429':
$ref: >-
#/components/responses/TooManyRequests429
'500':
$ref: >-
#/components/responses/InternalServerError500
default:
$ref: '#/components/responses/DefaultError'
tags:
- VectorIO
description: >-
Retrieves the contents of a vector store file.
parameters:
- name: vector_store_id
in: path
description: >-
The ID of the vector store containing the file to retrieve.
required: true
schema:
type: string
- name: file_id
in: path
description: The ID of the file to retrieve.
required: true
schema:
type: string
/v1/openai/v1/vector_stores/{vector_store_id}/search:
post:
responses:
@ -8458,15 +8666,7 @@ components:
last_error:
$ref: '#/components/schemas/VectorStoreFileLastError'
status:
oneOf:
- type: string
const: completed
- type: string
const: in_progress
- type: string
const: cancelled
- type: string
const: failed
$ref: '#/components/schemas/VectorStoreFileStatus'
usage_bytes:
type: integer
default: 0
@ -8484,6 +8684,16 @@ components:
- vector_store_id
title: VectorStoreFileObject
description: OpenAI Vector Store File object.
VectorStoreFileStatus:
oneOf:
- type: string
const: completed
- type: string
const: in_progress
- type: string
const: cancelled
- type: string
const: failed
OpenAIJSONSchema:
type: object
properties:
@ -9031,6 +9241,27 @@ components:
required:
- name
title: OpenaiCreateVectorStoreRequest
VectorStoreFileCounts:
type: object
properties:
completed:
type: integer
cancelled:
type: integer
failed:
type: integer
in_progress:
type: integer
total:
type: integer
additionalProperties: false
required:
- completed
- cancelled
- failed
- in_progress
- total
title: VectorStoreFileCounts
VectorStoreObject:
type: object
properties:
@ -9047,9 +9278,7 @@ components:
type: integer
default: 0
file_counts:
type: object
additionalProperties:
type: integer
$ref: '#/components/schemas/VectorStoreFileCounts'
status:
type: string
default: completed
@ -9129,6 +9358,25 @@ components:
- deleted
title: VectorStoreDeleteResponse
description: Response from deleting a vector store.
VectorStoreFileDeleteResponse:
type: object
properties:
id:
type: string
object:
type: string
default: vector_store.file.deleted
deleted:
type: boolean
default: true
additionalProperties: false
required:
- id
- object
- deleted
title: VectorStoreFileDeleteResponse
description: >-
Response from deleting a vector store file.
OpenaiEmbeddingsRequest:
type: object
properties:
@ -9320,6 +9568,30 @@ components:
title: OpenAIFileObject
description: >-
OpenAI File object as defined in the OpenAI Files API.
VectorStoreListFilesResponse:
type: object
properties:
object:
type: string
default: list
data:
type: array
items:
$ref: '#/components/schemas/VectorStoreFileObject'
first_id:
type: string
last_id:
type: string
has_more:
type: boolean
default: false
additionalProperties: false
required:
- object
- data
- has_more
title: VectorStoreListFilesResponse
description: Response from listing vector stores.
OpenAIModel:
type: object
properties:
@ -9379,6 +9651,49 @@ components:
Response:
type: object
title: Response
VectorStoreContent:
type: object
properties:
type:
type: string
const: text
text:
type: string
additionalProperties: false
required:
- type
- text
title: VectorStoreContent
VectorStoreFileContentsResponse:
type: object
properties:
file_id:
type: string
filename:
type: string
attributes:
type: object
additionalProperties:
oneOf:
- type: 'null'
- type: boolean
- type: number
- type: string
- type: array
- type: object
content:
type: array
items:
$ref: '#/components/schemas/VectorStoreContent'
additionalProperties: false
required:
- file_id
- filename
- attributes
- content
title: VectorStoreFileContentsResponse
description: >-
Response from retrieving the contents of a vector store file.
OpenaiSearchVectorStoreRequest:
type: object
properties:
@ -9426,19 +9741,6 @@ components:
required:
- query
title: OpenaiSearchVectorStoreRequest
VectorStoreContent:
type: object
properties:
type:
type: string
const: text
text:
type: string
additionalProperties: false
required:
- type
- text
title: VectorStoreContent
VectorStoreSearchResponse:
type: object
properties:
@ -9524,6 +9826,25 @@ components:
Set of 16 key-value pairs that can be attached to an object.
additionalProperties: false
title: OpenaiUpdateVectorStoreRequest
OpenaiUpdateVectorStoreFileRequest:
type: object
properties:
attributes:
type: object
additionalProperties:
oneOf:
- type: 'null'
- type: boolean
- type: number
- type: string
- type: array
- type: object
description: >-
The updated key-value attributes to store with the file.
additionalProperties: false
required:
- attributes
title: OpenaiUpdateVectorStoreFileRequest
DPOAlignmentConfig:
type: object
properties: