phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-19 20:48:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: Making static prompt values in Rag/File Search configurable in Vector Store Config (#4368)

Browse source 
# What does this PR do?

- Enables users to configure prompts used throughout the File Search /
Vector Retrieval
- Configuration is defined in the Vector Stores Config so they can be
modified at runtime
- Backwards compatible, which means the fields are optional and default
to the previously used values

This is the summary of the new options in the `run.yaml`
```yaml
vector_stores:
  file_search_params:
    header_template: 'knowledge_search tool found {num_chunks} chunks:\nBEGIN of knowledge_search tool results.\n'
    footer_template: 'END of knowledge_search tool results.\n'
  context_prompt_params:
    chunk_annotation_template: 'Result {index}\nContent: {chunk.content}\nMetadata: {metadata}\n'
    context_template: 'The above results were retrieved to help answer the user\'s query: "{query}". Use them as supporting information only in answering this query.{annotation_instruction}\n'
  annotation_prompt_params:
    enable_annotations: true
    annotation_instruction_template: 'Cite sources immediately at the end of sentences before punctuation, using `<|file-id|>` format like \'This is a fact <|file-Cn3MSNn72ENTiiq11Qda4A|>.\'. Do not add
extra punctuation. Use only the file IDs provided, do not invent new ones.'
    chunk_annotation_template: '[{index}] {metadata_text} cite as <|{file_id}|>\n{chunk_text}\n'
```

<!-- If resolving an issue, uncomment and update the line below -->
<!-- Closes #[issue-number] -->

## Test Plan
Added tests.

---------

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
This commit is contained in:
Francisco Javier Arceo 2025-12-15 11:39:01 -05:00 • committed by GitHub
parent 4043dedeea
commit 62005dc1a9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
47 changed files with 42640 additions and 40 deletions
Download patch file Download diff file Expand all files Collapse all files

33
src/llama_stack/distributions/starter-gpu/run-with-postgres-store.yaml
View file

@ -289,5 +289,38 @@ vector_stores:
default_embedding_model:
provider_id: sentence-transformers
model_id: nomic-ai/nomic-embed-text-v1.5
file_search_params:
header_template: 'knowledge_search tool found {num_chunks} chunks:
BEGIN of knowledge_search tool results.
'
footer_template: 'END of knowledge_search tool results.
'
context_prompt_params:
chunk_annotation_template: 'Result {index}
Content: {chunk.content}
Metadata: {metadata}
'
context_template: 'The above results were retrieved to help answer the user''s
query: "{query}". Use them as supporting information only in answering this
query.{annotation_instruction}
'
annotation_prompt_params:
enable_annotations: true
annotation_instruction_template: ' Cite sources immediately at the end of sentences
before punctuation, using `<|file-id|>` format like ''This is a fact <|file-Cn3MSNn72ENTiiq11Qda4A|>.''.
Do not add extra punctuation. Use only the file IDs provided, do not invent
new ones.'
chunk_annotation_template: '[{index}] {metadata_text} cite as <|{file_id}|>
{chunk_text}
'
safety:
default_shield_id: llama-guard