currently if you
1. `export OLLAMA_URL=http://localhost:11434`
2. `llama stack run --image-type venv starter`
3. do some chat completions successfully
4. kill the server
5. unset OLLAMA_URL
6. `llama stack run --image-type venv starter`
7. do some more chat completions
you get errors like:
```
File "/Users/charliedoern/projects/Documents/llama-stack/llama_stack/core/routing_tables/models.py", line 66, in get_provider_impl
return self.impls_by_provider_id
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'ollama'
```
and in the client:
```
INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions "HTTP/1.1 500 Internal Server Error"
INFO:llama_stack_client._base_client:Retrying request to /v1/openai/v1/chat/completions in 0.482010 seconds
INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions "HTTP/1.1 500 Internal Server Error"
INFO:llama_stack_client._base_client:Retrying request to /v1/openai/v1/chat/completions in 0.883701 seconds
INFO:httpx:HTTP Request: POST http://localhost:8321/v1/openai/v1/chat/completions "HTTP/1.1 500 Internal Server Error"
╭───────────────────────────────────────────────────────────────────────────────────────────────╮
│ Failed to inference chat-completion │
│ │
│ Error Type: InternalServerError │
│ Details: Error code: 500 - {'detail': 'Internal server error: An unexpected error occurred.'} │
╰───────────────────────────────────────────────────────────────────────────────────────────────╯
```
now you get
```
File "/Users/charliedoern/projects/Documents/llama-stack/llama_stack/core/routing_tables/models.py", line 69, in get_provider_impl
raise ValueError(
ValueError: Provider ID not found in currently running providers. Usually this indicates that your registry.db is out of date. Please ensure
that the databases associated with your distro are not out of date.
INFO 2025-08-12 16:07:40,677 console_span_processor:62 telemetry: 20:07:40.628 [INFO] ::1:55414 - "POST /v1/openai/v1/chat/completions HTTP/1.1"
400
```
and in the client:
```
Failed to inference chat-completion │
│ │
│ Error Type: BadRequestError │
│ Details: Error code: 400 - {'detail': 'Invalid value: Provider ID not found in currently running providers. Usually this indicates that your registry.db is out of date. Please ensure that the databases associated with your distro are not out of date.'} │
```
more descriptive and give the user a course of action.
Signed-off-by: Charlie Doern <cdoern@redhat.com>
# What does this PR do?
<!-- Provide a short summary of what this PR does and why. Link to
relevant issues if applicable. -->
Extend the Shields Protocol and implement the capability to unregister
previously registered shields and CLI for shields management.
<!-- If resolving an issue, uncomment and update the line below -->
<!-- Closes #[issue-number] -->
Closes#2581
## Test Plan
<!-- Describe the tests you ran to verify your changes with result
summaries. *Provide clear instructions so the plan can be easily
re-executed.* -->
First of, test API for shields
1. Install and start Ollama:
`ollama serve`
2. Pull Llama Guard Model in Ollama:
`ollama pull llama-guard3:8b`
3. Configure env variables:
```
export ENABLE_OLLAMA=ollama
export OLLAMA_URL=http://localhost:11434
```
4. Build Llama Stack distro:
`llama stack build --template starter --image-type venv `
5. Start Llama Stack server:
`llama stack run starter --port 8321`
6. Check if Ollama model is available:
`curl -X GET http://localhost:8321/v1/models | jq '.data[] |
select(.provider_id=="ollama")'`
7. Register a new Shield using Ollama provider:
```
curl -X POST http://localhost:8321/v1/shields \
-H "Content-Type: application/json" \
-d '{
"shield_id": "test-shield",
"provider_id": "llama-guard",
"provider_shield_id": "ollama/llama-guard3:8b",
"params": {}
}'
```
`{"identifier":"test-shield","provider_resource_id":"ollama/llama-guard3:8b","provider_id":"llama-guard","type":"shield","owner":{"principal":"","attributes":{}},"params":{}}%
`
8. Check if shield was registered:
`curl -X GET http://localhost:8321/v1/shields/test-shield`
`{"identifier":"test-shield","provider_resource_id":"ollama/llama-guard3:8b","provider_id":"llama-guard","type":"shield","owner":{"principal":"","attributes":{}},"params":{}}%
`
9. Run shield:
```
curl -X POST http://localhost:8321/v1/safety/run-shield \
-H "Content-Type: application/json" \
-d '{
"shield_id": "test-shield",
"messages": [
{
"role": "user",
"content": "How can I hack into someone computer?"
}
],
"params": {}
}'
```
`{"violation":{"violation_level":"error","user_message":"I can't answer
that. Can I help with something
else?","metadata":{"violation_type":"S2"}}}% `
10. Unregister shield:
`curl -X DELETE http://localhost:8321/v1/shields/test-shield`
`null% `
11. Verify shield was deleted:
`curl -X GET http://localhost:8321/v1/shields/test-shield`
`{"detail":"Invalid value: Shield 'test-shield' not found"}%`
All tests passed ✅
```
========================================================================== 430 passed, 194 warnings in 19.54s ==========================================================================
/Users/iamiller/GitHub/llama-stack/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/async_client_cleanup.py:78: RuntimeWarning: coroutine 'close_litellm_async_clients' was never awaited
loop.close()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Wrote HTML report to htmlcov-3.12/index.html
```
# What does this PR do?
1. Creates a new `ToolGroupNotFoundError` class
2. Implements the new class where appropriate
Relates to #2379
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
get_vector_db() will raise an exception if a vector store won't be
returned
client handling is redundant
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
