phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 04:04:14 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: load config class when doing variable substitution

Browse source 
When using bash style substitution env variable in distribution
template, we are processing the string and convert it to the type
associated with the provider's config class. This allows us to return
the proper type. This is crucial for api key since they are not strings
anymore but SecretStr. If the key is unset we will get an empty string
which will result in a Pydantic error like:

```
ERROR    2025-09-25 21:40:44,565 __main__:527 core::server: Error creating app: 1 validation error for AnthropicConfig
         api_key
           Input should be a valid string
             For further information visit
             https://errors.pydantic.dev/2.11/v/string_type
```

Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is contained in:
Sébastien Han 2025-09-25 10:27:41 +02:00
parent 4af141292f
commit bc64635835
No known key found for this signature in database
79 changed files with 381 additions and 216 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/docs/providers/datasetio/remote_nvidia.mdx
View file

@ -14,7 +14,7 @@ NVIDIA's dataset I/O provider for accessing datasets from NVIDIA's data platform
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `str \| None` | No | | The NVIDIA API key. |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The NVIDIA API key. |
| `dataset_namespace` | `str \| None` | No | default | The NVIDIA dataset namespace. |
| `project_id` | `str \| None` | No | test-project | The NVIDIA project ID. |
| `datasets_url` | `<class 'str'>` | No | http://nemo.test | Base URL for the NeMo Dataset API |

2
docs/docs/providers/files/remote_s3.mdx
View file

@ -17,7 +17,7 @@ AWS S3-based file storage provider for scalable cloud file management with metad
| `bucket_name` | `<class 'str'>` | No | | S3 bucket name to store files |
| `region` | `<class 'str'>` | No | us-east-1 | AWS region where the bucket is located |
| `aws_access_key_id` | `str \| None` | No | | AWS access key ID (optional if using IAM roles) |
| `aws_secret_access_key` | `<class 'pydantic.types.SecretStr'>` | No | | AWS secret access key (optional if using IAM roles) |
| `aws_secret_access_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | AWS secret access key (optional if using IAM roles) |
| `endpoint_url` | `str \| None` | No | | Custom S3 endpoint URL (for MinIO, LocalStack, etc.) |
| `auto_create_bucket` | `<class 'bool'>` | No | False | Automatically create the S3 bucket if it doesn't exist |
| `metadata_store` | `utils.sqlstore.sqlstore.SqliteSqlStoreConfig \| utils.sqlstore.sqlstore.PostgresSqlStoreConfig` | No | sqlite | SQL store configuration for file metadata |

2
docs/docs/providers/inference/remote_anthropic.mdx
View file

@ -14,7 +14,7 @@ Anthropic inference provider for accessing Claude models and Anthropic's AI serv
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | API key for Anthropic models |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | API key for Anthropic models |
## Sample Configuration

2
docs/docs/providers/inference/remote_azure.mdx
View file

@ -21,7 +21,7 @@ https://learn.microsoft.com/en-us/azure/ai-foundry/openai/overview
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | Azure API key for Azure |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | Azure API key for Azure |
| `api_base` | `<class 'pydantic.networks.HttpUrl'>` | No | | Azure API base for Azure (e.g., https://your-resource-name.openai.azure.com) |
| `api_version` | `str \| None` | No | | Azure API version for Azure (e.g., 2024-12-01-preview) |
| `api_type` | `str \| None` | No | azure | Azure API type for Azure (e.g., azure) |

4
docs/docs/providers/inference/remote_bedrock.mdx
View file

@ -15,8 +15,8 @@ AWS Bedrock inference provider for accessing various AI models through AWS's man
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `aws_access_key_id` | `str \| None` | No | | The AWS access key to use. Default use environment variable: AWS_ACCESS_KEY_ID |
| `aws_secret_access_key` | `<class 'pydantic.types.SecretStr'>` | No | | The AWS secret access key to use. Default use environment variable: AWS_SECRET_ACCESS_KEY |
| `aws_session_token` | `<class 'pydantic.types.SecretStr'>` | No | | The AWS session token to use. Default use environment variable: AWS_SESSION_TOKEN |
| `aws_secret_access_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The AWS secret access key to use. Default use environment variable: AWS_SECRET_ACCESS_KEY |
| `aws_session_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The AWS session token to use. Default use environment variable: AWS_SESSION_TOKEN |
| `region_name` | `str \| None` | No | | The default AWS Region to use, for example, us-west-1 or us-west-2.Default use environment variable: AWS_DEFAULT_REGION |
| `profile_name` | `str \| None` | No | | The profile name that contains credentials to use.Default use environment variable: AWS_PROFILE |
| `total_max_attempts` | `int \| None` | No | | An integer representing the maximum number of attempts that will be made for a single request, including the initial attempt. Default use environment variable: AWS_MAX_ATTEMPTS |

2
docs/docs/providers/inference/remote_cerebras.mdx
View file

@ -15,7 +15,7 @@ Cerebras inference provider for running models on Cerebras Cloud platform.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `base_url` | `<class 'str'>` | No | https://api.cerebras.ai | Base URL for the Cerebras API |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | Cerebras API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | Cerebras API Key |
## Sample Configuration

2
docs/docs/providers/inference/remote_databricks.mdx
View file

@ -15,7 +15,7 @@ Databricks inference provider for running models on Databricks' unified analytic
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `<class 'str'>` | No | | The URL for the Databricks model serving endpoint |
| `api_token` | `<class 'pydantic.types.SecretStr'>` | No | | The Databricks API token |
| `api_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Databricks API token |
## Sample Configuration

2
docs/docs/providers/inference/remote_fireworks.mdx
View file

@ -16,7 +16,7 @@ Fireworks AI inference provider for Llama models and other AI models on the Fire
|-------|------|----------|---------|-------------|
| `allowed_models` | `list[str \| None` | No | | List of models that should be registered with the model registry. If None, all models are allowed. |
| `url` | `<class 'str'>` | No | https://api.fireworks.ai/inference/v1 | The URL for the Fireworks server |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Fireworks.ai API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Fireworks.ai API Key |
## Sample Configuration

2
docs/docs/providers/inference/remote_gemini.mdx
View file

@ -14,7 +14,7 @@ Google Gemini inference provider for accessing Gemini models and Google's AI ser
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | API key for Gemini models |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | API key for Gemini models |
## Sample Configuration

2
docs/docs/providers/inference/remote_groq.mdx
View file

@ -14,7 +14,7 @@ Groq inference provider for ultra-fast inference using Groq's LPU technology.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Groq API key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Groq API key |
| `url` | `<class 'str'>` | No | https://api.groq.com | The URL for the Groq AI server |
## Sample Configuration

2
docs/docs/providers/inference/remote_hf_endpoint.mdx
View file

@ -15,7 +15,7 @@ HuggingFace Inference Endpoints provider for dedicated model serving.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `endpoint_name` | `<class 'str'>` | No | | The name of the Hugging Face Inference Endpoint in the format of '&#123;namespace&#125;/&#123;endpoint_name&#125;' (e.g. 'my-cool-org/meta-llama-3-1-8b-instruct-rce'). Namespace is optional and will default to the user account if not provided. |
| `api_token` | `pydantic.types.SecretStr \| None` | No | | Your Hugging Face user access token (will default to locally saved token if not provided) |
| `api_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | Your Hugging Face user access token (will default to locally saved token if not provided) |
## Sample Configuration

2
docs/docs/providers/inference/remote_hf_serverless.mdx
View file

@ -15,7 +15,7 @@ HuggingFace Inference API serverless provider for on-demand model inference.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `huggingface_repo` | `<class 'str'>` | No | | The model ID of the model on the Hugging Face Hub (e.g. 'meta-llama/Meta-Llama-3.1-70B-Instruct') |
| `api_token` | `pydantic.types.SecretStr \| None` | No | | Your Hugging Face user access token (will default to locally saved token if not provided) |
| `api_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | Your Hugging Face user access token (will default to locally saved token if not provided) |
## Sample Configuration

2
docs/docs/providers/inference/remote_llama-openai-compat.mdx
View file

@ -14,7 +14,7 @@ Llama OpenAI-compatible provider for using Llama models with OpenAI API format.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Llama API key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Llama API key |
| `openai_compat_api_base` | `<class 'str'>` | No | https://api.llama.com/compat/v1/ | The URL for the Llama API server |
## Sample Configuration

2
docs/docs/providers/inference/remote_nvidia.mdx
View file

@ -15,7 +15,7 @@ NVIDIA inference provider for accessing NVIDIA NIM models and AI services.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `<class 'str'>` | No | https://integrate.api.nvidia.com | A base url for accessing the NVIDIA NIM |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The NVIDIA API key, only needed of using the hosted service |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The NVIDIA API key, only needed of using the hosted service |
| `timeout` | `<class 'int'>` | No | 60 | Timeout for the HTTP requests |
| `append_api_version` | `<class 'bool'>` | No | True | When set to false, the API version will not be appended to the base_url. By default, it is true. |

2
docs/docs/providers/inference/remote_openai.mdx
View file

@ -14,7 +14,7 @@ OpenAI inference provider for accessing GPT models and other OpenAI services.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | API key for OpenAI models |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | API key for OpenAI models |
| `base_url` | `<class 'str'>` | No | https://api.openai.com/v1 | Base URL for OpenAI API |
## Sample Configuration

2
docs/docs/providers/inference/remote_passthrough.mdx
View file

@ -15,7 +15,7 @@ Passthrough inference provider for connecting to any external inference service
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `<class 'str'>` | No | | The URL for the passthrough endpoint |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | API Key for the passthrouth endpoint |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | API Key for the passthrouth endpoint |
## Sample Configuration

2
docs/docs/providers/inference/remote_runpod.mdx
View file

@ -15,7 +15,7 @@ RunPod inference provider for running models on RunPod's cloud GPU platform.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `str \| None` | No | | The URL for the Runpod model serving endpoint |
| `api_token` | `<class 'pydantic.types.SecretStr'>` | No | | The API token |
| `api_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The API token |
## Sample Configuration

2
docs/docs/providers/inference/remote_sambanova.mdx
View file

@ -15,7 +15,7 @@ SambaNova inference provider for running models on SambaNova's dataflow architec
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `<class 'str'>` | No | https://api.sambanova.ai/v1 | The URL for the SambaNova AI server |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The SambaNova cloud API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The SambaNova cloud API Key |
## Sample Configuration

2
docs/docs/providers/inference/remote_together.mdx
View file

@ -16,7 +16,7 @@ Together AI inference provider for open-source models and collaborative AI devel
|-------|------|----------|---------|-------------|
| `allowed_models` | `list[str \| None` | No | | List of models that should be registered with the model registry. If None, all models are allowed. |
| `url` | `<class 'str'>` | No | https://api.together.xyz/v1 | The URL for the Together AI server |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Together AI API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Together AI API Key |
## Sample Configuration

2
docs/docs/providers/inference/remote_vllm.mdx
View file

@ -16,7 +16,7 @@ Remote vLLM inference provider for connecting to vLLM servers.
|-------|------|----------|---------|-------------|
| `url` | `str \| None` | No | | The URL for the vLLM model serving endpoint |
| `max_tokens` | `<class 'int'>` | No | 4096 | Maximum number of tokens to generate. |
| `api_token` | `<class 'pydantic.types.SecretStr'>` | No | | The API token |
| `api_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The API token |
| `tls_verify` | `bool \| str` | No | True | Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file. |
| `refresh_models` | `<class 'bool'>` | No | False | Whether to refresh models periodically |

2
docs/docs/providers/inference/remote_watsonx.mdx
View file

@ -15,7 +15,7 @@ IBM WatsonX inference provider for accessing AI models on IBM's WatsonX platform
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `<class 'str'>` | No | https://us-south.ml.cloud.ibm.com | A base url for accessing the watsonx.ai |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The watsonx API key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The watsonx API key |
| `project_id` | `str \| None` | No | | The Project ID key |
| `timeout` | `<class 'int'>` | No | 60 | Timeout for the HTTP requests |

2
docs/docs/providers/post_training/remote_nvidia.mdx
View file

@ -14,7 +14,7 @@ NVIDIA's post-training provider for fine-tuning models on NVIDIA's platform.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `str \| None` | No | | The NVIDIA API key. |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The NVIDIA API key. |
| `dataset_namespace` | `str \| None` | No | default | The NVIDIA dataset namespace. |
| `project_id` | `str \| None` | No | test-example-model@v1 | The NVIDIA project ID. |
| `customizer_url` | `str \| None` | No | | Base URL for the NeMo Customizer API |

4
docs/docs/providers/safety/remote_bedrock.mdx
View file

@ -15,8 +15,8 @@ AWS Bedrock safety provider for content moderation using AWS's safety services.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `aws_access_key_id` | `str \| None` | No | | The AWS access key to use. Default use environment variable: AWS_ACCESS_KEY_ID |
| `aws_secret_access_key` | `<class 'pydantic.types.SecretStr'>` | No | | The AWS secret access key to use. Default use environment variable: AWS_SECRET_ACCESS_KEY |
| `aws_session_token` | `<class 'pydantic.types.SecretStr'>` | No | | The AWS session token to use. Default use environment variable: AWS_SESSION_TOKEN |
| `aws_secret_access_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The AWS secret access key to use. Default use environment variable: AWS_SECRET_ACCESS_KEY |
| `aws_session_token` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The AWS session token to use. Default use environment variable: AWS_SESSION_TOKEN |
| `region_name` | `str \| None` | No | | The default AWS Region to use, for example, us-west-1 or us-west-2.Default use environment variable: AWS_DEFAULT_REGION |
| `profile_name` | `str \| None` | No | | The profile name that contains credentials to use.Default use environment variable: AWS_PROFILE |
| `total_max_attempts` | `int \| None` | No | | An integer representing the maximum number of attempts that will be made for a single request, including the initial attempt. Default use environment variable: AWS_MAX_ATTEMPTS |

2
docs/docs/providers/safety/remote_sambanova.mdx
View file

@ -15,7 +15,7 @@ SambaNova's safety provider for content moderation and safety filtering.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `url` | `<class 'str'>` | No | https://api.sambanova.ai/v1 | The URL for the SambaNova AI server |
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The SambaNova cloud API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The SambaNova cloud API Key |
## Sample Configuration

2
docs/docs/providers/scoring/inline_braintrust.mdx
View file

@ -14,7 +14,7 @@ Braintrust scoring provider for evaluation and scoring using the Braintrust plat
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `openai_api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The OpenAI API Key |
| `openai_api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The OpenAI API Key |
## Sample Configuration

2
docs/docs/providers/tool_runtime/remote_bing-search.mdx
View file

@ -14,7 +14,7 @@ Bing Search tool for web search capabilities using Microsoft's search engine.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Bing API key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Bing API key |
| `top_k` | `<class 'int'>` | No | 3 | |
## Sample Configuration

2
docs/docs/providers/tool_runtime/remote_brave-search.mdx
View file

@ -14,7 +14,7 @@ Brave Search tool for web search capabilities with privacy-focused results.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Brave Search API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Brave Search API Key |
| `max_results` | `<class 'int'>` | No | 3 | The maximum number of results to return |
## Sample Configuration

2
docs/docs/providers/tool_runtime/remote_tavily-search.mdx
View file

@ -14,7 +14,7 @@ Tavily Search tool for AI-optimized web search with structured results.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `<class 'pydantic.types.SecretStr'>` | No | | The Tavily Search API Key |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The Tavily Search API Key |
| `max_results` | `<class 'int'>` | No | 3 | The maximum number of results to return |
## Sample Configuration

2
docs/docs/providers/tool_runtime/remote_wolfram-alpha.mdx
View file

@ -14,7 +14,7 @@ Wolfram Alpha tool for computational knowledge and mathematical calculations.
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `api_key` | `str \| None` | No | | |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The WolframAlpha API Key |
## Sample Configuration

2
docs/docs/providers/vector_io/remote_pgvector.mdx
View file

@ -217,7 +217,7 @@ See [PGVector's documentation](https://github.com/pgvector/pgvector) for more de
| `port` | `int \| None` | No | 5432 | |
| `db` | `str \| None` | No | postgres | |
| `user` | `str \| None` | No | postgres | |
| `password` | `<class 'pydantic.types.SecretStr'>` | No | ********** | |
| `password` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | ********** | |
| `kvstore` | `utils.kvstore.config.RedisKVStoreConfig \| utils.kvstore.config.SqliteKVStoreConfig \| utils.kvstore.config.PostgresKVStoreConfig \| utils.kvstore.config.MongoDBKVStoreConfig, annotation=NoneType, required=False, default='sqlite', discriminator='type'` | No | | Config for KV store backend (SQLite only for now) |
## Sample Configuration

2
docs/docs/providers/vector_io/remote_qdrant.mdx
View file

@ -22,7 +22,7 @@ Please refer to the inline provider documentation.
| `grpc_port` | `<class 'int'>` | No | 6334 | |
| `prefer_grpc` | `<class 'bool'>` | No | False | |
| `https` | `bool \| None` | No | | |
| `api_key` | `str \| None` | No | | |
| `api_key` | `<class 'llama_stack.core.secret_types.MySecretStr'>` | No | | The API key for the Qdrant instance |
| `prefix` | `str \| None` | No | | |
| `timeout` | `int \| None` | No | | |
| `host` | `str \| None` | No | | |