phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-23 01:22:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: Add Google Vertex AI inference provider support

Browse source 
- Add new Vertex AI remote inference provider with litellm integration
- Support for Gemini models through Google Cloud Vertex AI platform
- Uses Google Cloud Application Default Credentials (ADC) for authentication
- Added VertexAI models: gemini-2.5-flash, gemini-2.5-pro, gemini-2.0-flash.
- Updated provider registry to include vertexai provider
- Add vertexai to INFERENCE_PROVIDER_IDS in starter distribution template
- Update VertexAI provider to be conditionally included in starter template when VERTEX_AI_PROJECT env var is set
- Added comprehensive documentation and sample configuration

Signed-off-by: Eran Cohen <eranco@redhat.com>
This commit is contained in:
Eran Cohen 2025-07-22 15:27:33 +03:00
parent 1677d6bffd
commit 4c1f715171
14 changed files with 227 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
llama_stack/distributions/starter/build.yaml
View file

@ -14,6 +14,7 @@ distribution_spec:
- provider_type: remote::openai
- provider_type: remote::anthropic
- provider_type: remote::gemini
- provider_type: remote::vertexai
- provider_type: remote::groq
- provider_type: remote::sambanova
- provider_type: inline::sentence-transformers