phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-22 22:42:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/docs/source/providers/inference/remote_vertexai.md
Francisco Javier Arceo ae3cbde8a4 docs: Add documentation on how to contribute a Vector DB provider and 
updated Test documentation and added a shortcut

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
2025-08-11 10:45:14 -04:00

1.4 KiB
Raw Blame History

remote::vertexai

Description

Google Vertex AI inference provider enables you to use Google's Gemini models through Google Cloud's Vertex AI platform, providing several advantages:

• Enterprise-grade security: Uses Google Cloud's security controls and IAM • Better integration: Seamless integration with other Google Cloud services • Advanced features: Access to additional Vertex AI features like model tuning and monitoring • Authentication: Uses Google Cloud Application Default Credentials (ADC) instead of API keys

Configuration:

  • Set VERTEX_AI_PROJECT environment variable (required)
  • Set VERTEX_AI_LOCATION environment variable (optional, defaults to us-central1)
  • Use Google Cloud Application Default Credentials or service account key

Authentication Setup: Option 1 (Recommended): gcloud auth application-default login Option 2: Set GOOGLE_APPLICATION_CREDENTIALS to service account key path

Available Models:

  • vertex_ai/gemini-2.0-flash
  • vertex_ai/gemini-2.5-flash
  • vertex_ai/gemini-2.5-pro

Configuration

Field Type Required Default Description
project <class 'str'> No Google Cloud project ID for Vertex AI
location <class 'str'> No us-central1 Google Cloud location for Vertex AI

Sample Configuration

project: ${env.VERTEX_AI_PROJECT:=}
location: ${env.VERTEX_AI_LOCATION:=us-central1}