llama-stack-mirror/docs/source/providers/inference
Eran Cohen a4bad6c0b4
Some checks failed
Integration Tests (Replay) / discover-tests (push) Successful in 9s
Python Package Build Test / build (3.12) (push) Failing after 4s
Vector IO Integration Tests / test-matrix (3.12, inline::milvus) (push) Failing after 10s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 12s
Python Package Build Test / build (3.13) (push) Failing after 4s
Vector IO Integration Tests / test-matrix (3.12, remote::chromadb) (push) Failing after 10s
Test Llama Stack Build / generate-matrix (push) Successful in 8s
Test Llama Stack Build / build-custom-container-distribution (push) Failing after 13s
Test External API and Providers / test-external (venv) (push) Failing after 11s
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 17s
Test Llama Stack Build / build-ubi9-container-distribution (push) Failing after 10s
Test Llama Stack Build / build-single-provider (push) Failing after 16s
Vector IO Integration Tests / test-matrix (3.13, inline::faiss) (push) Failing after 8s
Unit Tests / unit-tests (3.12) (push) Failing after 10s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 26s
Vector IO Integration Tests / test-matrix (3.12, remote::qdrant) (push) Failing after 15s
Update ReadTheDocs / update-readthedocs (push) Failing after 9s
Integration Tests (Replay) / Integration Tests (, , , client=, vision=) (push) Failing after 7s
Vector IO Integration Tests / test-matrix (3.13, remote::weaviate) (push) Failing after 11s
Vector IO Integration Tests / test-matrix (3.12, remote::pgvector) (push) Failing after 23s
Vector IO Integration Tests / test-matrix (3.13, remote::pgvector) (push) Failing after 16s
Vector IO Integration Tests / test-matrix (3.12, remote::weaviate) (push) Failing after 18s
Test Llama Stack Build / build (push) Failing after 8s
Vector IO Integration Tests / test-matrix (3.13, remote::qdrant) (push) Failing after 17s
Vector IO Integration Tests / test-matrix (3.12, inline::faiss) (push) Failing after 16s
Vector IO Integration Tests / test-matrix (3.13, inline::sqlite-vec) (push) Failing after 8s
Vector IO Integration Tests / test-matrix (3.13, remote::chromadb) (push) Failing after 21s
Vector IO Integration Tests / test-matrix (3.13, inline::milvus) (push) Failing after 47s
Vector IO Integration Tests / test-matrix (3.12, inline::sqlite-vec) (push) Failing after 49s
Unit Tests / unit-tests (3.13) (push) Failing after 39s
Pre-commit / pre-commit (push) Successful in 1m37s
feat: Add Google Vertex AI inference provider support (#2841)
# What does this PR do?
- Add new Vertex AI remote inference provider with litellm integration
- Support for Gemini models through Google Cloud Vertex AI platform
- Uses Google Cloud Application Default Credentials (ADC) for
authentication
- Added VertexAI models: gemini-2.5-flash, gemini-2.5-pro,
gemini-2.0-flash.
- Updated provider registry to include vertexai provider
- Updated starter template to support Vertex AI configuration
- Added comprehensive documentation and sample configuration

<!-- If resolving an issue, uncomment and update the line below -->
relates to https://github.com/meta-llama/llama-stack/issues/2747

## Test Plan
<!-- Describe the tests you ran to verify your changes with result
summaries. *Provide clear instructions so the plan can be easily
re-executed.* -->

Signed-off-by: Eran Cohen <eranco@redhat.com>
Co-authored-by: Francisco Arceo <arceofrancisco@gmail.com>
2025-08-11 08:22:04 -04:00
..
index.md feat: Add Google Vertex AI inference provider support (#2841) 2025-08-11 08:22:04 -04:00
inline_meta-reference.md docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
inline_sentence-transformers.md docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
remote_anthropic.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_bedrock.md docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
remote_cerebras.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_databricks.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_fireworks.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_gemini.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_groq.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_hf_endpoint.md chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
remote_hf_serverless.md chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
remote_llama-openai-compat.md docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
remote_nvidia.md fix: allow default empty vars for conditionals (#2570) 2025-07-01 14:42:05 +02:00
remote_ollama.md feat(registry): make the Stack query providers for model listing (#2862) 2025-07-24 10:39:53 -07:00
remote_openai.md feat(openai): add configurable base_url support with OPENAI_BASE_URL env var (#2919) 2025-07-28 10:16:02 -07:00
remote_passthrough.md docs: auto generated documentation for providers (#2543) 2025-06-30 15:13:20 +02:00
remote_runpod.md feat: consolidate most distros into "starter" (#2516) 2025-07-04 15:58:03 +02:00
remote_sambanova-openai-compat.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_sambanova.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_tgi.md chore(misc): make tests and starter faster (#3042) 2025-08-05 14:55:05 -07:00
remote_together.md feat(starter)!: simplify starter distro; litellm model registry changes (#2916) 2025-07-25 15:02:04 -07:00
remote_vertexai.md feat: Add Google Vertex AI inference provider support (#2841) 2025-08-11 08:22:04 -04:00
remote_vllm.md feat(registry): make the Stack query providers for model listing (#2862) 2025-07-24 10:39:53 -07:00
remote_watsonx.md fix: allow default empty vars for conditionals (#2570) 2025-07-01 14:42:05 +02:00