mirror of https://github.com/meta-llama/llama-stack.git synced 2025-06-28 19:04:19 +00:00

feat: Adding TLS support for Remote::Milvus vector_io (#2011 )

# What does this PR do?
For the Issue :-
#[2010](https://github.com/meta-llama/llama-stack/issues/2010)
Currently, if we try to connect the Llama stack server to a remote
Milvus instance that has TLS enabled, the connection fails because TLS
support is not implemented in the Llama stack codebase. As a result,
users are unable to use secured Milvus deployments out of the box.

After adding this , the user will be able to connect to remote::Milvus
which is TLS enabled .
if TLS enabled :-
```
vector_io:
  - provider_id: milvus
    provider_type: remote::milvus
    config:
      uri: "http://<host>:<port>"
      token: "<user>:<password>"
      secure: True
      server_pem_path: "path/to/server.pem"
```
[//]: # (If resolving an issue, uncomment and update the line below)
[//]: # (Closes #[issue-number])

## Test Plan
I have already tested it by connecting to a Milvus instance which is TLS
enabled and i was able to start llama stack server .

2025-05-06 14:15:34 +02:00

2.9 KiB

Raw Blame History

orphan
true

Milvus

Milvus is an inline and remote vector database provider for Llama Stack. It allows you to store and query vectors directly within a Milvus database. That means you're not limited to storing vectors in memory or in a separate service.

Features

Easy to use
Fully integrated with Llama Stack

Usage

To use Milvus in your Llama Stack project, follow these steps:

Install the necessary dependencies.
Configure your Llama Stack project to use Milvus.
Start storing and querying vectors.

Installation

You can install Milvus using pymilvus:

pip install pymilvus

Configuration

In Llama Stack, Milvus can be configured in two ways:

Inline (Local) Configuration - Uses Milvus-Lite for local storage
Remote Configuration - Connects to a remote Milvus server

Inline (Local) Configuration

The simplest method is local configuration, which requires setting db_path, a path for locally storing Milvus-Lite files:

vector_io:
  - provider_id: milvus
    provider_type: inline::milvus
    config:
      db_path: ~/.llama/distributions/together/milvus_store.db

Remote Configuration

Remote configuration is suitable for larger data storage requirements:

Standard Remote Connection

vector_io:
  - provider_id: milvus
    provider_type: remote::milvus
    config:
      uri: "http://<host>:<port>"
      token: "<user>:<password>"

TLS-Enabled Remote Connection (One-way TLS)

For connections to Milvus instances with one-way TLS enabled:

vector_io:
  - provider_id: milvus
    provider_type: remote::milvus
    config:
      uri: "https://<host>:<port>"
      token: "<user>:<password>"
      secure: True
      server_pem_path: "/path/to/server.pem"

Mutual TLS (mTLS) Remote Connection

For connections to Milvus instances with mutual TLS (mTLS) enabled:

vector_io:
  - provider_id: milvus
    provider_type: remote::milvus
    config:
      uri: "https://<host>:<port>"
      token: "<user>:<password>"
      secure: True
      ca_pem_path: "/path/to/ca.pem"
      client_pem_path: "/path/to/client.pem"
      client_key_path: "/path/to/client.key"

Key Parameters for TLS Configuration

secure: Enables TLS encryption when set to true. Defaults to false.
server_pem_path: Path to the server certificate for verifying the server’s identity (used in one-way TLS).
ca_pem_path: Path to the Certificate Authority (CA) certificate for validating the server certificate (required in mTLS).
client_pem_path: Path to the client certificate file (required for mTLS).
client_key_path: Path to the client private key file (required for mTLS).

Documentation

See the Milvus documentation for more details about Milvus in general.

For more details on TLS configuration, refer to the TLS setup guide.

2.9 KiB Raw Blame History Unescape Escape