forked from phoenix/litellm-mirror
(docs) litellm cloud formation stack
This commit is contained in:
parent
4a4d36d923
commit
b4e12fb8fd
1 changed files with 42 additions and 1 deletions
|
@ -218,8 +218,49 @@ Provide an ssl certificate when starting litellm proxy server
|
|||
|
||||
## Platform-specific Guide
|
||||
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="aws-stack" label="AWS Cloud Formation Stack">
|
||||
|
||||
### AWS Cloud Formation Stack
|
||||
LiteLLM AWS Cloudformation Stack - **Get the best LiteLLM AutoScaling Policy and Provision the DB for LiteLLM Proxy**
|
||||
|
||||
This will provision:
|
||||
- LiteLLMServer - EC2 Instance
|
||||
- LiteLLMServerAutoScalingGroup
|
||||
- LiteLLMServerScalingPolicy (autoscaling policy)
|
||||
- LiteLLMDB - RDS::DBInstance
|
||||
|
||||
#### Using AWS Cloud Formation Stack
|
||||
**LiteLLM Cloudformation stack is located [here - litellm.yaml](https://github.com/BerriAI/litellm/blob/main/enterprise/cloudformation_stack/litellm.yaml)**
|
||||
|
||||
#### 1. Create the CloudFormation Stack:
|
||||
In the AWS Management Console, navigate to the CloudFormation service, and click on "Create Stack."
|
||||
|
||||
On the "Create Stack" page, select "Upload a template file" and choose the litellm.yaml file
|
||||
|
||||
Now monitor the stack was created successfully.
|
||||
|
||||
#### 2. Get the Database URL:
|
||||
Once the stack is created, get the DatabaseURL of the Database resource, copy this value
|
||||
|
||||
#### 3. Connect to the EC2 Instance and deploy litellm on the EC2 container
|
||||
From the EC2 console, connect to the instance created by the stack (e.g., using SSH).
|
||||
|
||||
Run the following command, replacing <database_url> with the value you copied in step 2
|
||||
|
||||
```shell
|
||||
docker run --name litellm-proxy \
|
||||
-e DATABASE_URL=<database_url> \
|
||||
-p 4000:4000 \
|
||||
ghcr.io/berriai/litellm-database:main-latest
|
||||
```
|
||||
|
||||
#### 4. Access the Application:
|
||||
|
||||
Once the container is running, you can access the application by going to `http://<ec2-public-ip>:4000` in your browser.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="google-cloud-run" label="Google Cloud Run">
|
||||
|
||||
### Deploy on Google Cloud Run
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue