forked from phoenix/litellm-mirror
doc fix Using Http/2 with Hypercorn
This commit is contained in:
parent
4192d7ec6f
commit
86607a2018
1 changed files with 30 additions and 3 deletions
|
@ -688,8 +688,35 @@ Provide an ssl certificate when starting litellm proxy server
|
||||||
|
|
||||||
Use this if you want to run the proxy with hypercorn to support http/2
|
Use this if you want to run the proxy with hypercorn to support http/2
|
||||||
|
|
||||||
**Usage**
|
Step 1. Build your custom docker image with hypercorn
|
||||||
Pass the `--run_hypercorn` flag when starting the proxy
|
|
||||||
|
```shell
|
||||||
|
# Use the provided base image
|
||||||
|
FROM ghcr.io/berriai/litellm:main-latest
|
||||||
|
|
||||||
|
# Set the working directory to /app
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Copy the configuration file into the container at /app
|
||||||
|
COPY config.yaml .
|
||||||
|
|
||||||
|
# Make sure your docker/entrypoint.sh is executable
|
||||||
|
RUN chmod +x ./docker/entrypoint.sh
|
||||||
|
|
||||||
|
# Expose the necessary port
|
||||||
|
EXPOSE 4000/tcp
|
||||||
|
|
||||||
|
# 👉 Key Change: Install hypercorn
|
||||||
|
RUN pip install hypercorn
|
||||||
|
|
||||||
|
# Override the CMD instruction with your desired command and arguments
|
||||||
|
# WARNING: FOR PROD DO NOT USE `--detailed_debug` it slows down response times, instead use the following CMD
|
||||||
|
# CMD ["--port", "4000", "--config", "config.yaml"]
|
||||||
|
|
||||||
|
CMD ["--port", "4000", "--config", "config.yaml", "--detailed_debug"]
|
||||||
|
```
|
||||||
|
|
||||||
|
Step 2. Pass the `--run_hypercorn` flag when starting the proxy
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
docker run \
|
docker run \
|
||||||
|
@ -699,7 +726,7 @@ docker run \
|
||||||
-e SERVER_ROOT_PATH="/api/v1"\
|
-e SERVER_ROOT_PATH="/api/v1"\
|
||||||
-e DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname> \
|
-e DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname> \
|
||||||
-e LITELLM_MASTER_KEY="sk-1234"\
|
-e LITELLM_MASTER_KEY="sk-1234"\
|
||||||
ghcr.io/berriai/litellm:main-latest \
|
your_custom_docker_image \
|
||||||
--config /app/config.yaml
|
--config /app/config.yaml
|
||||||
--run_hypercorn
|
--run_hypercorn
|
||||||
```
|
```
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue