From ebc34ddeb68ad6856018f3ed875e1776a4a5dd68 Mon Sep 17 00:00:00 2001 From: Krrish Dholakia Date: Tue, 17 Oct 2023 14:33:36 -0700 Subject: [PATCH] docs(proxy_server.md): adding perf details --- docs/my-website/docs/proxy_server.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/my-website/docs/proxy_server.md b/docs/my-website/docs/proxy_server.md index 9599be9ab..a6a02994a 100644 --- a/docs/my-website/docs/proxy_server.md +++ b/docs/my-website/docs/proxy_server.md @@ -633,6 +633,21 @@ You can view costs on the cli using litellm --cost ``` +### Performance + +We load-tested 500,000 HTTP connections on the FastAPI server for 1 minute, using [wrk](https://github.com/wg/wrk). + +There are our results: + +```shell +Thread Stats Avg Stdev Max +/- Stdev + Latency 156.38ms 25.52ms 361.91ms 84.73% + Req/Sec 13.61 5.13 40.00 57.50% + 383625 requests in 1.00m, 391.10MB read + Socket errors: connect 0, read 1632, write 1, timeout 0 +``` + + ## Support/ talk with founders - [Schedule Demo 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)