forked from phoenix/litellm-mirror
docs(proxy_server.md): adding perf details
This commit is contained in:
parent
efe04cab5a
commit
ebc34ddeb6
1 changed files with 15 additions and 0 deletions
|
@ -633,6 +633,21 @@ You can view costs on the cli using
|
||||||
litellm --cost
|
litellm --cost
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Performance
|
||||||
|
|
||||||
|
We load-tested 500,000 HTTP connections on the FastAPI server for 1 minute, using [wrk](https://github.com/wg/wrk).
|
||||||
|
|
||||||
|
There are our results:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
Thread Stats Avg Stdev Max +/- Stdev
|
||||||
|
Latency 156.38ms 25.52ms 361.91ms 84.73%
|
||||||
|
Req/Sec 13.61 5.13 40.00 57.50%
|
||||||
|
383625 requests in 1.00m, 391.10MB read
|
||||||
|
Socket errors: connect 0, read 1632, write 1, timeout 0
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
## Support/ talk with founders
|
## Support/ talk with founders
|
||||||
|
|
||||||
- [Schedule Demo 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
|
- [Schedule Demo 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue