phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity

docs - show correct rpm - > tpm conversion for Azure

Browse source
This commit is contained in:
Ishaan Jaff 2024-09-27 17:18:55 -07:00
parent 70df474e64
commit bbf4db79c1
2 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/my-website/docs/routing.md
View file

@ -167,7 +167,7 @@ Routes to **deployment with lowest TPM usage** for that minute.
In production, we use Redis to track usage (TPM/RPM) across multiple deployments. This implementation uses **async redis calls** (redis.incr and redis.mget). In production, we use Redis to track usage (TPM/RPM) across multiple deployments. This implementation uses **async redis calls** (redis.incr and redis.mget).
For Azure, your RPM = TPM/6. For Azure, [you get 6 RPM per 1000 TPM](https://stackoverflow.com/questions/77368844/what-is-the-request-per-minute-rate-limit-for-azure-openai-models-for-gpt-3-5-tu)
<Tabs> <Tabs>
<TabItem value="sdk" label="sdk"> <TabItem value="sdk" label="sdk">

2
docs/my-website/sidebars.js
View file

@ -228,12 +228,12 @@ const sidebars = {
}, },
], ],
}, },
"routing",
"scheduler", "scheduler",
{ {
type: "category", type: "category",
label: "🚅 LiteLLM Python SDK", label: "🚅 LiteLLM Python SDK",
items: [ items: [
"routing",
"set_keys", "set_keys",
"completion/token_usage", "completion/token_usage",
"sdk_custom_pricing", "sdk_custom_pricing",