diff --git a/docs/my-website/docs/proxy/users.md b/docs/my-website/docs/proxy/users.md index 6d9c43c5f..866c0c4bb 100644 --- a/docs/my-website/docs/proxy/users.md +++ b/docs/my-website/docs/proxy/users.md @@ -374,6 +374,62 @@ curl --location 'http://0.0.0.0:4000/key/generate' \ } ``` + + + +Use this to set rate limits for `user` passed to `/chat/completions`, without needing to create a key for every user + +#### Step 1. Create Budget + +Set a `tpm_limit` on the budget (You can also pass `rpm_limit` if needed) + +```shell +curl --location 'http://0.0.0.0:4000/budget/new' \ +--header 'Authorization: Bearer sk-1234' \ +--header 'Content-Type: application/json' \ +--data '{ + "budget_id" : "free-tier", + "tpm_limit": 5 +}' +``` + + +#### Step 2. Create `End-User` with Budget + +We use `budget_id="free-tier"` from Step 1 when creating this new end user + +```shell +curl --location 'http://0.0.0.0:4000/end_user/new' \ +--header 'Authorization: Bearer sk-1234' \ +--header 'Content-Type: application/json' \ +--data '{ + "user_id" : "palantir", + "budget_id": "free-tier" +}' +``` + + +#### Step 3. Pass end user id in `/chat/completions` requests + +Pass the `user_id` from Step 2 as `user="palantir"` + +```shell +curl --location 'http://localhost:4000/chat/completions' \ + --header 'Authorization: Bearer sk-1234' \ + --header 'Content-Type: application/json' \ + --data '{ + "model": "llama3", + "user": "palantir", + "messages": [ + { + "role": "user", + "content": "gm" + } + ] +}' +``` + +