Krish Dholakia
ea976d8c30
Merge pull request #3663 from msabramo/msabramo/allow-non-admins-to-use-openai-routes
...
Allow non-admins to use `/engines/{model}/chat/completions`
2024-05-16 12:17:50 -07:00
Ishaan Jaff
579bc09c37
fix - flush langfuse logs on proxy shutdown
2024-05-16 09:21:48 -07:00
Krish Dholakia
152946e910
Merge branch 'main' into litellm_end_user_cost_tracking
2024-05-15 22:27:04 -07:00
Krish Dholakia
57d425aed7
Merge pull request #3666 from BerriAI/litellm_jwt_fix
...
feat(proxy_server.py): JWT-Auth improvements
2024-05-15 22:22:44 -07:00
Krrish Dholakia
600b6f7e1d
feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth
2024-05-15 22:19:59 -07:00
Krrish Dholakia
f48cd87cf3
feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set)
...
Allows users to use jwt auth for internal chat apps
2024-05-15 21:05:14 -07:00
Ishaan Jaff
370049c549
Merge pull request #3661 from BerriAI/litellm_enable_alerts_completions_endpoint
...
[FIX] raise alerts for exceptions on `/completions` endpoint
2024-05-15 20:46:02 -07:00
Krish Dholakia
25e4b34574
Merge pull request #3660 from BerriAI/litellm_proxy_ui_general_settings
...
feat(proxy_server.py): Enabling Admin to control general settings on proxy ui
2024-05-15 20:36:42 -07:00
Krrish Dholakia
594ca947c8
fix(parallel_request_limiter.py): fix max parallel request limiter on retries
2024-05-15 20:16:11 -07:00
Krrish Dholakia
153ce0d085
build(ui/general_settings.tsx): support updating global max parallel requests on the ui
2024-05-15 19:26:57 -07:00
Marc Abramowitz
ddf77656fb
Allow non-admins to use OpenAI routes
...
The problem we were having is non-admin users trying to use
`/engines/{model}/chat/completions` were getting an HTTP 401 error.
```shell
$ curl -sSL 'http://localhost:4000/engines/gpt-35-turbo-0125/chat/completions ' \
--header "Authorization: Bearer ${LITELLM_KEY}" \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-35-turbo-0125",
"messages": [
{
"role": "user",
"content": "Write a poem about LiteLLM"
}
]
}' \
| jq '.'
{
"error": {
"message": "Authentication Error, Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/engines/gpt-35-turbo-0125/chat/completions. Your role=unknown. Your user_id=someone@company.com",
"type": "auth_error",
"param": "None",
"code": 401
}
}
```
This seems to be related to code in `user_api_key_auth` that checks that the URL
matches a list of routes that are allowed for non-admin users, where the list of
routes is in `LiteLLMRoutes.openai_routes.value`. The problem is that the route
`/engines/{model}/chat/completions` is not in that list and furthermore, that
wouldn't even work if it were, because the comparison is done with
`request.url.path` and that will have the actual model name in it (e.g.:
`gpt-35-turbo-0125`), rather than `{model}`.
I added a new list `LiteLLMRoutes.openai_route_names` and added the route
**names** to that list. Then I added a check in `user_api_key_auth` to see if
the route name is in the list of route names.
2024-05-15 17:36:11 -07:00
Ishaan Jaff
1262c5cb39
fix - /organization/new logic
2024-05-15 17:27:46 -07:00
Ishaan Jaff
9f43761000
fix - show alerts on /completions endpoint
2024-05-15 15:45:21 -07:00
Krrish Dholakia
6a357b4275
feat(proxy_server.py): Enabling Admin to control general settings on proxy ui
2024-05-15 15:26:57 -07:00
Ishaan Jaff
c00d232900
Merge pull request #3655 from BerriAI/litellm_only_show_debug_info_server_side
...
Fix Proxy Server - only show API base, Model server log exceptions, not on client side
2024-05-15 14:20:14 -07:00
Ishaan Jaff
1a11b151b7
fix - show litellm debug info in errors
2024-05-15 14:18:12 -07:00
Ishaan Jaff
2e81347607
fix - don't raise hanging request alert for /completions
2024-05-15 10:27:02 -07:00
Krrish Dholakia
5dcf3d672c
feat(proxy_server.py): new /end_user/info endpoint
...
get spend for a specific end-user
2024-05-15 09:50:52 -07:00
Krrish Dholakia
51a02de4cf
refactor(proxy_server.py): update doc string for /user/update
2024-05-15 08:25:14 -07:00
Ishaan Jaff
e7af8d61cd
fix check_request_disconnected = None case
2024-05-14 20:38:32 -07:00
Ishaan Jaff
4466982507
feat - cancel tasks when fast api request is cancelled
2024-05-14 19:58:51 -07:00
Ishaan Jaff
787c02c8db
fix - return success spend reset
2024-05-14 11:38:39 -07:00
Ishaan Jaff
7e56e27226
fix security for global_spend_reset
2024-05-14 11:04:50 -07:00
Ishaan Jaff
3686789c36
feat - reset spend per team, api_key
2024-05-13 18:23:23 -07:00
Ishaan Jaff
6ea5a276f8
backend - show spend per tag by time
2024-05-13 17:13:48 -07:00
Ishaan Jaff
12cf9d71c7
feat - /spend/report endpoint
2024-05-13 15:01:02 -07:00
Ishaan Jaff
9d4b727913
fix - show team based spend reports
2024-05-13 13:56:48 -07:00
Ishaan Jaff
21845bc061
Merge pull request #3609 from BerriAI/litellm_send_daily_spend_report
...
[Feat] send weekly spend reports by Team/Tag
2024-05-13 12:45:37 -07:00
Ishaan Jaff
40b2f33a80
fix - only schedule spend alerting when db is not none
2024-05-13 12:30:54 -07:00
Ishaan Jaff
aac81c59b5
test - weekly / monthly spend report alerts on /health/services
2024-05-13 10:50:26 -07:00
Ishaan Jaff
4a679bb640
schedule weekly/monthly spend reports
2024-05-13 10:44:19 -07:00
Ishaan Jaff
07247452c5
feat - show monthly spend reports
2024-05-13 10:10:44 -07:00
Ishaan Jaff
50f3677989
feat - _get_weekly_spend_reports
2024-05-13 09:26:51 -07:00
Krrish Dholakia
c3293474dd
fix(proxy_server.py): return 'allowed-model-region' in headers
2024-05-13 08:48:16 -07:00
Ishaan Jaff
514c5737f8
Merge pull request #3587 from BerriAI/litellm_proxy_use_batch_completions_model_csv
...
[Feat] Use csv values for proxy batch completions (OpenAI Python compatible)
2024-05-13 07:55:12 -07:00
Krrish Dholakia
b4684d5132
fix(proxy_server.py): linting fix
2024-05-11 22:05:01 -07:00
Ishaan Jaff
d4288b134b
fix - use csv list for batch completions
2024-05-11 14:24:48 -07:00
Ishaan Jaff
bf909a89f8
Merge pull request #3585 from BerriAI/litellm_router_batch_comp
...
[Litellm Proxy + litellm.Router] - Pass the same message/prompt to N models
2024-05-11 13:51:45 -07:00
Ishaan Jaff
b8c7bbcb9f
support batch /chat/completions on proxy
2024-05-11 13:24:25 -07:00
Krish Dholakia
bbe1300c5b
Merge branch 'main' into feat/add-azure-content-filter
2024-05-11 09:30:38 -07:00
Ishaan Jaff
933f8ed16b
fix - proxy_server.py
2024-05-10 13:47:35 -07:00
Nick Wong
d3a228d03b
added changes from upstream
...
Merge branch 'main' into fix/error-on-get-user-role
2024-05-09 16:14:14 -07:00
Nick Wong
c42f1ce2c6
removed extra default dict return, which causes error if user_role is a string
2024-05-09 16:13:26 -07:00
Krrish Dholakia
927d36148f
feat(proxy_server.py): expose new /team/list endpoint
...
Closes https://github.com/BerriAI/litellm/issues/3523
2024-05-09 13:21:00 -07:00
Krrish Dholakia
e3f25a4a1f
fix(auth_checks.py): fix 'get_end_user_object'
...
await cache get
2024-05-09 13:05:56 -07:00
Krish Dholakia
8ad979cdfe
Merge branch 'main' into litellm_region_based_routing
2024-05-08 22:19:51 -07:00
Krrish Dholakia
3d18897d69
feat(router.py): enable filtering model group by 'allowed_model_region'
2024-05-08 22:10:17 -07:00
Krrish Dholakia
db666b01e5
feat(proxy_server.py): add CRUD endpoints for 'end_user' management
...
allow admin to specify region + default models for end users
2024-05-08 18:50:36 -07:00
Ishaan Jaff
6d955ef457
Merge branch 'main' into ui_show_spend_end_user
2024-05-08 18:29:25 -07:00
Ishaan Jaff
a38d9e35fd
feat - get price by end_user
2024-05-08 18:19:27 -07:00
