Ishaan Jaff
22ba5fa186
feat - try using hf tokenizer
2024-05-16 10:59:29 -07:00
Krrish Dholakia
e273e66618
feat(lago.py): adding support for usage-based billing with lago
...
Closes https://github.com/BerriAI/litellm/issues/3639
2024-05-16 10:54:18 -07:00
Marc Abramowitz
c427ea3781
Add "/engines/{model}/chat/completions" to openai_routes
...
I don't think that this helps with the issue that I'm seeing, but I
think it might be nice to have this model listed in the openai_routes
list so that it's documented that it's a valid chat_completion route.
2024-05-16 10:03:23 -07:00
Ishaan Jaff
c646b809a6
fix token counter endpoint
2024-05-16 10:03:21 -07:00
Ishaan Jaff
b790d65d28
fix make token counter a /utils/token_counter
2024-05-16 10:00:34 -07:00
Ishaan Jaff
d42e5fcbd5
working token counter endpoint
2024-05-16 09:58:22 -07:00
Ishaan Jaff
e50284bc72
dev - token_counter endpoint
2024-05-16 09:47:07 -07:00
Ishaan Jaff
579bc09c37
fix - flush langfuse logs on proxy shutdown
2024-05-16 09:21:48 -07:00
Marc Abramowitz
b079f4cb79
model_info_v{1,2} (LiteLLM) => model_list (OpenAI)
2024-05-16 07:54:09 -07:00
Krish Dholakia
152946e910
Merge branch 'main' into litellm_end_user_cost_tracking
2024-05-15 22:27:04 -07:00
Krish Dholakia
57d425aed7
Merge pull request #3666 from BerriAI/litellm_jwt_fix
...
feat(proxy_server.py): JWT-Auth improvements
2024-05-15 22:22:44 -07:00
Krrish Dholakia
600b6f7e1d
feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth
2024-05-15 22:19:59 -07:00
Krrish Dholakia
99653d2d3e
feat(handle_jwt.py): add support for 'team_id_default
...
allows admin to set a default team id for spend-tracking + permissions
2024-05-15 21:33:35 -07:00
Krrish Dholakia
f48cd87cf3
feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set)
...
Allows users to use jwt auth for internal chat apps
2024-05-15 21:05:14 -07:00
Ishaan Jaff
ac71a89b0e
ui - new build
2024-05-15 21:04:09 -07:00
Ishaan Jaff
370049c549
Merge pull request #3661 from BerriAI/litellm_enable_alerts_completions_endpoint
...
[FIX] raise alerts for exceptions on `/completions` endpoint
2024-05-15 20:46:02 -07:00
Krish Dholakia
25e4b34574
Merge pull request #3660 from BerriAI/litellm_proxy_ui_general_settings
...
feat(proxy_server.py): Enabling Admin to control general settings on proxy ui
2024-05-15 20:36:42 -07:00
Krrish Dholakia
594ca947c8
fix(parallel_request_limiter.py): fix max parallel request limiter on retries
2024-05-15 20:16:11 -07:00
Ishaan Jaff
8e3aa14287
fix revert 3600
2024-05-15 19:40:34 -07:00
Krrish Dholakia
153ce0d085
build(ui/general_settings.tsx): support updating global max parallel requests on the ui
2024-05-15 19:26:57 -07:00
Marc Abramowitz
ddf77656fb
Allow non-admins to use OpenAI routes
...
The problem we were having is non-admin users trying to use
`/engines/{model}/chat/completions` were getting an HTTP 401 error.
```shell
$ curl -sSL 'http://localhost:4000/engines/gpt-35-turbo-0125/chat/completions ' \
--header "Authorization: Bearer ${LITELLM_KEY}" \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-35-turbo-0125",
"messages": [
{
"role": "user",
"content": "Write a poem about LiteLLM"
}
]
}' \
| jq '.'
{
"error": {
"message": "Authentication Error, Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/engines/gpt-35-turbo-0125/chat/completions. Your role=unknown. Your user_id=someone@company.com",
"type": "auth_error",
"param": "None",
"code": 401
}
}
```
This seems to be related to code in `user_api_key_auth` that checks that the URL
matches a list of routes that are allowed for non-admin users, where the list of
routes is in `LiteLLMRoutes.openai_routes.value`. The problem is that the route
`/engines/{model}/chat/completions` is not in that list and furthermore, that
wouldn't even work if it were, because the comparison is done with
`request.url.path` and that will have the actual model name in it (e.g.:
`gpt-35-turbo-0125`), rather than `{model}`.
I added a new list `LiteLLMRoutes.openai_route_names` and added the route
**names** to that list. Then I added a check in `user_api_key_auth` to see if
the route name is in the list of route names.
2024-05-15 17:36:11 -07:00
Ishaan Jaff
89dd1ffee8
fix deep copy logic in during_call_hook
2024-05-15 17:34:54 -07:00
Ishaan Jaff
1262c5cb39
fix - /organization/new logic
2024-05-15 17:27:46 -07:00
Ishaan Jaff
9f43761000
fix - show alerts on /completions endpoint
2024-05-15 15:45:21 -07:00
Krrish Dholakia
6a357b4275
feat(proxy_server.py): Enabling Admin to control general settings on proxy ui
2024-05-15 15:26:57 -07:00
Ishaan Jaff
c00d232900
Merge pull request #3655 from BerriAI/litellm_only_show_debug_info_server_side
...
Fix Proxy Server - only show API base, Model server log exceptions, not on client side
2024-05-15 14:20:14 -07:00
Ishaan Jaff
1a11b151b7
fix - show litellm debug info in errors
2024-05-15 14:18:12 -07:00
Ishaan Jaff
f6e342303e
show debug info on slack alerts
2024-05-15 13:23:44 -07:00
Ishaan Jaff
74f093bb4b
Merge pull request #3653 from BerriAI/litellm_fix_text_completions
...
[Fix] - Alerting on `/completions` - don't raise hanging request alert for /completions
2024-05-15 11:28:55 -07:00
Krrish Dholakia
dba713ea43
fix(router.py): add validation for how router fallbacks are setup
...
prevent user errors
2024-05-15 10:44:16 -07:00
Ishaan Jaff
2e81347607
fix - don't raise hanging request alert for /completions
2024-05-15 10:27:02 -07:00
Krrish Dholakia
5dcf3d672c
feat(proxy_server.py): new /end_user/info endpoint
...
get spend for a specific end-user
2024-05-15 09:50:52 -07:00
Krrish Dholakia
51a02de4cf
refactor(proxy_server.py): update doc string for /user/update
2024-05-15 08:25:14 -07:00
Krrish Dholakia
8117af664c
fix(huggingface_restapi.py): fix task extraction from model name
2024-05-15 07:28:19 -07:00
Ishaan Jaff
e7af8d61cd
fix check_request_disconnected = None case
2024-05-14 20:38:32 -07:00
Ishaan Jaff
4466982507
feat - cancel tasks when fast api request is cancelled
2024-05-14 19:58:51 -07:00
Krrish Dholakia
9eee2f3889
docs(prod.md): add 'disable load_dotenv' tutorial to docs
2024-05-14 19:13:22 -07:00
Krrish Dholakia
1ab4974773
fix: disable 'load_dotenv' for prod environments
2024-05-14 19:09:36 -07:00
Krrish Dholakia
1db1af1154
fix(types): fix typing
2024-05-14 17:09:36 -07:00
Krrish Dholakia
888c53e774
fix(proxy/_types.py): fix linting errors
2024-05-14 17:02:11 -07:00
Ishaan Jaff
aa1615c757
Merge pull request #3626 from BerriAI/litellm_reset_spend_per_team_api_key
...
feat - reset spend per team, api_key [Only Master Key]
2024-05-14 11:49:07 -07:00
Ishaan Jaff
787c02c8db
fix - return success spend reset
2024-05-14 11:38:39 -07:00
Ishaan Jaff
7e56e27226
fix security for global_spend_reset
2024-05-14 11:04:50 -07:00
Krish Dholakia
2c867ea9a5
Merge pull request #3600 from msabramo/msabramo/fix-pydantic-warnings
...
Update pydantic code to fix warnings
2024-05-13 22:00:39 -07:00
Ishaan Jaff
3686789c36
feat - reset spend per team, api_key
2024-05-13 18:23:23 -07:00
Ishaan Jaff
f19e6724ee
ui - new build
2024-05-13 17:48:24 -07:00
Ishaan Jaff
6ea5a276f8
backend - show spend per tag by time
2024-05-13 17:13:48 -07:00
Krrish Dholakia
c7b3193944
fix(proxy/_types.py): allow jwt admin to access /team/list route
2024-05-13 16:07:31 -07:00
Ishaan Jaff
ea9b4dc439
Merge pull request #3619 from BerriAI/litellm_show_spend_reports
...
[Feat] - `/global/spend/report`
2024-05-13 16:06:02 -07:00
Ishaan Jaff
eb2d6ba20a
ui - new build
2024-05-13 15:56:59 -07:00
