Ishaan Jaff
22ba5fa186
feat - try using hf tokenizer
2024-05-16 10:59:29 -07:00
Krrish Dholakia
e273e66618
feat(lago.py): adding support for usage-based billing with lago
...
Closes https://github.com/BerriAI/litellm/issues/3639
2024-05-16 10:54:18 -07:00
Marc Abramowitz
cf71857354
Add more routes to test_generate_and_call_with_valid_key
2024-05-16 10:44:36 -07:00
Marc Abramowitz
dc52c83b88
Add more routes to test_generate_and_call_with_valid_key
2024-05-16 10:05:35 -07:00
Marc Abramowitz
c427ea3781
Add "/engines/{model}/chat/completions" to openai_routes
...
I don't think that this helps with the issue that I'm seeing, but I
think it might be nice to have this model listed in the openai_routes
list so that it's documented that it's a valid chat_completion route.
2024-05-16 10:03:23 -07:00
Ishaan Jaff
c646b809a6
fix token counter endpoint
2024-05-16 10:03:21 -07:00
Ishaan Jaff
b790d65d28
fix make token counter a /utils/token_counter
2024-05-16 10:00:34 -07:00
Ishaan Jaff
d42e5fcbd5
working token counter endpoint
2024-05-16 09:58:22 -07:00
Marc Abramowitz
d5b2e8e7e8
Make test_generate_and_call_with_valid_key parametrized
...
This allows us to test the same code with different routes.
For example, it lets us test the `/engines/{model}/chat/completions`
route, which https://github.com/BerriAI/litellm/pull/3663 fixes.
2024-05-16 09:54:10 -07:00
Ishaan Jaff
e50284bc72
dev - token_counter endpoint
2024-05-16 09:47:07 -07:00
Ishaan Jaff
5310706d1d
Merge pull request #3681 from BerriAI/litellm_fix_langfuse_flush_on_shutdown
...
[Fix] Flush langfuse logs on proxy shutdown
2024-05-16 09:45:33 -07:00
Ishaan Jaff
579bc09c37
fix - flush langfuse logs on proxy shutdown
2024-05-16 09:21:48 -07:00
Ishaan Jaff
4d97ca4500
Merge pull request #3679 from BerriAI/litellm_fix_show_better_client_side_error
...
[Fix]- router/proxy show better client side errors when `no_healthy deployments available`
2024-05-16 09:21:41 -07:00
Ishaan Jaff
848561a8a7
fix - router show better client side errors
2024-05-16 09:01:27 -07:00
Marc Abramowitz
b079f4cb79
model_info_v{1,2} (LiteLLM) => model_list (OpenAI)
2024-05-16 07:54:09 -07:00
Krish Dholakia
fda3914f4c
Merge pull request #3652 from BerriAI/litellm_end_user_cost_tracking
...
feat(proxy_server.py): new `/end_user/info` endpoint
2024-05-15 22:27:45 -07:00
Krish Dholakia
152946e910
Merge branch 'main' into litellm_end_user_cost_tracking
2024-05-15 22:27:04 -07:00
Krrish Dholakia
00b9f1290e
bump: version 1.37.11 → 1.37.12
2024-05-15 22:24:55 -07:00
Krish Dholakia
57d425aed7
Merge pull request #3666 from BerriAI/litellm_jwt_fix
...
feat(proxy_server.py): JWT-Auth improvements
2024-05-15 22:22:44 -07:00
Krrish Dholakia
600b6f7e1d
feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth
2024-05-15 22:19:59 -07:00
Krrish Dholakia
99653d2d3e
feat(handle_jwt.py): add support for 'team_id_default
...
allows admin to set a default team id for spend-tracking + permissions
2024-05-15 21:33:35 -07:00
Ishaan Jaff
bb86d2510f
(ci/cd) run again
2024-05-15 21:07:55 -07:00
Krrish Dholakia
f48cd87cf3
feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set)
...
Allows users to use jwt auth for internal chat apps
2024-05-15 21:05:14 -07:00
Ishaan Jaff
ac71a89b0e
ui - new build
2024-05-15 21:04:09 -07:00
Ishaan Jaff
775eb0ba99
bump: version 1.37.10 → 1.37.11
2024-05-15 21:01:02 -07:00
Ishaan Jaff
5300a4c9b3
fix model prices
2024-05-15 21:00:56 -07:00
Ishaan Jaff
881812d5de
Merge pull request #3543 from kmheckel/main
...
Updated Ollama cost models to include LLaMa3 and Mistral/Mixtral Instruct series
2024-05-15 20:50:50 -07:00
Ishaan Jaff
370049c549
Merge pull request #3661 from BerriAI/litellm_enable_alerts_completions_endpoint
...
[FIX] raise alerts for exceptions on `/completions` endpoint
2024-05-15 20:46:02 -07:00
Ishaan Jaff
7aac76b485
Merge pull request #3662 from BerriAI/litellm_feat_predibase_exceptions
...
[Fix] Mask API Keys from Predibase AuthenticationErrors
2024-05-15 20:45:40 -07:00
Krish Dholakia
25e4b34574
Merge pull request #3660 from BerriAI/litellm_proxy_ui_general_settings
...
feat(proxy_server.py): Enabling Admin to control general settings on proxy ui
2024-05-15 20:36:42 -07:00
Ishaan Jaff
e49fa9bd2c
(ci/cd) run again
2024-05-15 20:29:23 -07:00
Ishaan Jaff
b3d472f910
Merge pull request #3664 from BerriAI/litellm_revert_3600
...
[Fix] Revert #3600 https://github.com/BerriAI/litellm/pull/3600
2024-05-15 20:19:35 -07:00
Krrish Dholakia
594ca947c8
fix(parallel_request_limiter.py): fix max parallel request limiter on retries
2024-05-15 20:16:11 -07:00
Ishaan Jaff
c2a306c4dd
(ci/cd) run again
2024-05-15 20:03:30 -07:00
Ishaan Jaff
765d43d241
fix utils.py
2024-05-15 19:54:52 -07:00
Ishaan Jaff
1a67f244fb
fix types/router.py
2024-05-15 19:46:30 -07:00
Ishaan Jaff
86223bc703
revert changes to types/completion.py
2024-05-15 19:45:16 -07:00
Ishaan Jaff
136746abc9
fix test config
2024-05-15 19:42:39 -07:00
Ishaan Jaff
8e3aa14287
fix revert 3600
2024-05-15 19:40:34 -07:00
Krrish Dholakia
153ce0d085
build(ui/general_settings.tsx): support updating global max parallel requests on the ui
2024-05-15 19:26:57 -07:00
Ishaan Jaff
d208dedb35
(ci/cd) run again
2024-05-15 17:39:21 -07:00
Marc Abramowitz
ddf77656fb
Allow non-admins to use OpenAI routes
...
The problem we were having is non-admin users trying to use
`/engines/{model}/chat/completions` were getting an HTTP 401 error.
```shell
$ curl -sSL 'http://localhost:4000/engines/gpt-35-turbo-0125/chat/completions ' \
--header "Authorization: Bearer ${LITELLM_KEY}" \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-35-turbo-0125",
"messages": [
{
"role": "user",
"content": "Write a poem about LiteLLM"
}
]
}' \
| jq '.'
{
"error": {
"message": "Authentication Error, Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/engines/gpt-35-turbo-0125/chat/completions. Your role=unknown. Your user_id=someone@company.com",
"type": "auth_error",
"param": "None",
"code": 401
}
}
```
This seems to be related to code in `user_api_key_auth` that checks that the URL
matches a list of routes that are allowed for non-admin users, where the list of
routes is in `LiteLLMRoutes.openai_routes.value`. The problem is that the route
`/engines/{model}/chat/completions` is not in that list and furthermore, that
wouldn't even work if it were, because the comparison is done with
`request.url.path` and that will have the actual model name in it (e.g.:
`gpt-35-turbo-0125`), rather than `{model}`.
I added a new list `LiteLLMRoutes.openai_route_names` and added the route
**names** to that list. Then I added a check in `user_api_key_auth` to see if
the route name is in the list of route names.
2024-05-15 17:36:11 -07:00
Ishaan Jaff
89dd1ffee8
fix deep copy logic in during_call_hook
2024-05-15 17:34:54 -07:00
Ishaan Jaff
240b183d7a
ci/cd run again
2024-05-15 17:31:14 -07:00
Ishaan Jaff
1262c5cb39
fix - /organization/new logic
2024-05-15 17:27:46 -07:00
Ishaan Jaff
ed0a815c2b
test - exceptions predibase
2024-05-15 16:53:41 -07:00
Ishaan Jaff
b88ad61490
feat - predibase exceptions
2024-05-15 16:52:33 -07:00
Ishaan Jaff
f138c15859
(ci/cd) fix test_vertex_ai_stream
2024-05-15 16:32:40 -07:00
Ishaan Jaff
f2e8b2500f
fix function calling mistral large latest
2024-05-15 16:05:17 -07:00
Ishaan Jaff
9f43761000
fix - show alerts on /completions endpoint
2024-05-15 15:45:21 -07:00
