phoenix/litellm
1
0
Fork 0
forked from phoenix/litellm-mirror
Code Pull requests Activity
18448 commits 1603 branches 645 tags 320 MiB
Commit graph

1844 commits

Author SHA1 Message Date
Krrish Dholakia
b41f30ca60 fix(proxy_server.py): fixes for making rejected responses work with streaming 2024-05-20 12:32:19 -07:00
Ishaan Jaff
b60a098251 docs - update openapi swagger docs 2024-05-20 12:26:42 -07:00
Krrish Dholakia
f11f207ae6 feat(proxy_server.py): refactor returning rejected message, to work with error logging 
log the rejected request as a failed call to langfuse/slack alerting
 2024-05-20 11:14:36 -07:00
Krrish Dholakia
372323c38a feat(proxy_server.py): allow admin to return rejected response as string to user 
Closes https://github.com/BerriAI/litellm/issues/3671
 2024-05-20 10:30:23 -07:00
Krrish Dholakia
25df95ab10 feat(proxy_server.py): new 'supported_openai_params' endpoint 
get supported openai params for a given model
 2024-05-20 08:39:50 -07:00
Krish Dholakia
5e5179e476
Merge branch 'main' into litellm_model_id_fix 2024-05-17 22:36:17 -07:00
Ishaan Jaff
8281c150f0
Merge pull request #3713 from BerriAI/litellm_ui_infer_azure_prefix 
[Feat] Admin UI - use `base_model` for Slack Alerts
 2024-05-17 21:55:23 -07:00
Krrish Dholakia
4b3551abfc fix(slack_alerting.py): show langfuse traces on error messages 2024-05-17 18:42:30 -07:00
Krish Dholakia
3a06fe2818
Merge branch 'main' into litellm_bedrock_anthropic_fix 2024-05-17 17:47:32 -07:00
Krrish Dholakia
b137cea230 fix(proxy_server.py): fix setting model id for db models 
get model_id and use that as it's id in router, this enables `/model/delete` to work with the given id from `/model/info`
 2024-05-17 17:45:05 -07:00
Ishaan Jaff
be273b3c3b fix - show correct base_model in slack alerts 2024-05-17 16:07:02 -07:00
Krrish Dholakia
c0d62e94ae feat(proxy_server.py): enable custom branding + routes on openapi docs 
Allows user to add their branding + show only openai routes on docs
 2024-05-17 15:21:29 -07:00
Krrish Dholakia
180bc46ca4 fix(bedrock_httpx.py): move anthropic bedrock calls to httpx 
Fixing https://github.com/BerriAI/litellm/issues/2921
 2024-05-16 21:51:55 -07:00
Krrish Dholakia
10a672634d fix(proxy_server.py): fix invalid header string 2024-05-16 21:05:40 -07:00
Krish Dholakia
7502e15295
Merge pull request #3701 from paneru-rajan/Issue-3675-remove-empty-valued-header 
Exclude custom headers from response if the value is None or empty string
 2024-05-16 17:42:07 -07:00
Rajan Paneru
e4ce10038a use default empty str if the allowed_model_region attribute is not present 2024-05-17 10:05:18 +09:30
Rajan Paneru
54f8d06057 handle exception and logged it 2024-05-17 09:55:13 +09:30
Ishaan Jaff
a292583ff1 fix - allow users to opt into specific alert types 2024-05-16 16:52:44 -07:00
Rajan Paneru
85679470c2 Exclude custom headers from response if the value is None or empty string 
This will return clean header, sending a header with empty value is not standard which
is being avoided from this fix.
 2024-05-17 09:06:58 +09:30
Krrish Dholakia
48714805bd fix(proxy_server.py): fix code 2024-05-16 15:02:39 -07:00
Krish Dholakia
0a775821db
Merge branch 'main' into litellm_end_user_obj 2024-05-16 14:16:09 -07:00
Ishaan Jaff
0a816b2c45
Merge pull request #3682 from BerriAI/litellm_token_counter_endpoint 
[Feat] `token_counter` endpoint
 2024-05-16 13:39:23 -07:00
Ishaan Jaff
4a5e6aa43c test - token count response 2024-05-16 13:20:01 -07:00
Krish Dholakia
d43f75150a
Merge pull request #3685 from BerriAI/litellm_lago_integration 
feat(lago.py): Enable Usage-based billing with lago
 2024-05-16 13:09:48 -07:00
Ishaan Jaff
d16a6c03a2 feat - include model name in cool down alerts 2024-05-16 12:52:15 -07:00
Ishaan Jaff
8c3657bad0
Merge pull request #3686 from msabramo/msabramo/fix-datetime-utcnow-deprecation-warnings 
Fix `datetime.datetime.utcnow` `DeprecationWarning`
 2024-05-16 12:19:06 -07:00
Krish Dholakia
ea976d8c30
Merge pull request #3663 from msabramo/msabramo/allow-non-admins-to-use-openai-routes 
Allow non-admins to use `/engines/{model}/chat/completions`
 2024-05-16 12:17:50 -07:00
Marc Abramowitz
4af6638be6 Fix datetime.datetime.utcnow DeprecationWarning 
Eliminates these warning when running tests:

```
$ cd litellm/tests
pytest test_key_generate_prisma.py -x -vv
...
====================================================================== warnings summary =======================================================================
...
test_key_generate_prisma.py::test_generate_and_call_with_expired_key
test_key_generate_prisma.py::test_key_with_no_permissions
  /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/proxy_server.py:2934: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    expires = datetime.utcnow() + timedelta(seconds=duration_s)
...
```
 2024-05-16 11:56:02 -07:00
Ishaan Jaff
22ba5fa186 feat - try using hf tokenizer 2024-05-16 10:59:29 -07:00
Krrish Dholakia
e273e66618 feat(lago.py): adding support for usage-based billing with lago 
Closes https://github.com/BerriAI/litellm/issues/3639
 2024-05-16 10:54:18 -07:00
Ishaan Jaff
c646b809a6 fix token counter endpoint 2024-05-16 10:03:21 -07:00
Ishaan Jaff
b790d65d28 fix make token counter a /utils/token_counter 2024-05-16 10:00:34 -07:00
Ishaan Jaff
d42e5fcbd5 working token counter endpoint 2024-05-16 09:58:22 -07:00
Ishaan Jaff
e50284bc72 dev - token_counter endpoint 2024-05-16 09:47:07 -07:00
Ishaan Jaff
579bc09c37 fix - flush langfuse logs on proxy shutdown 2024-05-16 09:21:48 -07:00
Krish Dholakia
152946e910
Merge branch 'main' into litellm_end_user_cost_tracking 2024-05-15 22:27:04 -07:00
Krish Dholakia
57d425aed7
Merge pull request #3666 from BerriAI/litellm_jwt_fix 
feat(proxy_server.py): JWT-Auth improvements
 2024-05-15 22:22:44 -07:00
Krrish Dholakia
600b6f7e1d feat(proxy_server.py): support 'user_id_upsert' flag for jwt_auth 2024-05-15 22:19:59 -07:00
Krrish Dholakia
f48cd87cf3 feat(proxy_server.py): make team_id optional for jwt token auth (only enforced, if set) 
Allows users to use jwt auth for internal chat apps
 2024-05-15 21:05:14 -07:00
Ishaan Jaff
370049c549
Merge pull request #3661 from BerriAI/litellm_enable_alerts_completions_endpoint 
[FIX]  raise alerts for exceptions on `/completions` endpoint
 2024-05-15 20:46:02 -07:00
Krish Dholakia
25e4b34574
Merge pull request #3660 from BerriAI/litellm_proxy_ui_general_settings 
feat(proxy_server.py): Enabling Admin to control general settings on proxy ui
 2024-05-15 20:36:42 -07:00
Krrish Dholakia
594ca947c8 fix(parallel_request_limiter.py): fix max parallel request limiter on retries 2024-05-15 20:16:11 -07:00
Krrish Dholakia
153ce0d085 build(ui/general_settings.tsx): support updating global max parallel requests on the ui 2024-05-15 19:26:57 -07:00
Marc Abramowitz
ddf77656fb Allow non-admins to use OpenAI routes 
The problem we were having is non-admin users trying to use
`/engines/{model}/chat/completions` were getting an HTTP 401 error.

```shell
$ curl -sSL 'http://localhost:4000/engines/gpt-35-turbo-0125/chat/completions' \
    --header "Authorization: Bearer ${LITELLM_KEY}" \
    --header 'Content-Type: application/json' \
    --data ' {
        "model": "gpt-35-turbo-0125",
        "messages": [
            {
                "role": "user",
                "content": "Write a poem about LiteLLM"
            }
        ]
    }' \
    | jq '.'
{
  "error": {
    "message": "Authentication Error, Only proxy admin can be used to generate, delete, update info for new keys/users/teams. Route=/engines/gpt-35-turbo-0125/chat/completions. Your role=unknown. Your user_id=someone@company.com",
    "type": "auth_error",
    "param": "None",
    "code": 401
  }
}
```

This seems to be related to code in `user_api_key_auth` that checks that the URL
matches a list of routes that are allowed for non-admin users, where the list of
routes is in `LiteLLMRoutes.openai_routes.value`. The problem is that the route
`/engines/{model}/chat/completions` is not in that list and furthermore, that
wouldn't even work if it were, because the comparison is done with
`request.url.path` and that will have the actual model name in it (e.g.:
`gpt-35-turbo-0125`), rather than `{model}`.

I added a new list `LiteLLMRoutes.openai_route_names` and added the route
**names** to that list. Then I added a check in `user_api_key_auth` to see if
the route name is in the list of route names.
 2024-05-15 17:36:11 -07:00
Ishaan Jaff
1262c5cb39 fix - /organization/new logic 2024-05-15 17:27:46 -07:00
Ishaan Jaff
9f43761000 fix - show alerts on /completions endpoint 2024-05-15 15:45:21 -07:00
Krrish Dholakia
6a357b4275 feat(proxy_server.py): Enabling Admin to control general settings on proxy ui 2024-05-15 15:26:57 -07:00
Ishaan Jaff
c00d232900
Merge pull request #3655 from BerriAI/litellm_only_show_debug_info_server_side 
Fix Proxy Server - only show API base, Model server log exceptions, not on client side
 2024-05-15 14:20:14 -07:00
Ishaan Jaff
1a11b151b7 fix - show litellm debug info in errors 2024-05-15 14:18:12 -07:00
Ishaan Jaff
2e81347607 fix - don't raise hanging request alert for /completions 2024-05-15 10:27:02 -07:00