Commit graph

39 commits

Author SHA1 Message Date
Krish Dholakia
2cab33b061 LiteLLM Minor Fixes and Improvements (08/06/2024) (#5567)
* fix(utils.py): return citations for perplexity streaming

Fixes https://github.com/BerriAI/litellm/issues/5535

* fix(anthropic/chat.py): support fallbacks for anthropic streaming (#5542)

* fix(anthropic/chat.py): support fallbacks for anthropic streaming

Fixes https://github.com/BerriAI/litellm/issues/5512

* fix(anthropic/chat.py): use module level http client if none given (prevents early client closure)

* fix: fix linting errors

* fix(http_handler.py): fix raise_for_status error handling

* test: retry flaky test

* fix otel type

* fix(bedrock/embed): fix error raising

* test(test_openai_batches_and_files.py): skip azure batches test (for now) quota exceeded

* fix(test_router.py): skip azure batch route test (for now) - hit batch quota limits

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* All `model_group_alias` should show up in `/models`, `/model/info` , `/model_group/info` (#5539)

* fix(router.py): support returning model_alias model names in `/v1/models`

* fix(proxy_server.py): support returning model alias'es on `/model/info`

* feat(router.py): support returning model group alias for `/model_group/info`

* fix(proxy_server.py): fix linting errors

* fix(proxy_server.py): fix linting errors

* build(model_prices_and_context_window.json): add amazon titan text premier pricing information

Closes https://github.com/BerriAI/litellm/issues/5560

* feat(litellm_logging.py): log standard logging response object for pass through endpoints. Allows bedrock /invoke agent calls to be correctly logged to langfuse + s3

* fix(success_handler.py): fix linting error

* fix(success_handler.py): fix linting errors

* fix(team_endpoints.py): Allows admin to update team member budgets

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2024-09-06 17:16:24 -07:00
Ishaan Jaff
784ceaad0d rename type 2024-09-04 16:33:36 -07:00
Ishaan Jaff
b336977ff6 add doc on PassthroughStandardLoggingObject 2024-09-04 16:30:47 -07:00
Ishaan Jaff
5e121660d5 feat log request / response on pass through endpoints 2024-09-04 16:26:32 -07:00
Ishaan Jaff
7f8b43542b fix success handler typing 2024-09-02 19:42:36 -07:00
Ishaan Jaff
778cba702e fix linting errors 2024-09-02 19:39:10 -07:00
Ishaan Jaff
de3fab70bd fix linting error 2024-09-02 18:14:15 -07:00
Ishaan Jaff
dc042d1a00 add cost tracking for pass through imagen 2024-09-02 18:10:46 -07:00
Ishaan Jaff
54fbea1a82 track image gen in spend logs 2024-09-02 17:36:25 -07:00
Ishaan Jaff
a9c9967b6d fix lining 2024-09-02 17:08:30 -07:00
Ishaan Jaff
e60c7a3b85 track /embedding in spendLogs 2024-09-02 17:05:53 -07:00
Ishaan Jaff
5876d043b4 code cleanup 2024-09-02 16:36:19 -07:00
Ishaan Jaff
3f9c58507e pass through track usage for streaming endpoints 2024-09-02 16:11:20 -07:00
Ishaan Jaff
ef6b90a657 use chunk_processort 2024-09-02 15:51:52 -07:00
Ishaan Jaff
fbeb6941f1 new streaming handler fn 2024-09-02 15:51:21 -07:00
Ishaan Jaff
dd2aaf33fa use helper class for pass through success handler 2024-08-30 15:52:47 -07:00
Ishaan Jaff
c20cb7b586 fix pass through endpoints 2024-08-21 17:21:22 -07:00
Krrish Dholakia
3713101904 fix(pass_through_endpoints.py): fix query param pass through 2024-08-19 21:38:30 -07:00
Krrish Dholakia
c5d1899940 feat(Support-pass-through-for-bedrock-endpoints): Allows pass-through support for bedrock endpoints 2024-08-17 17:57:43 -07:00
Krrish Dholakia
1856ac585d feat(pass_through_endpoints.py): add pass-through support for all cohere endpoints 2024-08-17 16:57:55 -07:00
Krrish Dholakia
5dc52aedc9 style(vertex_httpx.py): make vertex error string more helpful 2024-08-17 15:09:55 -07:00
Krrish Dholakia
b2ffa564d1 feat(pass_through_endpoints.py): support streaming requests 2024-08-17 12:46:57 -07:00
Krrish Dholakia
29bedae79f feat(google_ai_studio_endpoints.py): support pass-through endpoint for all google ai studio requests
New Feature
2024-08-17 10:46:59 -07:00
Krrish Dholakia
668ea6cbc7 fix(pass_through_endpoints.py): fix returned response headers for pass-through endpoitns 2024-08-17 09:00:00 -07:00
Krrish Dholakia
2874b94fb1 refactor: replace .error() with .exception() logging for better debugging on sentry 2024-08-16 09:22:47 -07:00
Krrish Dholakia
362605ba65 feat(pass_through_endpoints.py): initial working CRUD endpoints for /pass_through_endoints 2024-08-15 21:23:26 -07:00
Krrish Dholakia
7a67551927 feat(pass_through_endpoints.py): initial commit of crud endpoints for pass through endpoints 2024-08-14 21:36:07 -07:00
Krrish Dholakia
8b4b378562 feat(user_api_key_auth.py): support calling langfuse with litellm user_api_key_auth 2024-08-13 20:26:24 -07:00
Ishaan Jaff
5eabef4318 fix pass through endpoint tests 2024-08-06 21:16:07 -07:00
Ishaan Jaff
32e531eb07 init pass through endpoints 2024-08-06 12:04:04 -07:00
Ishaan Jaff
4656f7981d use helper to forward headers from request 2024-08-06 11:34:10 -07:00
Ishaan Jaff
e8f4ee56ad forward headers from request 2024-08-05 21:45:44 -07:00
Krrish Dholakia
7a95132a03 fix(pass_through_endpoints.py): fix client init 2024-07-13 14:46:56 -07:00
Krrish Dholakia
1d6643df22 feat(pass_through_endpoint.py): support enforcing key rpm limits on pass through endpoints
Closes https://github.com/BerriAI/litellm/issues/4698
2024-07-13 13:29:44 -07:00
Krrish Dholakia
c8a2782df8 docs(pass_through.md): add doc on creating custom chat endpoints on proxy
Allows developers to call proxy with anthropic sdk/boto3/etc.
2024-07-12 18:48:40 -07:00
Ishaan Jaff
3dc6430fef feat - setting up auth on pass through endpoint 2024-06-29 08:38:44 -07:00
Ishaan Jaff
033da54a03 feat - pass through langfuse requests 2024-06-28 17:27:13 -07:00
Ishaan Jaff
69deb65c04 fix use os.environ/ vars for pass through endpoints 2024-06-28 15:30:31 -07:00
Ishaan Jaff
a8c8138552 feat - support pass through endpoints 2024-06-28 15:03:21 -07:00