Commit graph

20 commits

Author SHA1 Message Date
Krish Dholakia
6fdee99632 LiteLLM Minor fixes + improvements (08/04/2024) (#5505)
* Minor IAM AWS OIDC Improvements (#5246)

* AWS IAM: Temporary tokens are valid across all regions after being issued, so it is wasteful to request one for each region.

* AWS IAM: Include an inline policy, to help reduce misuse of overly permissive IAM roles.

* (test_bedrock_completion.py): Ensure we are testing cross AWS region OIDC flow.

* fix(router.py): log rejected requests

Fixes https://github.com/BerriAI/litellm/issues/5498

* refactor: don't use verbose_logger.exception, if exception is raised

User might already have handling for this. But alerting systems in prod will raise this as an unhandled error.

* fix(datadog.py): support setting datadog source as an env var

Fixes https://github.com/BerriAI/litellm/issues/5508

* docs(logging.md): add dd_source to datadog docs

* fix(proxy_server.py): expose `/customer/list` endpoint for showing all customers

* (bedrock): Fix usage with Cloudflare AI Gateway, and proxies in general. (#5509)

* feat(anthropic.py): support 'cache_control' param for content when it is a string

* Revert "(bedrock): Fix usage with Cloudflare AI Gateway, and proxies in gener…" (#5519)

This reverts commit 3fac0349c2.

* refactor: ci/cd run again

---------

Co-authored-by: David Manouchehri <david.manouchehri@ai.moda>
2024-09-04 22:16:55 -07:00
Krrish Dholakia
2874b94fb1 refactor: replace .error() with .exception() logging for better debugging on sentry 2024-08-16 09:22:47 -07:00
Krrish Dholakia
8a671aa20b fix(main.py): log hidden params for text completion calls 2024-08-05 21:26:48 -07:00
Krrish Dholakia
368af6975a fix(http_handler.py): correctly re-raise timeout exception 2024-07-31 14:51:28 -07:00
Krrish Dholakia
c88c689705 fix: fix linting errors 2024-07-31 07:19:45 -07:00
Ishaan Jaff
6171fbc5fd fix timeouts for predibase - they are unstable af 2024-07-30 22:48:30 -07:00
Krrish Dholakia
c69193c321 fix: move to using pydantic obj for setting values 2024-07-11 13:18:36 -07:00
Krrish Dholakia
91bbef4bcd fix(predibase.py): support json schema on predibase 2024-06-25 16:03:47 -07:00
Krrish Dholakia
1e51b8894f fix(utils.py): predibase exception mapping - map 424 as a badrequest error 2024-06-25 13:47:38 -07:00
Krrish Dholakia
9d7f5d503c refactor(utils.py): refactor Logging to it's own class. Cut down utils.py to <10k lines.
Easier debugging

 Reference: https://github.com/BerriAI/litellm/issues/4206
2024-06-15 10:57:20 -07:00
Krrish Dholakia
39ee6be477 fix(utils.py): improved predibase exception mapping
adds unit testing + better coverage for predibase errors
2024-06-08 14:32:43 -07:00
Krrish Dholakia
9bd5c20803 fix(predibase.py): add better error message 2024-06-04 19:55:15 -07:00
Krrish Dholakia
3896e3e88f fix: fix streaming with httpx client
prevent overwriting streams in parallel streaming calls
2024-05-31 10:55:18 -07:00
Krrish Dholakia
bd0c3a81cb fix(bedrock_httpx.py): working async bedrock command r calls 2024-05-11 16:45:20 -07:00
Krrish Dholakia
910c2cd36a fix(predibase.py): fix async streaming 2024-05-09 22:18:16 -07:00
Krrish Dholakia
30136493b4 fix(predibase.py): fix event loop closed error 2024-05-09 19:07:19 -07:00
Krrish Dholakia
61fc042f9b fix(predibase.py): fix async completion call 2024-05-09 18:44:19 -07:00
Krrish Dholakia
ef72f25ab6 fix: fix linting errors 2024-05-09 17:55:27 -07:00
Krrish Dholakia
7c0ab40fd5 feat(predibase.py): support async_completion + streaming (sync + async)
finishes up pr
2024-05-09 17:41:27 -07:00
Krrish Dholakia
f660d21743 feat(predibase.py): add support for predibase provider
Closes https://github.com/BerriAI/litellm/issues/1253
2024-05-09 16:39:43 -07:00