Litellm stable dev (#5711)

Browse source

* feat(aws_base_llm.py): prevents recreating boto3 credentials during high traffic

Leads to 100ms perf boost in local testing

* fix(base_aws_llm.py): fix credential caching check to see if token is set

* refactor(bedrock/chat): separate converse api and invoke api + isolate converse api transformation logic

Make it easier to see how requests are transformed for /converse

* fix: fix imports

* fix(bedrock/embed): fix reordering of headers

* fix(base_aws_llm.py): fix get credential logic

* fix(converse_handler.py): fix ai21 streaming response

...

This commit is contained in:

Krish Dholakia 2024-09-14 23:22:59 -07:00 • committed by GitHub

parent 2efdd2a6a4

commit da77706c26

No known key found for this signature in database

GPG key ID: B5690EEEBB952194

14 changed files with 1073 additions and 1039 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1344

litellm/llms/bedrock/chat/invoke_handler.py Normal file

View file

File diff suppressed because it is too large Load diff