Ishaan Jaff
d4b9a1307d
rename get_async_httpx_client
2024-09-10 10:38:01 -07:00
Ishaan Jaff
81ee1653af
use correct type hints for audio transcriptions
2024-09-05 09:12:27 -07:00
Krish Dholakia
1e7e538261
LiteLLM Minor fixes + improvements (08/04/2024) ( #5505 )
...
* Minor IAM AWS OIDC Improvements (#5246 )
* AWS IAM: Temporary tokens are valid across all regions after being issued, so it is wasteful to request one for each region.
* AWS IAM: Include an inline policy, to help reduce misuse of overly permissive IAM roles.
* (test_bedrock_completion.py): Ensure we are testing cross AWS region OIDC flow.
* fix(router.py): log rejected requests
Fixes https://github.com/BerriAI/litellm/issues/5498
* refactor: don't use verbose_logger.exception, if exception is raised
User might already have handling for this. But alerting systems in prod will raise this as an unhandled error.
* fix(datadog.py): support setting datadog source as an env var
Fixes https://github.com/BerriAI/litellm/issues/5508
* docs(logging.md): add dd_source to datadog docs
* fix(proxy_server.py): expose `/customer/list` endpoint for showing all customers
* (bedrock): Fix usage with Cloudflare AI Gateway, and proxies in general. (#5509 )
* feat(anthropic.py): support 'cache_control' param for content when it is a string
* Revert "(bedrock): Fix usage with Cloudflare AI Gateway, and proxies in gener…" (#5519 )
This reverts commit 3fac0349c2
2024-09-04 22:16:55 -07:00
Ishaan Jaff
ca5a117544
dual cache use always read redis as True by default
2024-09-04 08:01:55 -07:00
Ishaan Jaff
fd122cb759
fix always read redis
2024-09-02 21:08:32 -07:00
Ishaan Jaff
15296b4fb7
fix allow qdrant api key to be optional
2024-08-30 11:13:23 -07:00
Krrish Dholakia
0eea01dae9
feat(vertex_ai_context_caching.py): check gemini cache, if key already exists
2024-08-26 22:19:01 -07:00
Ishaan Jaff
feb354d3bc
fix should_use_cache
2024-08-24 09:37:41 -07:00
Ishaan Jaff
3c1da2e823
feat - allow setting cache mode
2024-08-24 09:03:59 -07:00
Krrish Dholakia
e2d7539690
feat(caching.py): redis cluster support
...
Closes https://github.com/BerriAI/litellm/issues/4358
2024-08-21 15:01:52 -07:00
Ishaan Jaff
e7ecb2fe3a
fix qdrant litellm on proxy
2024-08-21 12:52:29 -07:00
Ishaan Jaff
c6dfd2d276
fixes for using qdrant with litellm proxy
2024-08-21 12:36:41 -07:00
Ishaan Jaff
428a74be07
fix drant url
2024-08-21 12:09:09 -07:00
Ishaan Jaff
7d0196191f
Merge pull request #5018 from haadirakhangi/main
...
Qdrant Semantic Caching
2024-08-21 08:50:43 -07:00
Haadi Rakhangi
7f1c3f5edf
implemented RestAPI and added support for cloud and local Qdrant clusters
2024-08-19 20:46:30 +05:30
Krrish Dholakia
61f4b71ef7
refactor: replace .error() with .exception() logging for better debugging on sentry
2024-08-16 09:22:47 -07:00
prd-tuong-nguyen
3445174ebe
feat: hash prompt when caching
2024-08-08 16:19:14 +07:00
Ishaan Jaff
467c506e33
caching use file_checksum
2024-08-06 13:03:14 -07:00
Krrish Dholakia
a9fdfb5a99
fix(init.py): rename feature_flag
2024-08-05 11:23:20 -07:00
Krrish Dholakia
3c4c78a71f
feat(caching.py): enable caching on provider-specific optional params
...
Closes https://github.com/BerriAI/litellm/issues/5049
2024-08-05 11:18:59 -07:00
Ishaan Jaff
b6b19dc128
use file name when getting cache key
2024-08-02 14:52:08 -07:00
Haadi Rakhangi
851db5ecea
qdrant semantic caching added
2024-08-02 21:07:19 +05:30
Krrish Dholakia
31445ab20a
fix(caching.py): support /completion caching by default
...
updates supported call types in redis cache to cover text_completion caching
2024-07-29 08:19:30 -07:00
Ishaan Jaff
19fb5cc11c
use common helpers for writing to otel
2024-07-27 11:40:39 -07:00
Ishaan Jaff
2a89486948
move _get_parent_otel_span_from_kwargs to otel.py
2024-07-27 11:12:13 -07:00
Ishaan Jaff
677db38f8b
add doc string to explain what delete cache does
2024-07-13 12:25:31 -07:00
Ishaan Jaff
0099bf7859
de-ref unused cache items
2024-07-12 16:38:36 -07:00
Krrish Dholakia
3f83e8a8d4
fix(caching.py): fix async redis health check
2024-07-06 09:14:29 -07:00
Ishaan Jaff
511dd18e4b
remove debug print statement
2024-06-27 20:58:29 -07:00
Ishaan Jaff
e899359427
ci/cd add debugging for cache eviction
2024-06-25 08:14:09 -07:00
Ishaan Jaff
05fe43f495
fix default ttl for InMemoryCache
2024-06-24 21:21:38 -07:00
Ishaan Jaff
fa57d2e823
feat use custom eviction policy
2024-06-24 20:28:03 -07:00
Ishaan Jaff
effc7579ac
fix install on python 3.8
2024-06-24 17:27:14 -07:00
Ishaan Jaff
b13a93d9bc
cleanup InMemoryCache
2024-06-24 17:24:59 -07:00
Ishaan Jaff
4053c7aeb3
use lru cache
2024-06-24 17:15:53 -07:00
Ishaan Jaff
e5ab0d4ecd
fix InMemoryCache
2024-06-24 17:08:30 -07:00
Ishaan Jaff
974d92ff45
fix use caching lib
2024-06-24 17:03:23 -07:00
Ishaan Jaff
8a66e074ce
fix in mem cache tests
2024-06-22 19:52:18 -07:00
Ishaan Jaff
fbef5013a1
Merge branch 'main' into litellm_fix_in_mem_usage
2024-06-22 19:23:37 -07:00
Ishaan Jaff
0418db3044
fix caching clear in memory cache mem util
2024-06-22 19:21:37 -07:00
Ishaan Jaff
fa554ae218
fix - clean up in memory cache
2024-06-22 18:46:30 -07:00
Krrish Dholakia
a028600932
feat(dynamic_rate_limiter.py): update cache with active project
2024-06-21 20:25:40 -07:00
David Manouchehri
cf10d13ac5
fix(caching.py): Stop throwing constant spam errors on every single S3 cache miss. Fixes #4146 .
2024-06-13 20:58:18 +00:00
Ishaan Jaff
f152b5eb1d
feat - final working redis cache otel
2024-06-07 16:36:04 -07:00
Ishaan Jaff
5a5dd33b24
feat - working exception logs for Redis errors
2024-06-07 16:30:29 -07:00
Ishaan Jaff
e86fa19257
fix - basic success logging for redis cache
2024-06-07 16:20:23 -07:00
Krrish Dholakia
6cca5612d2
refactor: replace 'traceback.print_exc()' with logging library
...
allows error logs to be in json format for otel logging
2024-06-06 13:47:43 -07:00
Krrish Dholakia
19321094da
fix(proxy_server.py): allow user_api_key_cache_ttl to be a controllable param
2024-05-25 12:07:28 -07:00
Ishaan Jaff
0c8f5e5649
Merge pull request #3266 from antonioloison/litellm_add_disk_cache
...
[Feature] Add cache to disk
2024-05-14 09:24:01 -07:00
Rahul Kataria
689221b5c7
[Optimize] Optimize code in caching file
2024-05-12 17:04:18 +05:30
