Ishaan Jaff
81ef2c38dc
fix InMemoryCache
2024-06-24 17:08:30 -07:00
Ishaan Jaff
21fd91fe94
fix use caching lib
2024-06-24 17:03:23 -07:00
Ishaan Jaff
0c4c6bfa5e
fix in mem cache tests
2024-06-22 19:52:18 -07:00
Ishaan Jaff
8e3a073323
Merge branch 'main' into litellm_fix_in_mem_usage
2024-06-22 19:23:37 -07:00
Ishaan Jaff
5b2d4da43f
fix caching clear in memory cache mem util
2024-06-22 19:21:37 -07:00
Ishaan Jaff
c4ae06576b
fix - clean up in memory cache
2024-06-22 18:46:30 -07:00
Krrish Dholakia
0430807178
feat(dynamic_rate_limiter.py): update cache with active project
2024-06-21 20:25:40 -07:00
David Manouchehri
47e3880638
fix(caching.py): Stop throwing constant spam errors on every single S3 cache miss. Fixes #4146 .
2024-06-13 20:58:18 +00:00
Ishaan Jaff
786e6b4ae3
feat - final working redis cache otel
2024-06-07 16:36:04 -07:00
Ishaan Jaff
72a6d49b21
feat - working exception logs for Redis errors
2024-06-07 16:30:29 -07:00
Ishaan Jaff
ef99e3e1d0
fix - basic success logging for redis cache
2024-06-07 16:20:23 -07:00
Krrish Dholakia
e391e30285
refactor: replace 'traceback.print_exc()' with logging library
...
allows error logs to be in json format for otel logging
2024-06-06 13:47:43 -07:00
Krrish Dholakia
2db3819142
fix(proxy_server.py): allow user_api_key_cache_ttl to be a controllable param
2024-05-25 12:07:28 -07:00
Ishaan Jaff
a3fb6e8c34
Merge pull request #3266 from antonioloison/litellm_add_disk_cache
...
[Feature] Add cache to disk
2024-05-14 09:24:01 -07:00
Rahul Kataria
927b6fa967
[Optimize] Optimize code in caching file
2024-05-12 17:04:18 +05:30
Rahul Kataria
6c16f2924b
[Reset Code]
2024-05-12 16:37:50 +05:30
Rahul Kataria
9b77b8c90b
[Optimize] Optimize the code in caching file
2024-05-12 15:04:45 +05:30
Antonio Loison
dabf4cf8a8
build(caching.py): move diskcache import inside class and add cache_dir argument to Cache
2024-05-10 12:04:54 +02:00
Antonio Loison
79d774e330
chore: add diskcache as extra dependency
2024-05-10 11:19:14 +02:00
Antonio Loison
04b5dbef1a
build(caching.py): add disk option for cache
2024-05-10 10:03:38 +02:00
Antonio Loison
34f4f719aa
build(caching.py) - add disk cache object
2024-05-10 10:03:35 +02:00
Krrish Dholakia
0c1fc34e9f
feat(slack_alerting.py): support sending daily reports on deployments
...
allow admin to easily know slow + failing deployments
Closes https://github.com/BerriAI/litellm/issues/3483
2024-05-06 17:18:42 -07:00
Krrish Dholakia
da4991de05
fix(caching.py): fix redis caching ping check
...
don't fail to startup. Log an error message.
2024-05-04 08:48:53 -07:00
Krish Dholakia
a9dc93e860
Merge branch 'main' into litellm_ssl_caching_fix
2024-04-19 17:20:27 -07:00
Krrish Dholakia
b3a8c2885b
test(test_prometheus_services.py): fix testing to handle caching ping in init
2024-04-19 16:15:29 -07:00
Krrish Dholakia
5bc0437680
fix(_redis.py): fix args passed to redis.from_url argument
2024-04-19 15:27:11 -07:00
Krrish Dholakia
5da934099f
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Krrish Dholakia
7065e4ee12
fix(caching.py): remove url parsing logic - causing redis ssl connections to fail
...
this reverts a change that was causing redis url w/ ssl to fail. this also adds unit testing for this sc
enario, to prevent future regressions
2024-04-19 14:01:13 -07:00
Krrish Dholakia
5bb73dc9c0
fix(router.py): instrument pre-call-checks for all openai endpoints
2024-04-18 21:54:25 -07:00
Krrish Dholakia
376ee4e9d7
fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2
2024-04-18 21:38:00 -07:00
Krrish Dholakia
72691e05f4
fix(tpm_rpm_routing_v2.py): fix tpm rpm routing
2024-04-18 20:01:22 -07:00
Krrish Dholakia
852455e7fc
fix(caching.py): fix error logging
2024-04-18 17:21:10 -07:00
Krrish Dholakia
27ea7b5fef
fix(caching.py): add call type to service logger object
2024-04-18 16:53:34 -07:00
Krrish Dholakia
7f5bcf38b7
feat(prometheus_services.py): emit proxy latency for successful llm api requests
...
uses prometheus histogram for this
2024-04-18 16:04:35 -07:00
Krrish Dholakia
b0fc2b342d
fix(caching.py): don't decode a string
2024-04-13 18:48:03 -07:00
Krrish Dholakia
866259f95f
feat(prometheus_services.py): monitor health of proxy adjacent services (redis / postgres / etc.)
2024-04-13 18:15:02 -07:00
Krrish Dholakia
d9b8f63e86
fix(router.py): support pre_call_rpm_check for lowest_tpm_rpm_v2 routing
...
have routing strategies expose an ‘update rpm’ function; for checking + updating rpm pre call
2024-04-12 18:25:14 -07:00
Krrish Dholakia
fb6ded6e40
fix(caching.py): fix async batch redis get request
2024-04-10 21:37:45 -07:00
Krrish Dholakia
31e2d4e6d1
feat(lowest_tpm_rpm_v2.py): move to using redis.incr and redis.mget for getting model usage from redis
...
makes routing work across multiple instances
2024-04-10 14:56:23 -07:00
Ishaan Jaff
a515441ad1
feat - delete cache key
2024-04-04 10:56:47 -07:00
Krish Dholakia
6a1e08125c
Merge pull request #2789 from BerriAI/litellm_set_ttl
...
fix(proxy_server.py): allow user to set in-memory + redis ttl
2024-04-02 08:53:34 -07:00
Krrish Dholakia
aa5f88ec45
fix(proxy_server.py): allow user to set in-memory + redis ttl
...
addresses - https://github.com/BerriAI/litellm/issues/2700
2024-04-01 19:13:23 -07:00
Ishaan Jaff
41f7f0fbfe
(fix) handle urls correctly
2024-04-01 17:08:48 -07:00
Ishaan Jaff
c1080e04c0
redis show client_list, info
2024-04-01 16:54:42 -07:00
Krrish Dholakia
b2e7866ea9
fix(caching.py): respect redis namespace for all redis get/set requests
2024-03-30 20:20:29 -07:00
Krrish Dholakia
555f0af027
fix(tpm_rpm_limiter.py): enable redis caching for tpm/rpm checks on keys/user/teams
...
allows tpm/rpm checks to work across instances
https://github.com/BerriAI/litellm/issues/2730
2024-03-30 20:01:36 -07:00
Ishaan Jaff
ee54bbcd89
(fix) undo changes from other branches
2024-03-26 09:22:19 -07:00
Ishaan Jaff
f1ebbd32b8
(feat) /cache/flushall
2024-03-26 09:18:58 -07:00
Ishaan Jaff
336fe2f876
(fix) in mem redis reads
2024-03-26 09:10:49 -07:00
Ishaan Jaff
c898ffe636
(feat) improve cache debugging litellm
2024-03-25 18:26:58 -07:00
