Ishaan Jaff
4053c7aeb3
use lru cache
2024-06-24 17:15:53 -07:00
Ishaan Jaff
e5ab0d4ecd
fix InMemoryCache
2024-06-24 17:08:30 -07:00
Ishaan Jaff
974d92ff45
fix use caching lib
2024-06-24 17:03:23 -07:00
Ishaan Jaff
8a66e074ce
fix in mem cache tests
2024-06-22 19:52:18 -07:00
Ishaan Jaff
fbef5013a1
Merge branch 'main' into litellm_fix_in_mem_usage
2024-06-22 19:23:37 -07:00
Ishaan Jaff
0418db3044
fix caching clear in memory cache mem util
2024-06-22 19:21:37 -07:00
Ishaan Jaff
fa554ae218
fix - clean up in memory cache
2024-06-22 18:46:30 -07:00
Krrish Dholakia
a028600932
feat(dynamic_rate_limiter.py): update cache with active project
2024-06-21 20:25:40 -07:00
David Manouchehri
cf10d13ac5
fix(caching.py): Stop throwing constant spam errors on every single S3 cache miss. Fixes #4146 .
2024-06-13 20:58:18 +00:00
Ishaan Jaff
f152b5eb1d
feat - final working redis cache otel
2024-06-07 16:36:04 -07:00
Ishaan Jaff
5a5dd33b24
feat - working exception logs for Redis errors
2024-06-07 16:30:29 -07:00
Ishaan Jaff
e86fa19257
fix - basic success logging for redis cache
2024-06-07 16:20:23 -07:00
Krrish Dholakia
6cca5612d2
refactor: replace 'traceback.print_exc()' with logging library
...
allows error logs to be in json format for otel logging
2024-06-06 13:47:43 -07:00
Krrish Dholakia
19321094da
fix(proxy_server.py): allow user_api_key_cache_ttl to be a controllable param
2024-05-25 12:07:28 -07:00
Ishaan Jaff
0c8f5e5649
Merge pull request #3266 from antonioloison/litellm_add_disk_cache
...
[Feature] Add cache to disk
2024-05-14 09:24:01 -07:00
Rahul Kataria
689221b5c7
[Optimize] Optimize code in caching file
2024-05-12 17:04:18 +05:30
Rahul Kataria
3f146b2c7e
[Reset Code]
2024-05-12 16:37:50 +05:30
Rahul Kataria
6304760dd0
[Optimize] Optimize the code in caching file
2024-05-12 15:04:45 +05:30
Antonio Loison
79c3d39d67
build(caching.py): move diskcache import inside class and add cache_dir argument to Cache
2024-05-10 12:04:54 +02:00
Antonio Loison
c1ba4ec078
chore: add diskcache as extra dependency
2024-05-10 11:19:14 +02:00
Antonio Loison
004877c7e5
build(caching.py): add disk option for cache
2024-05-10 10:03:38 +02:00
Antonio Loison
c1a81d90a6
build(caching.py) - add disk cache object
2024-05-10 10:03:35 +02:00
Krrish Dholakia
718f423d7d
feat(slack_alerting.py): support sending daily reports on deployments
...
allow admin to easily know slow + failing deployments
Closes https://github.com/BerriAI/litellm/issues/3483
2024-05-06 17:18:42 -07:00
Krrish Dholakia
5a79f648c6
fix(caching.py): fix redis caching ping check
...
don't fail to startup. Log an error message.
2024-05-04 08:48:53 -07:00
Krish Dholakia
6d9f0f1839
Merge branch 'main' into litellm_ssl_caching_fix
2024-04-19 17:20:27 -07:00
Krrish Dholakia
0d9c96bebf
test(test_prometheus_services.py): fix testing to handle caching ping in init
2024-04-19 16:15:29 -07:00
Krrish Dholakia
84685b5f34
fix(_redis.py): fix args passed to redis.from_url argument
2024-04-19 15:27:11 -07:00
Krrish Dholakia
01a1a8f731
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Krrish Dholakia
08cf77623b
fix(caching.py): remove url parsing logic - causing redis ssl connections to fail
...
this reverts a change that was causing redis url w/ ssl to fail. this also adds unit testing for this sc
enario, to prevent future regressions
2024-04-19 14:01:13 -07:00
Krrish Dholakia
9c42c847a5
fix(router.py): instrument pre-call-checks for all openai endpoints
2024-04-18 21:54:25 -07:00
Krrish Dholakia
81573b2dd9
fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2
2024-04-18 21:38:00 -07:00
Krrish Dholakia
a05f148c17
fix(tpm_rpm_routing_v2.py): fix tpm rpm routing
2024-04-18 20:01:22 -07:00
Krrish Dholakia
f65005f273
fix(caching.py): fix error logging
2024-04-18 17:21:10 -07:00
Krrish Dholakia
0afb5d6573
fix(caching.py): add call type to service logger object
2024-04-18 16:53:34 -07:00
Krrish Dholakia
0f95a824c4
feat(prometheus_services.py): emit proxy latency for successful llm api requests
...
uses prometheus histogram for this
2024-04-18 16:04:35 -07:00
Krrish Dholakia
bef24cd4ab
fix(caching.py): don't decode a string
2024-04-13 18:48:03 -07:00
Krrish Dholakia
4e81acf2c6
feat(prometheus_services.py): monitor health of proxy adjacent services (redis / postgres / etc.)
2024-04-13 18:15:02 -07:00
Krrish Dholakia
c03b0bbb24
fix(router.py): support pre_call_rpm_check for lowest_tpm_rpm_v2 routing
...
have routing strategies expose an ‘update rpm’ function; for checking + updating rpm pre call
2024-04-12 18:25:14 -07:00
Krrish Dholakia
76bd667b12
fix(caching.py): fix async batch redis get request
2024-04-10 21:37:45 -07:00
Krrish Dholakia
180cf9bd5c
feat(lowest_tpm_rpm_v2.py): move to using redis.incr and redis.mget for getting model usage from redis
...
makes routing work across multiple instances
2024-04-10 14:56:23 -07:00
Ishaan Jaff
c4cb0afa98
feat - delete cache key
2024-04-04 10:56:47 -07:00
Krish Dholakia
7233e5ab25
Merge pull request #2789 from BerriAI/litellm_set_ttl
...
fix(proxy_server.py): allow user to set in-memory + redis ttl
2024-04-02 08:53:34 -07:00
Krrish Dholakia
203e2776f8
fix(proxy_server.py): allow user to set in-memory + redis ttl
...
addresses - https://github.com/BerriAI/litellm/issues/2700
2024-04-01 19:13:23 -07:00
Ishaan Jaff
4ebb4570be
(fix) handle urls correctly
2024-04-01 17:08:48 -07:00
Ishaan Jaff
916e4032ee
redis show client_list, info
2024-04-01 16:54:42 -07:00
Krrish Dholakia
5926792de6
fix(caching.py): respect redis namespace for all redis get/set requests
2024-03-30 20:20:29 -07:00
Krrish Dholakia
f58fefd589
fix(tpm_rpm_limiter.py): enable redis caching for tpm/rpm checks on keys/user/teams
...
allows tpm/rpm checks to work across instances
https://github.com/BerriAI/litellm/issues/2730
2024-03-30 20:01:36 -07:00
Ishaan Jaff
098a03facc
(fix) undo changes from other branches
2024-03-26 09:22:19 -07:00
Ishaan Jaff
b8af946fb9
(feat) /cache/flushall
2024-03-26 09:18:58 -07:00
Ishaan Jaff
ade5d58331
(fix) in mem redis reads
2024-03-26 09:10:49 -07:00
