171 commits

Author	SHA1	Message	Date
Krrish Dholakia	3c4c78a71f	feat(caching.py): enable caching on provider-specific optional params ... Closes https://github.com/BerriAI/litellm/issues/5049	2024-08-05 11:18:59 -07:00
Ishaan Jaff	b6b19dc128	use file name when getting cache key	2024-08-02 14:52:08 -07:00
Krrish Dholakia	31445ab20a	fix(caching.py): support /completion caching by default ... updates supported call types in redis cache to cover text_completion caching	2024-07-29 08:19:30 -07:00
Ishaan Jaff	19fb5cc11c	use common helpers for writing to otel	2024-07-27 11:40:39 -07:00
Ishaan Jaff	2a89486948	move _get_parent_otel_span_from_kwargs to otel.py	2024-07-27 11:12:13 -07:00
Ishaan Jaff	677db38f8b	add doc string to explain what delete cache does	2024-07-13 12:25:31 -07:00
Ishaan Jaff	0099bf7859	de-ref unused cache items	2024-07-12 16:38:36 -07:00
Krrish Dholakia	3f83e8a8d4	fix(caching.py): fix async redis health check	2024-07-06 09:14:29 -07:00
Ishaan Jaff	511dd18e4b	remove debug print statement	2024-06-27 20:58:29 -07:00
Ishaan Jaff	e899359427	ci/cd add debugging for cache eviction	2024-06-25 08:14:09 -07:00
Ishaan Jaff	05fe43f495	fix default ttl for InMemoryCache	2024-06-24 21:21:38 -07:00
Ishaan Jaff	fa57d2e823	feat use custom eviction policy	2024-06-24 20:28:03 -07:00
Ishaan Jaff	effc7579ac	fix install on python 3.8	2024-06-24 17:27:14 -07:00
Ishaan Jaff	b13a93d9bc	cleanup InMemoryCache	2024-06-24 17:24:59 -07:00
Ishaan Jaff	4053c7aeb3	use lru cache	2024-06-24 17:15:53 -07:00
Ishaan Jaff	e5ab0d4ecd	fix InMemoryCache	2024-06-24 17:08:30 -07:00
Ishaan Jaff	974d92ff45	fix use caching lib	2024-06-24 17:03:23 -07:00
Ishaan Jaff	8a66e074ce	fix in mem cache tests	2024-06-22 19:52:18 -07:00
Ishaan Jaff	fbef5013a1	Merge branch 'main' into litellm_fix_in_mem_usage	2024-06-22 19:23:37 -07:00
Ishaan Jaff	0418db3044	fix caching clear in memory cache mem util	2024-06-22 19:21:37 -07:00
Ishaan Jaff	fa554ae218	fix - clean up in memory cache	2024-06-22 18:46:30 -07:00
Krrish Dholakia	a028600932	feat(dynamic_rate_limiter.py): update cache with active project	2024-06-21 20:25:40 -07:00
David Manouchehri	cf10d13ac5	fix(caching.py): Stop throwing constant spam errors on every single S3 cache miss. Fixes #4146.	2024-06-13 20:58:18 +00:00
Ishaan Jaff	f152b5eb1d	feat - final working redis cache otel	2024-06-07 16:36:04 -07:00
Ishaan Jaff	5a5dd33b24	feat - working exception logs for Redis errors	2024-06-07 16:30:29 -07:00
Ishaan Jaff	e86fa19257	fix - basic success logging for redis cache	2024-06-07 16:20:23 -07:00
Krrish Dholakia	6cca5612d2	refactor: replace 'traceback.print_exc()' with logging library ... allows error logs to be in json format for otel logging	2024-06-06 13:47:43 -07:00
Krrish Dholakia	19321094da	fix(proxy_server.py): allow user_api_key_cache_ttl to be a controllable param	2024-05-25 12:07:28 -07:00
Ishaan Jaff	0c8f5e5649	Merge pull request #3266 from antonioloison/litellm_add_disk_cache ... [Feature] Add cache to disk	2024-05-14 09:24:01 -07:00
Rahul Kataria	689221b5c7	[Optimize] Optimize code in caching file	2024-05-12 17:04:18 +05:30
Rahul Kataria	3f146b2c7e	[Reset Code]	2024-05-12 16:37:50 +05:30
Rahul Kataria	6304760dd0	[Optimize] Optimize the code in caching file	2024-05-12 15:04:45 +05:30
Antonio Loison	79c3d39d67	build(caching.py): move diskcache import inside class and add cache_dir argument to Cache	2024-05-10 12:04:54 +02:00
Antonio Loison	c1ba4ec078	chore: add diskcache as extra dependency	2024-05-10 11:19:14 +02:00
Antonio Loison	004877c7e5	build(caching.py): add disk option for cache	2024-05-10 10:03:38 +02:00
Antonio Loison	c1a81d90a6	build(caching.py) - add disk cache object	2024-05-10 10:03:35 +02:00
Krrish Dholakia	718f423d7d	feat(slack_alerting.py): support sending daily reports on deployments ... allow admin to easily know slow + failing deployments Closes https://github.com/BerriAI/litellm/issues/3483	2024-05-06 17:18:42 -07:00
Krrish Dholakia	5a79f648c6	fix(caching.py): fix redis caching ping check ... don't fail to startup. Log an error message.	2024-05-04 08:48:53 -07:00
Krish Dholakia	6d9f0f1839	Merge branch 'main' into litellm_ssl_caching_fix	2024-04-19 17:20:27 -07:00
Krrish Dholakia	0d9c96bebf	test(test_prometheus_services.py): fix testing to handle caching ping in init	2024-04-19 16:15:29 -07:00
Krrish Dholakia	84685b5f34	fix(_redis.py): fix args passed to redis.from_url argument	2024-04-19 15:27:11 -07:00
Krrish Dholakia	01a1a8f731	fix(caching.py): dual cache async_batch_get_cache fix + testing ... this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)	2024-04-19 15:03:25 -07:00
Krrish Dholakia	08cf77623b	fix(caching.py): remove url parsing logic - causing redis ssl connections to fail ... this reverts a change that was causing redis url w/ ssl to fail. this also adds unit testing for this sc enario, to prevent future regressions	2024-04-19 14:01:13 -07:00
Krrish Dholakia	9c42c847a5	fix(router.py): instrument pre-call-checks for all openai endpoints	2024-04-18 21:54:25 -07:00
Krrish Dholakia	81573b2dd9	fix(test_lowest_tpm_rpm_routing_v2.py): unit testing for usage-based-routing-v2	2024-04-18 21:38:00 -07:00
Krrish Dholakia	a05f148c17	fix(tpm_rpm_routing_v2.py): fix tpm rpm routing	2024-04-18 20:01:22 -07:00
Krrish Dholakia	f65005f273	fix(caching.py): fix error logging	2024-04-18 17:21:10 -07:00
Krrish Dholakia	0afb5d6573	fix(caching.py): add call type to service logger object	2024-04-18 16:53:34 -07:00
Krrish Dholakia	0f95a824c4	feat(prometheus_services.py): emit proxy latency for successful llm api requests ... uses prometheus histogram for this	2024-04-18 16:04:35 -07:00
Krrish Dholakia	bef24cd4ab	fix(caching.py): don't decode a string	2024-04-13 18:48:03 -07:00