168 commits

Author	SHA1	Message	Date
Krrish Dholakia	180cf9bd5c	feat(lowest_tpm_rpm_v2.py): move to using redis.incr and redis.mget for getting model usage from redis ... makes routing work across multiple instances	2024-04-10 14:56:23 -07:00
Ishaan Jaff	c4cb0afa98	feat - delete cache key	2024-04-04 10:56:47 -07:00
Krish Dholakia	7233e5ab25	Merge pull request #2789 from BerriAI/litellm_set_ttl ... fix(proxy_server.py): allow user to set in-memory + redis ttl	2024-04-02 08:53:34 -07:00
Krrish Dholakia	203e2776f8	fix(proxy_server.py): allow user to set in-memory + redis ttl ... addresses - https://github.com/BerriAI/litellm/issues/2700	2024-04-01 19:13:23 -07:00
Ishaan Jaff	4ebb4570be	(fix) handle urls correctly	2024-04-01 17:08:48 -07:00
Ishaan Jaff	916e4032ee	redis show client_list, info	2024-04-01 16:54:42 -07:00
Krrish Dholakia	5926792de6	fix(caching.py): respect redis namespace for all redis get/set requests	2024-03-30 20:20:29 -07:00
Krrish Dholakia	f58fefd589	fix(tpm_rpm_limiter.py): enable redis caching for tpm/rpm checks on keys/user/teams ... allows tpm/rpm checks to work across instances https://github.com/BerriAI/litellm/issues/2730	2024-03-30 20:01:36 -07:00
Ishaan Jaff	098a03facc	(fix) undo changes from other branches	2024-03-26 09:22:19 -07:00
Ishaan Jaff	b8af946fb9	(feat) /cache/flushall	2024-03-26 09:18:58 -07:00
Ishaan Jaff	ade5d58331	(fix) in mem redis reads	2024-03-26 09:10:49 -07:00
Ishaan Jaff	553ac19f9f	(feat) improve cache debugging litellm	2024-03-25 18:26:58 -07:00
Ishaan Jaff	32d4756943	(fix) print verbose in batch writing redis	2024-03-25 18:02:31 -07:00
Ishaan Jaff	7134d66fae	Merge branch 'main' into litellm_batch_write_redis_cache	2024-03-25 16:41:29 -07:00
Ishaan Jaff	abce689862	(feat) batch write redis cache output	2024-03-25 16:39:47 -07:00
Ishaan Jaff	37aadba959	(feat) v0 batch redis cache writes	2024-03-25 15:20:10 -07:00
Krrish Dholakia	591a0a376e	fix(caching.py): support default ttl for caching	2024-03-25 13:40:17 -07:00
Krish Dholakia	007d439017	Merge pull request #2606 from BerriAI/litellm_jwt_auth_updates ... fix(handle_jwt.py): track spend for user using jwt auth	2024-03-20 19:40:17 -07:00
Krrish Dholakia	8bb00c4ae8	fix(caching.py): enable async setting of cache for dual cache	2024-03-20 18:42:34 -07:00
Ishaan Jaff	cace0bd6fb	(fix) self.redis_version issue	2024-03-20 10:36:08 -07:00
Ishaan Jaff	7b29273b14	(fix) redis 6.2 version incompatibility issue	2024-03-20 09:38:21 -07:00
Ishaan Jaff	2256ece5a9	(feat) litellm cache ping	2024-03-20 08:24:13 -07:00
Krrish Dholakia	03acc07380	fix(caching.py): pass redis kwargs to connection pool init	2024-03-18 08:21:36 -07:00
Krrish Dholakia	48f96dd563	fix(caching.py): close redis connection pool upon proxy shutdown	2024-03-16 10:39:58 -07:00
Krrish Dholakia	3072137739	test(test_caching.py): fix async tests	2024-03-15 18:09:25 -07:00
Krrish Dholakia	226953e1d8	feat(batch_redis_get.py): batch redis GET requests for a given key + call type ... reduces the number of GET requests we're making in high-throughput scenarios	2024-03-15 14:40:11 -07:00
Krrish Dholakia	7c72cc0ec9	fix(caching.py): support redis caching with namespaces	2024-03-14 13:35:17 -07:00
Krrish Dholakia	d6537a05ca	fix(caching.py): fix print statements	2024-03-14 12:58:34 -07:00
Krrish Dholakia	1b807fa3f5	fix(proxy_server.py): fix key caching logic	2024-03-13 19:10:24 -07:00
Krish Dholakia	c7d0af0a2e	Merge pull request #2426 from BerriAI/litellm_whisper_cost_tracking ... feat: add cost tracking + caching for `/audio/transcription` calls	2024-03-09 19:12:06 -08:00
ishaan-jaff	1917ee7fcf	(feat) debug when in meory cache is used	2024-03-09 16:24:04 -08:00
Krrish Dholakia	9b7c8880c0	fix(caching.py): only add unique kwargs for transcription_only_kwargs in caching	2024-03-09 16:09:12 -08:00
Krrish Dholakia	fa45c569fd	feat: add cost tracking + caching for transcription calls	2024-03-09 15:43:38 -08:00
Krrish Dholakia	726dad5756	fix(caching.py): add s3 path as a top-level param	2024-03-06 18:07:28 -08:00
Krrish Dholakia	fb2ae3a032	fix(utils.py): only return cached streaming object for streaming calls	2024-02-21 21:27:40 -08:00
Krrish Dholakia	74381a2d06	fix(caching.py): use print verbose for logging error	2024-02-15 18:12:09 -08:00
Krrish Dholakia	66c7e35a97	fix(redis.py): fix instantiating redis client from url	2024-02-15 17:48:00 -08:00
ishaan-jaff	d87c4a4d23	(fix) s3 cache proxy - fix notImplemented error	2024-02-13 16:34:43 -08:00
ishaan-jaff	8197b3de0a	(fix) remove extra statement	2024-02-07 19:24:27 -08:00
ishaan-jaff	8425a8ba22	(fix) track cost for semantic_caching, place on langfuse trace	2024-02-07 19:21:50 -08:00
ishaan-jaff	3d0ece828a	(feat) show semantic-cache on health/readiness	2024-02-06 13:35:34 -08:00
ishaan-jaff	05f379234d	allow setting redis_semantic cache_embedding model	2024-02-06 10:22:02 -08:00
ishaan-jaff	751fb1af89	(feat) log semantic_sim to langfuse	2024-02-06 09:31:57 -08:00
ishaan-jaff	6249a97098	(feat) working semantic-cache on litellm proxy	2024-02-06 08:52:57 -08:00
ishaan-jaff	76def20ffe	(feat) RedisSemanticCache - async	2024-02-06 08:13:12 -08:00
ishaan-jaff	ccc94128d3	(fix) semantic cache	2024-02-05 18:25:22 -08:00
ishaan-jaff	1b39454a08	(feat) working - sync semantic caching	2024-02-05 17:58:12 -08:00
ishaan-jaff	d4a799a3ca	(feat )add semantic cache	2024-02-05 12:28:21 -08:00
Krish Dholakia	9ab59045a3	Merge branch 'main' into litellm_embedding_caching_updates	2024-02-03 18:08:47 -08:00
ishaan-jaff	4d6ffe4400	(feat) - cache - add delete cache	2024-02-02 18:36:51 -08:00