Krrish Dholakia
|
130ef32e31
|
fix(batch_redis_get.py): handle custom namespace
Fix https://github.com/BerriAI/litellm/issues/5917
|
2024-09-28 21:08:14 -07:00 |
|
Krrish Dholakia
|
e391e30285
|
refactor: replace 'traceback.print_exc()' with logging library
allows error logs to be in json format for otel logging
|
2024-06-06 13:47:43 -07:00 |
|
Krrish Dholakia
|
31e2d4e6d1
|
feat(lowest_tpm_rpm_v2.py): move to using redis.incr and redis.mget for getting model usage from redis
makes routing work across multiple instances
|
2024-04-10 14:56:23 -07:00 |
|
Krrish Dholakia
|
3680f16cd7
|
feat(batch_redis_get.py): batch redis GET requests for a given key + call type
reduces number of redis requests. 85ms latency improvement over 3 minutes of load (19k requests).
|
2024-03-15 14:54:16 -07:00 |
|
Krrish Dholakia
|
8d1c60bfdc
|
feat(batch_redis_get.py): batch redis GET requests for a given key + call type
reduces the number of GET requests we're making in high-throughput scenarios
|
2024-03-15 14:40:11 -07:00 |
|