Ishaan Jaff
|
4d1b4beb3d
|
(refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208)
* use folder for caching
* fix importing caching
* fix clickhouse pyright
* fix linting
* fix correctly pass kwargs and args
* fix test case for embedding
* fix linting
* fix embedding caching logic
* fix refactor handle utils.py
* fix test_embedding_caching_azure_individual_items_reordered
|
2024-10-14 16:34:01 +05:30 |
|
Krrish Dholakia
|
b590e6607c
|
feat(scheduler.py): support redis caching for req. prioritization
enables req. prioritization to work across multiple instances of litellm
|
2024-06-06 14:19:21 -07:00 |
|
Krrish Dholakia
|
5e85a5b208
|
fix(test_scheduler.py): simplify scheduler testing. fix race condition
|
2024-06-01 18:57:47 -07:00 |
|
Krrish Dholakia
|
7715267989
|
fix(router.py): simplify scheduler
move the scheduler poll queuing logic into the router class, making it easier to use
|
2024-06-01 16:09:57 -07:00 |
|
Krrish Dholakia
|
f8d4be710e
|
docs(scheduler.md): add request prioritization to docs
|
2024-05-31 19:35:47 -07:00 |
|