mirror of https://github.com/BerriAI/litellm.git synced 2025-04-24 18:24:20 +00:00

History

Ishaan Jaff a40ecc3fe4 fix import loc		2025-04-23 17:33:29 -07:00
..
__init__.py	(Redis Cluster) - Fixes for using redis cluster + pipeline (#8442 )	2025-02-12 18:01:32 -08:00
_internal_lru_cache.py	(litellm SDK perf improvements) - handle cases when unable to lookup model in model cost map (#7750 )	2025-01-13 19:58:46 -08:00
base_cache.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
caching.py	Squashed commit of the following: (#9709 )	2025-04-02 21:24:54 -07:00
caching_handler.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
disk_cache.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
dual_cache.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
in_memory_cache.py	Squashed commit of the following: (#9709 )	2025-04-02 21:24:54 -07:00
llm_caching_handler.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
qdrant_semantic_cache.py	fix import loc	2025-04-23 17:33:29 -07:00
Readme.md	(refactor) - caching use separate files for each cache class (#6251 )	2024-10-16 13:17:21 +05:30
redis_cache.py	fix async_set_cache	2025-04-02 18:51:41 -07:00
redis_cluster_cache.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00
redis_semantic_cache.py	fix import loc	2025-04-23 17:33:29 -07:00
s3_cache.py	build(pyproject.toml): add new dev dependencies - for type checking (#9631 )	2025-03-29 11:02:13 -07:00

Readme.md

Caching on LiteLLM

LiteLLM supports multiple caching mechanisms. This allows users to choose the most suitable caching solution for their use case.

The following caching mechanisms are supported:

RedisCache
RedisSemanticCache
QdrantSemanticCache
InMemoryCache
DiskCache
S3Cache
DualCache (updates both Redis and an in-memory cache simultaneously)

Folder Structure

litellm/caching/
├── base_cache.py
├── caching.py
├── caching_handler.py
├── disk_cache.py
├── dual_cache.py
├── in_memory_cache.py
├── qdrant_semantic_cache.py
├── redis_cache.py
├── redis_semantic_cache.py
├── s3_cache.py

Readme.md

Caching on LiteLLM

Folder Structure

Documentation