feat(cache): add cache store abstraction layer

Implement reusable cache store abstraction with in-memory and Redis
backends as foundation for prompt caching feature (PR1 of progressive delivery).

- Add CacheStore protocol defining cache interface
- Implement MemoryCacheStore with LRU, LFU, and TTL-only eviction policies
- Implement RedisCacheStore with connection pooling and retry logic
- Add CircuitBreaker for cache backend failure protection
- Include comprehensive unit tests (55 tests, >80% coverage)
- Add dependencies: cachetools>=5.5.0, redis>=5.2.0

This abstraction enables flexible caching implementations for the prompt
caching middleware without coupling to specific storage backends.

Signed-by: William Caban <willliam.caban@gmail.com>
This commit is contained in:
William Caban 2025-11-15 14:45:49 -05:00
parent 97f535c4f1
commit 299c575daa
10 changed files with 2175 additions and 1 deletions

View file

@ -26,6 +26,7 @@ classifiers = [
dependencies = [
"PyYAML>=6.0",
"aiohttp",
"cachetools>=5.5.0", # for prompt caching
"fastapi>=0.115.0,<1.0", # server
"fire", # for MCP in LLS client
"httpx",
@ -37,6 +38,7 @@ dependencies = [
"python-dotenv",
"pyjwt[crypto]>=2.10.0", # Pull crypto to support RS256 for jwt. Requires 2.10.0+ for ssl_context support.
"pydantic>=2.11.9",
"redis>=5.2.0", # for prompt caching (Redis backend)
"rich",
"starlette",
"termcolor",