(testing) add unit tests for LLMCachingHandler Class (#6279)

* add unit testing for test_async_set_cache

* test test_async_log_cache_hit_on_callbacks

* assert the correct response type is returned

* test_convert_cached_result_to_model_response

* unit testing for caching handler
This commit is contained in:
Ishaan Jaff 2024-10-17 19:12:57 +05:30 committed by GitHub
parent 202b5cc2cd
commit f724f3131d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 361 additions and 2 deletions

View file

@ -512,7 +512,16 @@ class LLMCachingHandler:
model: str,
args: Tuple[Any, ...],
custom_llm_provider: Optional[str] = None,
) -> Optional[Any]:
) -> Optional[
Union[
ModelResponse,
TextCompletionResponse,
EmbeddingResponse,
RerankResponse,
TranscriptionResponse,
CustomStreamWrapper,
]
]:
"""
Internal method to process the cached result