9 commits

Author	SHA1	Message	Date
Ishaan Jaff	f724f3131d	(testing) add unit tests for LLMCachingHandler Class (#6279) ... * add unit testing for test_async_set_cache * test test_async_log_cache_hit_on_callbacks * assert the correct response type is returned * test_convert_cached_result_to_model_response * unit testing for caching handler	2024-10-17 19:12:57 +05:30
Krish Dholakia	38a9a106d2	LiteLLM Minor Fixes & Improvements (10/16/2024) (#6265) ... * fix(caching_handler.py): handle positional arguments in add cache logic Fixes https://github.com/BerriAI/litellm/issues/6264 * feat(litellm_pre_call_utils.py): allow forwarding openai org id to backend client https://github.com/BerriAI/litellm/issues/6237 * docs(configs.md): add 'forward_openai_org_id' to docs * fix(proxy_server.py): return model info if user_model is set Fixes https://github.com/BerriAI/litellm/issues/6233 * fix(hosted_vllm/chat/transformation.py): don't set tools unless non-none * fix(openai.py): improve debug log for openai 'str' error Addresses https://github.com/BerriAI/litellm/issues/6272 * fix(proxy_server.py): fix linting error * fix(proxy_server.py): fix linting errors * test: skip WIP test * docs(openai.md): add docs on passing openai org id from client to openai	2024-10-16 22:16:23 -07:00
Ishaan Jaff	d9a71650e3	(refactor) - caching use separate files for each cache class (#6251) ... * fix remove qdrant semantic caching to it's own folder * refactor use 1 file for s3 caching * fix use sep files for in mem and redis caching * fix refactor caching * add readme.md for caching folder	2024-10-16 13:17:21 +05:30
Ishaan Jaff	97ba4eea7d	(refactor) sync caching - use LLMCachingHandler class for get_cache (#6249) ... * caching - use _sync_set_cache * add sync _sync_add_streaming_response_to_cache * use caching class for cache storage * fix use _sync_get_cache * fix circular import * use _update_litellm_logging_obj_environment * use one helper for _process_async_embedding_cached_response * fix _is_call_type_supported_by_cache * fix checking cache * fix sync get cache * fix use _combine_cached_embedding_response_with_api_result * fix _update_litellm_logging_obj_environment * adjust test_redis_cache_acompletion_stream_bedrock	2024-10-16 12:33:49 +05:30
Ishaan Jaff	da6a7c3a55	(refactor) caching - use _sync_set_cache (#6224) ... * caching - use _sync_set_cache * add sync _sync_add_streaming_response_to_cache * use caching class for cache storage	2024-10-16 10:38:07 +05:30
Ishaan Jaff	a69c670baa	(refactor) use helper function _assemble_complete_response_from_streaming_chunks to assemble complete responses in caching and logging callbacks (#6220) ... * (refactor) use _assemble_complete_response_from_streaming_chunks * add unit test for test_assemble_complete_response_from_streaming_chunks_1 * fix assemble complete_streaming_response * config add logging_testing * add logging_coverage in codecov * test test_assemble_complete_response_from_streaming_chunks_3 * add unit tests for _assemble_complete_response_from_streaming_chunks * fix remove unused / junk function * add test for streaming_chunks when error assembling	2024-10-15 12:45:12 +05:30
Ishaan Jaff	4ebe6813c0	(refactor caching) use common _retrieve_from_cache helper (#6212) ... * use folder for caching * fix importing caching * fix clickhouse pyright * fix linting * fix correctly pass kwargs and args * fix test case for embedding * fix linting * fix embedding caching logic * fix refactor handle utils.py * refactor async set stream cache * fix linting * refactor - use _retrieve_from_cache * refactor use _convert_cached_result_to_model_response * fix linting errors	2024-10-14 19:12:41 +05:30
Ishaan Jaff	d1bef4ad81	(refactor caching) use LLMCachingHandler for caching streaming responses (#6210) ... * use folder for caching * fix importing caching * fix clickhouse pyright * fix linting * fix correctly pass kwargs and args * fix test case for embedding * fix linting * fix embedding caching logic * fix refactor handle utils.py * refactor async set stream cache * fix linting	2024-10-14 17:46:45 +05:30
Ishaan Jaff	4d1b4beb3d	(refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208) ... * use folder for caching * fix importing caching * fix clickhouse pyright * fix linting * fix correctly pass kwargs and args * fix test case for embedding * fix linting * fix embedding caching logic * fix refactor handle utils.py * fix test_embedding_caching_azure_individual_items_reordered	2024-10-14 16:34:01 +05:30