Krrish Dholakia
3560f0ef2c
refactor: move all testing to top-level of repo
...
Closes https://github.com/BerriAI/litellm/issues/486
2024-09-28 21:08:14 -07:00
Krrish Dholakia
efc06d4a03
fix(batch_redis_get.py): handle custom namespace
...
Fix https://github.com/BerriAI/litellm/issues/5917
2024-09-28 21:08:14 -07:00
Ishaan Jaff
7cbcf538c6
[Feat] Improve OTEL Tracking - Require all Redis Cache reads to be logged on OTEL ( #5881 )
...
* fix use previous internal usage caching logic
* fix test_dual_cache_uses_redis
* redis track event_metadata in service logging
* show otel error on _get_parent_otel_span_from_kwargs
* track parent otel span on internal usage cache
* update_request_status
* fix internal usage cache
* fix linting
* fix test internal usage cache
* fix linting error
* show event metadata in redis set
* fix test_get_team_redis
* fix test_get_team_redis
* test_proxy_logging_setup
2024-09-25 10:57:08 -07:00
Ishaan Jaff
2000e8cde9
[Perf Fix] Don't always read from Redis by Default ( #5877 )
...
* fix use previous internal usage caching logic
* fix test_dual_cache_uses_redis
2024-09-24 21:34:18 -07:00
Krish Dholakia
d46660ea0f
LiteLLM Minor Fixes & Improvements (09/18/2024) ( #5772 )
...
* fix(proxy_server.py): fix azure key vault logic to not require client id/secret
* feat(cost_calculator.py): support fireworks ai cost tracking
* build(docker-compose.yml): add lines for mounting config.yaml to docker compose
Closes https://github.com/BerriAI/litellm/issues/5739
* fix(input.md): update docs to clarify litellm supports content as a list of dictionaries
Fixes https://github.com/BerriAI/litellm/issues/5755
* fix(input.md): update input.md to include all message values
* fix(image_handling.py): follow image url redirects
Fixes https://github.com/BerriAI/litellm/issues/5763
* fix(router.py): Fix model key/base leak in error message
Fixes https://github.com/BerriAI/litellm/issues/5762
* fix(http_handler.py): fix linting error
* fix(azure.py): fix logging to show azure_ad_token being used
Fixes https://github.com/BerriAI/litellm/issues/5767
* fix(_redis.py): add redis sentinel support
Closes https://github.com/BerriAI/litellm/issues/4381
* feat(_redis.py): add redis sentinel support
Closes https://github.com/BerriAI/litellm/issues/4381
* test(test_completion_cost.py): fix test
* Databricks Integration: Integrate Databricks SDK as optional mechanism for fetching API base and token, if unspecified (#5746 )
* LiteLLM Minor Fixes & Improvements (09/16/2024) (#5723 )
* coverage (#5713 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Move (#5714 )
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix(litellm_logging.py): fix logging client re-init (#5710 )
Fixes https://github.com/BerriAI/litellm/issues/5695
* fix(presidio.py): Fix logging_hook response and add support for additional presidio variables in guardrails config
Fixes https://github.com/BerriAI/litellm/issues/5682
* feat(o1_handler.py): fake streaming for openai o1 models
Fixes https://github.com/BerriAI/litellm/issues/5694
* docs: deprecated traceloop integration in favor of native otel (#5249 )
* fix: fix linting errors
* fix: fix linting errors
* fix(main.py): fix o1 import
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view (#5730 )
* feat(spend_management_endpoints.py): expose `/global/spend/refresh` endpoint for updating material view
Supports having `MonthlyGlobalSpend` view be a material view, and exposes an endpoint to refresh it
* fix(custom_logger.py): reset calltype
* fix: fix linting errors
* fix: fix linting error
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix: fix import
* Fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* DB test
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* Coverage
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* progress
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix
Signed-off-by: dbczumar <corey.zumar@databricks.com>
* fix test name
Signed-off-by: dbczumar <corey.zumar@databricks.com>
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
* test: fix test
* test(test_databricks.py): fix test
* fix(databricks/chat.py): handle custom endpoint (e.g. sagemaker)
* Apply code scanning fix for clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix(__init__.py): fix known fireworks ai models
---------
Signed-off-by: dbczumar <corey.zumar@databricks.com>
Co-authored-by: Corey Zumar <39497902+dbczumar@users.noreply.github.com>
Co-authored-by: Nir Gazit <nirga@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2024-09-19 13:25:29 -07:00
Ishaan Jaff
3bf2c06e06
add config for setting up redis cluster
2024-09-07 09:37:23 -07:00
Ishaan Jaff
eaab0e761d
add test_redis_cache_cluster_init_with_env_vars_unit_test
2024-09-07 08:56:00 -07:00
Ishaan Jaff
9c573cda0a
fix test_proxy_logging_setup
2024-09-04 10:59:15 -07:00
Ishaan Jaff
db0ad40e98
test for pl obj
2024-09-04 08:00:49 -07:00
Ishaan Jaff
657b7eef82
add always read redis test
2024-09-04 07:57:23 -07:00
Ishaan Jaff
5f025390bb
test test_image_generation_azure_dall_e_3
2024-08-27 22:52:11 -07:00
Ishaan Jaff
feb354d3bc
fix should_use_cache
2024-08-24 09:37:41 -07:00
Ishaan Jaff
01adc45f0f
test caching default on /off
2024-08-24 09:06:59 -07:00
Krrish Dholakia
f24075bcaf
test(test_caching.py): skip local test
2024-08-21 15:05:18 -07:00
Krrish Dholakia
e2d7539690
feat(caching.py): redis cluster support
...
Closes https://github.com/BerriAI/litellm/issues/4358
2024-08-21 15:01:52 -07:00
Ishaan Jaff
e7ecb2fe3a
fix qdrant litellm on proxy
2024-08-21 12:52:29 -07:00
Ishaan Jaff
228a0bd6f7
fix qdrant semantic caching test
2024-08-21 12:11:49 -07:00
Ishaan Jaff
428a74be07
fix drant url
2024-08-21 12:09:09 -07:00
Ishaan Jaff
7d0196191f
Merge pull request #5018 from haadirakhangi/main
...
Qdrant Semantic Caching
2024-08-21 08:50:43 -07:00
Haadi Rakhangi
df5074da56
added testing for qdrant semantic caching
2024-08-20 00:29:50 +05:30
Krrish Dholakia
3cafebbc65
test(test_caching.py): re-introduce testing for s3 cache w/ streaming
...
Closes https://github.com/BerriAI/litellm/issues/3268
2024-08-19 10:56:48 -07:00
Krrish Dholakia
aef25d5d00
fix: cleanup test
2024-08-05 11:23:49 -07:00
Krrish Dholakia
a9fdfb5a99
fix(init.py): rename feature_flag
2024-08-05 11:23:20 -07:00
Krrish Dholakia
3c4c78a71f
feat(caching.py): enable caching on provider-specific optional params
...
Closes https://github.com/BerriAI/litellm/issues/5049
2024-08-05 11:18:59 -07:00
Krrish Dholakia
d2e64f21f3
fix(litellm_logging.py): fix async caching for sync streaming calls (don't do it)
...
Checks if call is async before running async caching for streaming call
Fixes https://github.com/BerriAI/litellm/issues/4511#issuecomment-2233211808
2024-07-17 11:15:30 -07:00
Krrish Dholakia
3f83e8a8d4
fix(caching.py): fix async redis health check
2024-07-06 09:14:29 -07:00
Krrish Dholakia
606d04b05b
fix(_service_logging.py): only trigger otel if in service_callback
...
Fixes https://github.com/BerriAI/litellm/issues/4511
2024-07-03 09:48:38 -07:00
Antonio Loison
7ee07cd961
test(test_caching.py): use mock_response in disk cache test
2024-05-10 11:00:18 +02:00
Antonio Loison
ac27f431a4
test(test_caching.py): add disk cache test when using completion
2024-05-10 10:03:38 +02:00
Krrish Dholakia
d67e47d7fd
fix(test_caching.py): add longer delay for async test
2024-04-23 16:13:03 -07:00
Krrish Dholakia
161e836427
fix(utils.py): fix 'no-cache': true when caching is turned on
2024-04-23 12:58:30 -07:00
Krish Dholakia
6d9f0f1839
Merge branch 'main' into litellm_ssl_caching_fix
2024-04-19 17:20:27 -07:00
Krrish Dholakia
978c1a1976
test(test_caching.py): add sleep
2024-04-19 17:02:15 -07:00
Krrish Dholakia
01a1a8f731
fix(caching.py): dual cache async_batch_get_cache fix + testing
...
this fixes a bug in usage-based-routing-v2 which was caused b/c of how the result was being returned from dual cache async_batch_get_cache. it also adds unit testing for that function (and it's sync equivalent)
2024-04-19 15:03:25 -07:00
Ishaan Jaff
f617f5ebb5
fix - test caching atext_completion
2024-04-12 20:37:56 -07:00
Ishaan Jaff
11cd1ec6cf
test - atext_completion + caching
2024-04-12 12:32:21 -07:00
Ishaan Jaff
8bc02b34c2
test -base64 cache hits
2024-04-10 16:46:56 -07:00
Krrish Dholakia
48bfc45cb0
fix(utils.py): fix reordering of items for cached embeddings
...
ensures cached embedding item is returned in correct order
2024-04-08 12:18:24 -07:00
Krrish Dholakia
2472311a3f
test(test_caching.py): skip test - aws suspended account
...
will need to recreate these objects on a new aws account
2024-04-04 15:07:19 -07:00
Krish Dholakia
eb859f5dba
Merge pull request #2692 from BerriAI/litellm_streaming_fixes
...
fix(utils.py): ensure last chunk is always empty delta w/ finish reason
2024-03-25 21:57:04 -07:00
Krrish Dholakia
643fd6ac96
test(test_caching.py): fix test_redis_cache_acompletion_stream
2024-03-25 21:36:47 -07:00
Ishaan Jaff
07fe08d8b5
(test) no cache hit
2024-03-25 18:56:36 -07:00
Ishaan Jaff
3fcab0137a
(test) batch writing to cache
2024-03-25 18:04:04 -07:00
Krrish Dholakia
591a0a376e
fix(caching.py): support default ttl for caching
2024-03-25 13:40:17 -07:00
Krrish Dholakia
03acc07380
fix(caching.py): pass redis kwargs to connection pool init
2024-03-18 08:21:36 -07:00
Krrish Dholakia
3072137739
test(test_caching.py): fix async tests
2024-03-15 18:09:25 -07:00
Krrish Dholakia
4885bff9e3
test: reintegrate s3 testing
2024-03-07 08:56:59 -08:00
Krish Dholakia
06bde2b8c0
Merge pull request #2379 from BerriAI/litellm_s3_bucket_folder_path
...
fix(caching.py): add s3 path as a top-level param
2024-03-06 19:35:46 -08:00
Krrish Dholakia
726dad5756
fix(caching.py): add s3 path as a top-level param
2024-03-06 18:07:28 -08:00
Krrish Dholakia
8a4a14cc95
test(test_caching.py): fix test to check on id
2024-03-05 21:12:50 -08:00
