6 commits

Author	SHA1	Message	Date
Krish Dholakia	87733c8193	Fix anthropic prompt caching cost calc + trim logged message in db (#9838) ... * fix(spend_tracking_utils.py): prevent logging entire mp4 files to db Fixes https://github.com/BerriAI/litellm/issues/9732 * fix(anthropic/chat/transformation.py): Fix double counting cache creation input tokens Fixes https://github.com/BerriAI/litellm/issues/9812 * refactor(anthropic/chat/transformation.py): refactor streaming to use same usage calculation block as non-streaming reduce errors * fix(bedrock/chat/converse_transformation.py): don't increment prompt tokens with cache_creation_input_tokens * build: remove redisvl from requirements.txt (temporary) * fix(spend_tracking_utils.py): handle circular references * test: update code cov test * test: update test	2025-04-09 21:26:43 -07:00
Krrish Dholakia	9adad381b4	fix(common_utils.py): handle cris only model ... Fixes https://github.com/BerriAI/litellm/issues/9161#issuecomment-2734905153	2025-03-18 23:35:43 -07:00
Krrish Dholakia	8a6e4715aa	feat(converse_transformation.py): fix type for bedrock cache usage block	2025-03-13 19:33:22 -07:00
Krrish Dholakia	96bba9354e	test(tests/litellm): add unit test for transform usage function	2025-03-13 15:56:55 -07:00
Krish Dholakia	b682dc4ec8	Add cost tracking for rerank via bedrock (#8691) ... * feat(bedrock/rerank): infer model region if model given as arn * test: add unit testing to ensure bedrock region name inferred from arn on rerank * feat(bedrock/rerank/transformation.py): include search units for bedrock rerank result Resolves https://github.com/BerriAI/litellm/issues/7258#issuecomment-2671557137 * test(test_bedrock_completion.py): add testing for bedrock cohere rerank * feat(cost_calculator.py): refactor rerank cost tracking to support bedrock cost tracking * build(model_prices_and_context_window.json): add amazon.rerank model to model cost map * fix(cost_calculator.py): bedrock/common_utils.py get base model from model w/ arn -> handles rerank model * build(model_prices_and_context_window.json): add bedrock cohere rerank pricing * feat(bedrock/rerank): migrate bedrock config to basererank config * Revert "feat(bedrock/rerank): migrate bedrock config to basererank config" This reverts commit 84fae1f167. * test: add testing to ensure large doc / queries are correctly counted * Revert "test: add testing to ensure large doc / queries are correctly counted" This reverts commit 4337f1657e. * fix(migrate-jina-ai-to-rerank-config): enables cost tracking * refactor(jina_ai/): finish migrating jina ai to base rerank config enables cost tracking * fix(jina_ai/rerank): e2e jina ai rerank cost tracking * fix: cleanup dead code * fix: fix python3.8 compatibility error * test: fix test * test: add e2e testing for azure ai rerank * fix: fix linting error * test: mark cohere as flaky	2025-02-20 21:00:18 -08:00
Ishaan Jaff	300d7825f5	(Observability) - Add more detailed dd tracing on Proxy Auth, Bedrock Auth (#8693) ... * add dd tracer * fix dd tracing * add @tracer.wrap() on def user_api_key_auth * add async_function_with_retries * remove dead code * add tracer.wrap on base aws llm * add tracer.wrap on base aws llm * fix print verbose * fix dd tracing * trace base aws llm * fix test base aws llm * fix converse transform * test base aws llm * BASE_AWS_LLM_PATH * BASE_AWS_LLM_PATH * test dd tracing	2025-02-20 18:00:41 -08:00