litellm-mirror/docs/my-website/docs
Krish Dholakia b682dc4ec8
Add cost tracking for rerank via bedrock (#8691)
* feat(bedrock/rerank): infer model region if model given as arn

* test: add unit testing to ensure bedrock region name inferred from arn on rerank

* feat(bedrock/rerank/transformation.py): include search units for bedrock rerank result

Resolves https://github.com/BerriAI/litellm/issues/7258#issuecomment-2671557137

* test(test_bedrock_completion.py): add testing for bedrock cohere rerank

* feat(cost_calculator.py): refactor rerank cost tracking to support bedrock cost tracking

* build(model_prices_and_context_window.json): add amazon.rerank model to model cost map

* fix(cost_calculator.py): bedrock/common_utils.py

get base model from model w/ arn -> handles rerank model

* build(model_prices_and_context_window.json): add bedrock cohere rerank pricing

* feat(bedrock/rerank): migrate bedrock config to basererank config

* Revert "feat(bedrock/rerank): migrate bedrock config to basererank config"

This reverts commit 84fae1f167.

* test: add testing to ensure large doc / queries are correctly counted

* Revert "test: add testing to ensure large doc / queries are correctly counted"

This reverts commit 4337f1657e.

* fix(migrate-jina-ai-to-rerank-config): enables cost tracking

* refactor(jina_ai/): finish migrating jina ai to base rerank config

enables cost tracking

* fix(jina_ai/rerank): e2e jina ai rerank cost tracking

* fix: cleanup dead code

* fix: fix python3.8 compatibility error

* test: fix test

* test: add e2e testing for azure ai rerank

* fix: fix linting error

* test: mark cohere as flaky
2025-02-20 21:00:18 -08:00
..
adding_provider Litellm dev 12 25 2025 p2 (#7420) 2024-12-25 18:35:34 -08:00
caching (refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208) 2024-10-14 16:34:01 +05:30
completion Added compatibility guidance, etc. for xAI Grok model (#8282) 2025-02-05 17:21:47 -08:00
debugging Update local_debugging.md (#8308) 2025-02-06 16:19:32 -08:00
embedding (doc) Add nvidia as provider (#8023) 2025-01-27 21:18:34 -08:00
extras docs(code_quality.md): add doc on litellm code qa 2024-10-02 11:20:15 -04:00
guides LiteLLM Minor Fixes & Improvements (11/23/2024) (#6870) 2024-11-23 15:17:40 +05:30
langchain Update MLflow calllback and documentation (#7809) 2025-01-21 20:56:48 -08:00
observability Added custom_attributes to additional_keys which can be sent to athina (#8518) 2025-02-13 13:19:24 -08:00
pass_through docs assembly ai 2025-02-06 21:30:36 -08:00
projects Add Elroy to projects built with litellm (#8642) 2025-02-18 16:43:30 -08:00
providers Add cost tracking for rerank via bedrock (#8691) 2025-02-20 21:00:18 -08:00
proxy [Feature]: Redis Caching - Allow setting a namespace for redis cache (#8624) 2025-02-18 14:47:34 -08:00
tutorials Update MLflow calllback and documentation (#7809) 2025-01-21 20:56:48 -08:00
assistants.md [Feat-Proxy] Add Azure Assistants API - Create Assistant, Delete Assistant Support (#5777) 2024-09-18 16:27:33 -07:00
audio_transcription.md docs(deepgram.md): add deepgram model support to docs 2024-12-28 20:19:12 -08:00
batches.md docs - batches cost tracking (#7422) 2024-12-25 20:13:26 -08:00
benchmarks.md docs benchmark 2025-01-14 10:48:43 -08:00
budget_manager.md docs - use consistent name for LiteLLM proxy server 2024-08-03 12:54:07 -07:00
contact.md
contributing.md docs(contributing.md): add docs for contributing to litellm ui 2024-02-23 13:01:34 -08:00
data_retention.md docs - Custom Retention Policies 2025-01-20 07:29:48 -08:00
data_security.md docs data sec 2025-01-18 17:44:02 -08:00
default_code_snippet.md
enterprise.md docs(enterprise.md): cleanup docs and add faq 2025-01-11 10:46:55 -08:00
exception_mapping.md Litellm dev 10 26 2024 (#6472) 2024-10-28 15:05:43 -07:00
files_endpoints.md docs files api 2024-12-24 20:46:43 -08:00
fine_tuning.md (feat) POST /fine_tuning/jobs support passing vertex specific hyper params (#7490) 2025-01-01 07:44:48 -08:00
getting_started.md Update MLflow calllback and documentation (#7809) 2025-01-21 20:56:48 -08:00
hosted.md docs gdpr regions cloud 2024-07-10 18:00:16 -07:00
image_generation.md Litellm dev 11 21 2024 (#6837) 2024-11-22 01:53:52 +05:30
image_variations.md New stable release - release notes (#8148) 2025-01-31 10:02:59 -08:00
index.md Add xAI and fix some old model config (#8218) 2025-02-03 15:29:19 -08:00
load_test.md (docs) add 1k rps load test doc (#6059) 2024-10-04 16:56:34 +05:30
load_test_advanced.md update load test docs 2025-01-08 12:48:21 -08:00
load_test_rpm.md (docs) add 1k rps load test doc (#6059) 2024-10-04 16:56:34 +05:30
load_test_sdk.md (docs) add 1k rps load test doc (#6059) 2024-10-04 16:56:34 +05:30
migration.md
migration_policy.md docs migration policy 2024-08-09 18:06:37 -07:00
moderation.md (docs + fix) Add docs on Moderations endpoint, Text Completion (#6947) 2024-11-27 16:30:48 -08:00
oidc.md (oidc): Improve docs for unofficial provider. 2024-08-16 20:30:41 +00:00
old_guardrails.md docs move lakera to free 2024-08-20 16:38:37 -07:00
projects.md
prompt_injection.md docs(sidebars.js): refactor docs 2024-08-22 18:22:50 -07:00
proxy_api.md
proxy_server.md docs - use consistent name for LiteLLM proxy server 2024-08-03 12:54:07 -07:00
realtime.md Litellm dev 10 22 2024 (#6384) 2024-10-22 21:18:54 -07:00
rerank.md docs infinity rerank api docs 2024-12-19 18:51:55 -08:00
router_architecture.md docs(router_architecture.md): add router architecture docs 2024-11-26 12:54:38 +05:30
routing.md docs(routing.md): add section on weighted deployments 2025-02-17 17:02:06 -08:00
rules.md
scheduler.md feat(router.py): support request prioritization for text completion c… (#7540) 2025-01-03 19:35:44 -08:00
sdk_custom_pricing.md organize docs 2024-08-03 12:54:07 -07:00
secret.md Litellm dev 01 10 2025 p2 (#7679) 2025-01-10 21:50:53 -08:00
set_keys.md Added compatibility guidance, etc. for xAI Grok model (#8282) 2025-02-05 17:21:47 -08:00
simple_proxy_old_doc.md (docs + testing) Correctly document the timeout value used by litellm proxy is 6000 seconds + add to best practices for prod (#6339) 2024-10-23 14:09:35 +05:30
text_completion.md (docs + fix) Add docs on Moderations endpoint, Text Completion (#6947) 2024-11-27 16:30:48 -08:00
text_to_speech.md docs tts 2024-08-23 18:29:36 -07:00
troubleshoot.md docs(troubleshoot.md): cleanup docs 2024-05-27 09:51:35 -07:00
wildcard_routing.md Litellm dev 12 28 2024 p2 (#7458) 2024-12-28 19:38:06 -08:00