Commit graph

6 commits

Author SHA1 Message Date
Krish Dholakia
bd0ba63ac0 Litellm dev 12 26 2024 p4 (#7439)
* fix(model_dashboard.tsx): support setting model_info params - e.g. mode on ui

Closes https://github.com/BerriAI/litellm/issues/5270

* fix(lowest_tpm_rpm_v2.py): deployment rpm over limit check

fixes selection error when getting potential deployments below known tpm/rpm limit

 Fixes https://github.com/BerriAI/litellm/issues/7395

* fix(test_tpm_rpm_routing_v2.py): add unit test for https://github.com/BerriAI/litellm/issues/7395

* fix(lowest_tpm_rpm_v2.py): fix tpm key name in dict post rpm update

* test: rename test to run earlier

* test: skip flaky test
2024-12-27 12:01:42 -08:00
Ishaan Jaff
2301858379 test_router_get_available_deployments 2024-12-23 18:21:27 -08:00
Ishaan Jaff
5ba0fb27d8 [Bug Fix]: Errors in LiteLLM When Using Embeddings Model with Usage-Based Routing (#7390)
* use slp for usage based routing v2

* update error msg

* fix usage based routing v2

* test_tpm_rpm_updated

* fix unused imports

* fix unused imports
2024-12-23 17:42:24 -08:00
Krish Dholakia
345a70bdcd LiteLLM Minor Fixes & Improvements (11/26/2024) (#6913)
* docs(config_settings.md): document all router_settings

* ci(config.yml): add router_settings doc test to ci/cd

* test: debug test on ci/cd

* test: debug ci/cd test

* test: fix test

* fix(team_endpoints.py): skip invalid team object. don't fail `/team/list` call

Causes downstream errors if ui just fails to load team list

* test(base_llm_unit_tests.py): add 'response_format={"type": "text"}' test to base_llm_unit_tests

adds complete coverage for all 'response_format' values to ci/cd

* feat(router.py): support wildcard routes in `get_router_model_info()`

Addresses https://github.com/BerriAI/litellm/issues/6914

* build(model_prices_and_context_window.json): add tpm/rpm limits for all gemini models

Allows for ratelimit tracking for gemini models even with wildcard routing enabled

Addresses https://github.com/BerriAI/litellm/issues/6914

* feat(router.py): add tpm/rpm tracking on success/failure to global_router

Addresses https://github.com/BerriAI/litellm/issues/6914

* feat(router.py): support wildcard routes on router.get_model_group_usage()

* fix(router.py): fix linting error

* fix(router.py): implement get_remaining_tokens_and_requests

Addresses https://github.com/BerriAI/litellm/issues/6914

* fix(router.py): fix linting errors

* test: fix test

* test: fix tests

* docs(config_settings.md): add missing dd env vars to docs

* fix(router.py): check if hidden params is dict
2024-11-28 00:01:38 +05:30
Ishaan Jaff
ba56e37244 (refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208)
* use folder for caching

* fix importing caching

* fix clickhouse pyright

* fix linting

* fix correctly pass kwargs and args

* fix test case for embedding

* fix linting

* fix embedding caching logic

* fix refactor handle utils.py

* fix test_embedding_caching_azure_individual_items_reordered
2024-10-14 16:34:01 +05:30
Krrish Dholakia
ea96eebe85 refactor: move all testing to top-level of repo
Closes https://github.com/BerriAI/litellm/issues/486
2024-09-28 21:08:14 -07:00
Renamed from litellm/tests/test_tpm_rpm_routing_v2.py (Browse further)