forked from phoenix/litellm-mirror
3beecfb0d4
* fix(utils.py): add logprobs support for together ai Fixes https://github.com/BerriAI/litellm/issues/6724 * feat(pass_through_endpoints/): add anthropic/ pass-through endpoint adds new `anthropic/` pass-through endpoint + refactors docs * feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id enables seeing spend for a customer in a team * Add integration with MLflow Tracing (#6147) * Add MLflow logger Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * Streaming handling Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * lint Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * address comments and fix issues Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * address comments and fix issues Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * Move logger construction code Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * Add docs Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * async handlers Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * new picture Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> --------- Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * fix(mlflow.py): fix ruff linting errors * ci(config.yml): add mlflow to ci testing * fix: fix test * test: fix test * Litellm key update fix (#6710) * fix(caching): convert arg to equivalent kwargs in llm caching handler prevent unexpected errors * fix(caching_handler.py): don't pass args to caching * fix(caching): remove all *args from caching.py * fix(caching): consistent function signatures + abc method * test(caching_unit_tests.py): add unit tests for llm caching ensures coverage for common caching scenarios across different implementations * refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one * fix(router.py): drop redis password requirement * fix(proxy_server.py): fix faulty slack alerting check * fix(langfuse.py): avoid copying functions/thread lock objects in metadata fixes metadata copy error when parent otel span in metadata * test: update test * fix(key_management_endpoints.py): fix /key/update with metadata update * fix(key_management_endpoints.py): fix key_prepare_update helper * fix(key_management_endpoints.py): reset value to none if set in key update * fix: update test ' * Litellm dev 11 11 2024 (#6693) * fix(__init__.py): add 'watsonx_text' as mapped llm api route Fixes https://github.com/BerriAI/litellm/issues/6663 * fix(opentelemetry.py): fix passing parallel tool calls to otel Fixes https://github.com/BerriAI/litellm/issues/6677 * refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling reduces bugs in repo * fix(__init__.py): update provider-model mapping to include all known provider-model mappings Fixes https://github.com/BerriAI/litellm/issues/6669 * feat(anthropic): support passing document in llm api call * docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function * fix(factory.py): fix linting error * add clear doc string for GCS bucket logging * Add docs to export logs to Laminar (#6674) * Add docs to export logs to Laminar * minor fix: newline at end of file * place laminar after http and grpc * (Feat) Add langsmith key based logging (#6682) * add langsmith_api_key to StandardCallbackDynamicParams * create a file for langsmith types * langsmith add key / team based logging * add key based logging for langsmith * fix langsmith key based logging * fix linting langsmith * remove NOQA violation * add unit test coverage for all helpers in test langsmith * test_langsmith_key_based_logging * docs langsmith key based logging * run langsmith tests in logging callback tests * fix logging testing * test_langsmith_key_based_logging * test_add_callback_via_key_litellm_pre_call_utils_langsmith * add debug statement langsmith key based logging * test_langsmith_key_based_logging * (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701) * use helper for _transform_messages mistral * add test_message_with_name to base LLMChat test * fix linting * add xAI on Admin UI (#6680) * (docs) add benchmarks on 1K RPS (#6704) * docs litellm proxy benchmarks * docs GCS bucket * doc fix - reduce clutter on logging doc title * (feat) add cost tracking stable diffusion 3 on Bedrock (#6676) * add cost tracking for sd3 * test_image_generation_bedrock * fix get model info for image cost * add cost_calculator for stability 1 models * add unit testing for bedrock image cost calc * test_cost_calculator_with_no_optional_params * add test_cost_calculator_basic * correctly allow size Optional * fix cost_calculator * sd3 unit tests cost calc * fix raise correct error 404 when /key/info is called on non-existent key (#6653) * fix raise correct error on /key/info * add not_found_error error * fix key not found in DB error * use 1 helper for checking token hash * fix error code on key info * fix test key gen prisma * test_generate_and_call_key_info * test fix test_call_with_valid_model_using_all_models * fix key info tests * bump: version 1.52.4 → 1.52.5 * add defaults used for GCS logging * LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705) * fix(caching): convert arg to equivalent kwargs in llm caching handler prevent unexpected errors * fix(caching_handler.py): don't pass args to caching * fix(caching): remove all *args from caching.py * fix(caching): consistent function signatures + abc method * test(caching_unit_tests.py): add unit tests for llm caching ensures coverage for common caching scenarios across different implementations * refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one * fix(router.py): drop redis password requirement * fix(proxy_server.py): fix faulty slack alerting check * fix(langfuse.py): avoid copying functions/thread lock objects in metadata fixes metadata copy error when parent otel span in metadata * test: update test * bump: version 1.52.5 → 1.52.6 * (feat) helm hook to sync db schema (#6715) * v0 migration job * fix job * fix migrations job.yml * handle standalone DB on helm hook * fix argo cd annotations * fix db migration helm hook * fix migration job * doc fix Using Http/2 with Hypercorn * (fix proxy redis) Add redis sentinel support (#6154) * add sentinel_password support * add doc for setting redis sentinel password * fix redis sentinel - use sentinel password * Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714) Fixes #6713 * (fix) using Anthropic `response_format={"type": "json_object"}` (#6721) * add support for response_format=json anthropic * add test_json_response_format to baseLLM ChatTest * fix test_litellm_anthropic_prompt_caching_tools * fix test_anthropic_function_call_with_no_schema * test test_create_json_tool_call_for_response_format * (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716) * add BaseImageGenTest * use 1 class for unit testing * add debugging to BaseImageGenTest * TestAzureOpenAIDalle3 * fix response_cost_calculator * test_basic_image_generation * fix img gen basic test * fix _select_model_name_for_cost_calc * fix test_aimage_generation_bedrock_with_optional_params * fix undo changes cost tracking * fix response_cost_calculator * fix test_cost_azure_gpt_35 * fix remove dup test (#6718) * (build) update db helm hook * (build) helm db pre sync hook * (build) helm db sync hook * test: run test_team_logging firdst --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com> Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de> * test: update test * test: skip anthropic overloaded error * test: cleanup test * test: update tests * test: fix test * test: handle gemini overloaded model error * test: handle internal server error * test: handle anthropic overloaded error * test: handle claude instability --------- Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com> Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de> |
||
|---|---|---|
| .. | ||
| .litellm_cache | ||
| example_config_yaml | ||
| test_configs | ||
| test_model_response_typing | ||
| adroit-crow-413218-bc47f303efc9.json | ||
| azure_fine_tune.jsonl | ||
| batch_job_results_furniture.jsonl | ||
| cache_unit_tests.py | ||
| conftest.py | ||
| data_map.txt | ||
| eagle.wav | ||
| gettysburg.wav | ||
| large_text.py | ||
| log.txt | ||
| messages_with_counts.py | ||
| model_cost.json | ||
| openai_batch_completions.jsonl | ||
| openai_batch_completions_router.jsonl | ||
| speech_vertex.mp3 | ||
| stream_chunk_testdata.py | ||
| test_acompletion.py | ||
| test_acooldowns_router.py | ||
| test_add_function_to_prompt.py | ||
| test_add_update_models.py | ||
| test_alangfuse.py | ||
| test_alerting.py | ||
| test_amazing_s3_logs.py | ||
| test_amazing_vertex_completion.py | ||
| test_anthropic_prompt_caching.py | ||
| test_arize_ai.py | ||
| test_assistants.py | ||
| test_async_fn.py | ||
| test_audio_speech.py | ||
| test_auth_checks.py | ||
| test_auth_utils.py | ||
| test_aws_secret_manager.py | ||
| test_azure_content_safety.py | ||
| test_azure_openai.py | ||
| test_azure_perf.py | ||
| test_bad_params.py | ||
| test_batch_completion_return_exceptions.py | ||
| test_batch_completions.py | ||
| test_blocked_user_list.py | ||
| test_braintrust.py | ||
| test_budget_manager.py | ||
| test_caching.py | ||
| test_caching_handler.py | ||
| test_caching_ssl.py | ||
| test_clarifai_completion.py | ||
| test_class.py | ||
| test_cohere_completion.py | ||
| test_completion.py | ||
| test_completion_cost.py | ||
| test_completion_with_retries.py | ||
| test_config.py | ||
| test_cost_calc.py | ||
| test_custom_api_logger.py | ||
| test_custom_callback_input.py | ||
| test_custom_callback_router.py | ||
| test_custom_llm.py | ||
| test_custom_logger.py | ||
| test_datadog.py | ||
| test_disk_cache_unit_tests.py | ||
| test_dual_cache.py | ||
| test_dynamic_rate_limit_handler.py | ||
| test_dynamodb_logs.py | ||
| test_embedding.py | ||
| test_exceptions.py | ||
| test_file_types.py | ||
| test_fine_tuning_api.py | ||
| test_function_call_parsing.py | ||
| test_function_calling.py | ||
| test_function_setup.py | ||
| test_gcs_bucket.py | ||
| test_get_llm_provider.py | ||
| test_get_model_file.py | ||
| test_get_model_info.py | ||
| test_get_optional_params_embeddings.py | ||
| test_get_optional_params_functions_not_supported.py | ||
| test_get_secret.py | ||
| test_google_ai_studio_gemini.py | ||
| test_guardrails_ai.py | ||
| test_guardrails_config.py | ||
| test_health_check.py | ||
| test_helicone_integration.py | ||
| test_hf_prompt_templates.py | ||
| test_img_resize.py | ||
| test_lakera_ai_prompt_injection.py | ||
| test_langchain_ChatLiteLLM.py | ||
| test_langsmith.py | ||
| test_least_busy_routing.py | ||
| test_litellm_max_budget.py | ||
| test_literalai.py | ||
| test_llm_guard.py | ||
| test_load_test_router_s3.py | ||
| test_loadtest_router.py | ||
| test_logfire.py | ||
| test_logging.py | ||
| test_longer_context_fallback.py | ||
| test_lowest_cost_routing.py | ||
| test_lowest_latency_routing.py | ||
| test_lunary.py | ||
| test_max_tpm_rpm_limiter.py | ||
| test_mem_leak.py | ||
| test_mem_usage.py | ||
| test_mock_request.py | ||
| test_model_alias_map.py | ||
| test_model_max_token_adjust.py | ||
| test_multiple_deployments.py | ||
| test_ollama.py | ||
| test_ollama_local.py | ||
| test_ollama_local_chat.py | ||
| test_openai_batches_and_files.py | ||
| test_openai_moderations_hook.py | ||
| test_opik.py | ||
| test_parallel_request_limiter.py | ||
| test_pass_through_endpoints.py | ||
| test_presidio_masking.py | ||
| test_profiling_router.py | ||
| test_prometheus.py | ||
| test_prometheus_service.py | ||
| test_prompt_caching.py | ||
| test_prompt_injection_detection.py | ||
| test_promptlayer_integration.py | ||
| test_provider_specific_config.py | ||
| test_pydantic.py | ||
| test_pydantic_namespaces.py | ||
| test_python_38.py | ||
| test_register_model.py | ||
| test_rerank.py | ||
| test_router.py | ||
| test_router_batch_completion.py | ||
| test_router_caching.py | ||
| test_router_client_init.py | ||
| test_router_cooldowns.py | ||
| test_router_custom_routing.py | ||
| test_router_debug_logs.py | ||
| test_router_fallback_handlers.py | ||
| test_router_fallbacks.py | ||
| test_router_get_deployments.py | ||
| test_router_init.py | ||
| test_router_max_parallel_requests.py | ||
| test_router_pattern_matching.py | ||
| test_router_policy_violation.py | ||
| test_router_retries.py | ||
| test_router_tag_routing.py | ||
| test_router_timeout.py | ||
| test_router_utils.py | ||
| test_router_with_fallbacks.py | ||
| test_rules.py | ||
| test_sagemaker.py | ||
| test_scheduler.py | ||
| test_secret_detect_hook.py | ||
| test_secret_manager.py | ||
| test_simple_shuffle.py | ||
| test_spend_calculate_endpoint.py | ||
| test_spend_logs.py | ||
| test_stream_chunk_builder.py | ||
| test_streaming.py | ||
| test_supabase_integration.py | ||
| test_team_config.py | ||
| test_text_completion.py | ||
| test_timeout.py | ||
| test_together_ai.py | ||
| test_token_counter.py | ||
| test_tpm_rpm_routing_v2.py | ||
| test_traceloop.py | ||
| test_triton.py | ||
| test_ui_sso_helper_utils.py | ||
| test_unit_test_caching.py | ||
| test_update_spend.py | ||
| test_user_api_key_auth.py | ||
| test_utils.py | ||
| test_validate_environment.py | ||
| test_wandb.py | ||
| test_whisper.py | ||
| user_cost.json | ||
| vertex_ai.jsonl | ||
| vertex_key.json | ||