forked from phoenix/litellm-mirror
* fix(utils.py): add logprobs support for together ai Fixes https://github.com/BerriAI/litellm/issues/6724 * feat(pass_through_endpoints/): add anthropic/ pass-through endpoint adds new `anthropic/` pass-through endpoint + refactors docs * feat(spend_management_endpoints.py): allow /global/spend/report to query team + customer id enables seeing spend for a customer in a team * Add integration with MLflow Tracing (#6147) * Add MLflow logger Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * Streaming handling Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * lint Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * address comments and fix issues Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * address comments and fix issues Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * Move logger construction code Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * Add docs Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * async handlers Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * new picture Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> --------- Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> * fix(mlflow.py): fix ruff linting errors * ci(config.yml): add mlflow to ci testing * fix: fix test * test: fix test * Litellm key update fix (#6710) * fix(caching): convert arg to equivalent kwargs in llm caching handler prevent unexpected errors * fix(caching_handler.py): don't pass args to caching * fix(caching): remove all *args from caching.py * fix(caching): consistent function signatures + abc method * test(caching_unit_tests.py): add unit tests for llm caching ensures coverage for common caching scenarios across different implementations * refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one * fix(router.py): drop redis password requirement * fix(proxy_server.py): fix faulty slack alerting check * fix(langfuse.py): avoid copying functions/thread lock objects in metadata fixes metadata copy error when parent otel span in metadata * test: update test * fix(key_management_endpoints.py): fix /key/update with metadata update * fix(key_management_endpoints.py): fix key_prepare_update helper * fix(key_management_endpoints.py): reset value to none if set in key update * fix: update test ' * Litellm dev 11 11 2024 (#6693) * fix(__init__.py): add 'watsonx_text' as mapped llm api route Fixes https://github.com/BerriAI/litellm/issues/6663 * fix(opentelemetry.py): fix passing parallel tool calls to otel Fixes https://github.com/BerriAI/litellm/issues/6677 * refactor(test_opentelemetry_unit_tests.py): create a base set of unit tests for all logging integrations - test for parallel tool call handling reduces bugs in repo * fix(__init__.py): update provider-model mapping to include all known provider-model mappings Fixes https://github.com/BerriAI/litellm/issues/6669 * feat(anthropic): support passing document in llm api call * docs(anthropic.md): add pdf anthropic call to docs + expose new 'supports_pdf_input' function * fix(factory.py): fix linting error * add clear doc string for GCS bucket logging * Add docs to export logs to Laminar (#6674) * Add docs to export logs to Laminar * minor fix: newline at end of file * place laminar after http and grpc * (Feat) Add langsmith key based logging (#6682) * add langsmith_api_key to StandardCallbackDynamicParams * create a file for langsmith types * langsmith add key / team based logging * add key based logging for langsmith * fix langsmith key based logging * fix linting langsmith * remove NOQA violation * add unit test coverage for all helpers in test langsmith * test_langsmith_key_based_logging * docs langsmith key based logging * run langsmith tests in logging callback tests * fix logging testing * test_langsmith_key_based_logging * test_add_callback_via_key_litellm_pre_call_utils_langsmith * add debug statement langsmith key based logging * test_langsmith_key_based_logging * (fix) OpenAI's optional messages[].name does not work with Mistral API (#6701) * use helper for _transform_messages mistral * add test_message_with_name to base LLMChat test * fix linting * add xAI on Admin UI (#6680) * (docs) add benchmarks on 1K RPS (#6704) * docs litellm proxy benchmarks * docs GCS bucket * doc fix - reduce clutter on logging doc title * (feat) add cost tracking stable diffusion 3 on Bedrock (#6676) * add cost tracking for sd3 * test_image_generation_bedrock * fix get model info for image cost * add cost_calculator for stability 1 models * add unit testing for bedrock image cost calc * test_cost_calculator_with_no_optional_params * add test_cost_calculator_basic * correctly allow size Optional * fix cost_calculator * sd3 unit tests cost calc * fix raise correct error 404 when /key/info is called on non-existent key (#6653) * fix raise correct error on /key/info * add not_found_error error * fix key not found in DB error * use 1 helper for checking token hash * fix error code on key info * fix test key gen prisma * test_generate_and_call_key_info * test fix test_call_with_valid_model_using_all_models * fix key info tests * bump: version 1.52.4 → 1.52.5 * add defaults used for GCS logging * LiteLLM Minor Fixes & Improvements (11/12/2024) (#6705) * fix(caching): convert arg to equivalent kwargs in llm caching handler prevent unexpected errors * fix(caching_handler.py): don't pass args to caching * fix(caching): remove all *args from caching.py * fix(caching): consistent function signatures + abc method * test(caching_unit_tests.py): add unit tests for llm caching ensures coverage for common caching scenarios across different implementations * refactor(litellm_logging.py): move to using cache key from hidden params instead of regenerating one * fix(router.py): drop redis password requirement * fix(proxy_server.py): fix faulty slack alerting check * fix(langfuse.py): avoid copying functions/thread lock objects in metadata fixes metadata copy error when parent otel span in metadata * test: update test * bump: version 1.52.5 → 1.52.6 * (feat) helm hook to sync db schema (#6715) * v0 migration job * fix job * fix migrations job.yml * handle standalone DB on helm hook * fix argo cd annotations * fix db migration helm hook * fix migration job * doc fix Using Http/2 with Hypercorn * (fix proxy redis) Add redis sentinel support (#6154) * add sentinel_password support * add doc for setting redis sentinel password * fix redis sentinel - use sentinel password * Fix: Update gpt-4o costs to that of gpt-4o-2024-08-06 (#6714) Fixes #6713 * (fix) using Anthropic `response_format={"type": "json_object"}` (#6721) * add support for response_format=json anthropic * add test_json_response_format to baseLLM ChatTest * fix test_litellm_anthropic_prompt_caching_tools * fix test_anthropic_function_call_with_no_schema * test test_create_json_tool_call_for_response_format * (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass (#6716) * add BaseImageGenTest * use 1 class for unit testing * add debugging to BaseImageGenTest * TestAzureOpenAIDalle3 * fix response_cost_calculator * test_basic_image_generation * fix img gen basic test * fix _select_model_name_for_cost_calc * fix test_aimage_generation_bedrock_with_optional_params * fix undo changes cost tracking * fix response_cost_calculator * fix test_cost_azure_gpt_35 * fix remove dup test (#6718) * (build) update db helm hook * (build) helm db pre sync hook * (build) helm db sync hook * test: run test_team_logging firdst --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com> Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de> * test: update test * test: skip anthropic overloaded error * test: cleanup test * test: update tests * test: fix test * test: handle gemini overloaded model error * test: handle internal server error * test: handle anthropic overloaded error * test: handle claude instability --------- Signed-off-by: B-Step62 <yuki.watanabe@databricks.com> Co-authored-by: Yuki Watanabe <31463517+B-Step62@users.noreply.github.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: Dinmukhamed Mailibay <47117969+dinmukhamedm@users.noreply.github.com> Co-authored-by: Kilian Lieret <kilian.lieret@posteo.de>
371 lines
11 KiB
JavaScript
371 lines
11 KiB
JavaScript
/**
|
|
* Creating a sidebar enables you to:
|
|
- create an ordered group of docs
|
|
- render a sidebar for each doc of that group
|
|
- provide next/previous navigation
|
|
|
|
The sidebars can be generated from the filesystem, or explicitly defined here.
|
|
|
|
Create as many sidebars as you want.
|
|
*/
|
|
|
|
// @ts-check
|
|
|
|
/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
|
|
const sidebars = {
|
|
// // By default, Docusaurus generates a sidebar from the docs folder structure
|
|
|
|
// But you can create a sidebar manually
|
|
tutorialSidebar: [
|
|
{ type: "doc", id: "index" }, // NEW
|
|
{
|
|
type: "category",
|
|
label: "💥 LiteLLM Proxy Server",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "💥 LiteLLM Proxy Server (LLM Gateway)",
|
|
description: `OpenAI Proxy Server (LLM Gateway) to call 100+ LLMs in a unified interface & track spend, set budgets per virtual key/user`,
|
|
slug: "/simple_proxy",
|
|
},
|
|
items: [
|
|
"proxy/quick_start",
|
|
"proxy/docker_quick_start",
|
|
"proxy/deploy",
|
|
"proxy/demo",
|
|
"proxy/prod",
|
|
{
|
|
type: "category",
|
|
label: "Architecture",
|
|
items: ["proxy/architecture"],
|
|
},
|
|
{
|
|
type: "link",
|
|
label: "📖 All Endpoints (Swagger)",
|
|
href: "https://litellm-api.up.railway.app/",
|
|
},
|
|
"proxy/enterprise",
|
|
"proxy/user_keys",
|
|
"proxy/configs",
|
|
"proxy/response_headers",
|
|
"proxy/reliability",
|
|
{
|
|
type: "category",
|
|
label: "🔑 Authentication",
|
|
items: ["proxy/virtual_keys", "proxy/token_auth", "proxy/service_accounts", "proxy/access_control","proxy/ip_address"],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "💸 Spend Tracking + Budgets",
|
|
items: ["proxy/cost_tracking", "proxy/users", "proxy/custom_pricing", "proxy/team_budgets", "proxy/billing", "proxy/customers"],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Routing",
|
|
items: ["proxy/load_balancing", "proxy/tag_routing", "proxy/team_based_routing", "proxy/customer_routing",],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Pass-through Endpoints (Provider-specific)",
|
|
items: [
|
|
"pass_through/vertex_ai",
|
|
"pass_through/google_ai_studio",
|
|
"pass_through/cohere",
|
|
"pass_through/anthropic_completion",
|
|
"pass_through/bedrock",
|
|
"pass_through/langfuse"
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Admin UI",
|
|
items: [
|
|
"proxy/ui",
|
|
"proxy/self_serve",
|
|
"proxy/custom_sso"
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "🪢 Logging, Alerting, Metrics",
|
|
items: ["proxy/logging", "proxy/bucket", "proxy/team_logging","proxy/streaming_logging", "proxy/alerting", "proxy/prometheus",],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "🛡️ [Beta] Guardrails",
|
|
items: [
|
|
"proxy/guardrails/quick_start",
|
|
"proxy/guardrails/aporia_api",
|
|
"proxy/guardrails/guardrails_ai",
|
|
"proxy/guardrails/lakera_ai",
|
|
"proxy/guardrails/bedrock",
|
|
"proxy/guardrails/pii_masking_v2",
|
|
"proxy/guardrails/secret_detection",
|
|
"proxy/guardrails/custom_guardrail",
|
|
"prompt_injection"
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Secret Manager - storing LLM API Keys",
|
|
items: [
|
|
"secret",
|
|
"oidc"
|
|
]
|
|
},
|
|
"proxy/caching",
|
|
"proxy/pass_through",
|
|
"proxy/email",
|
|
"proxy/multiple_admins",
|
|
"proxy/model_management",
|
|
"proxy/health",
|
|
"proxy/debugging",
|
|
"proxy/call_hooks",
|
|
"proxy/rules",
|
|
"proxy/cli",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "💯 Supported Models & Providers",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Providers",
|
|
description:
|
|
"Learn how to deploy + call models from different providers on LiteLLM",
|
|
slug: "/providers",
|
|
},
|
|
items: [
|
|
"providers/openai",
|
|
"providers/text_completion_openai",
|
|
"providers/openai_compatible",
|
|
"providers/azure",
|
|
"providers/azure_ai",
|
|
"providers/vertex",
|
|
"providers/gemini",
|
|
"providers/anthropic",
|
|
"providers/aws_sagemaker",
|
|
"providers/bedrock",
|
|
"providers/litellm_proxy",
|
|
"providers/mistral",
|
|
"providers/codestral",
|
|
"providers/cohere",
|
|
"providers/anyscale",
|
|
"providers/huggingface",
|
|
"providers/databricks",
|
|
"providers/watsonx",
|
|
"providers/predibase",
|
|
"providers/nvidia_nim",
|
|
"providers/xai",
|
|
"providers/lm_studio",
|
|
"providers/cerebras",
|
|
"providers/volcano",
|
|
"providers/triton-inference-server",
|
|
"providers/ollama",
|
|
"providers/perplexity",
|
|
"providers/friendliai",
|
|
"providers/groq",
|
|
"providers/github",
|
|
"providers/deepseek",
|
|
"providers/fireworks_ai",
|
|
"providers/clarifai",
|
|
"providers/vllm",
|
|
"providers/xinference",
|
|
"providers/cloudflare_workers",
|
|
"providers/deepinfra",
|
|
"providers/ai21",
|
|
"providers/nlp_cloud",
|
|
"providers/replicate",
|
|
"providers/togetherai",
|
|
"providers/voyage",
|
|
"providers/jina_ai",
|
|
"providers/aleph_alpha",
|
|
"providers/baseten",
|
|
"providers/openrouter",
|
|
"providers/palm",
|
|
"providers/sambanova",
|
|
// "providers/custom_openai_proxy",
|
|
"providers/custom_llm_server",
|
|
"providers/petals",
|
|
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Chat Completions (litellm.completion + PROXY)",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Chat Completions",
|
|
description: "Details on the completion() function",
|
|
slug: "/completion",
|
|
},
|
|
items: [
|
|
"completion/input",
|
|
"completion/provider_specific_params",
|
|
"completion/json_mode",
|
|
"completion/prompt_caching",
|
|
"completion/audio",
|
|
"completion/vision",
|
|
"completion/predict_outputs",
|
|
"completion/prefix",
|
|
"completion/drop_params",
|
|
"completion/prompt_formatting",
|
|
"completion/output",
|
|
"completion/usage",
|
|
"exception_mapping",
|
|
"completion/stream",
|
|
"completion/message_trimming",
|
|
"completion/function_call",
|
|
"completion/model_alias",
|
|
"completion/batching",
|
|
"completion/mock_requests",
|
|
"completion/reliable_completions",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Supported Endpoints - /images, /audio/speech, /assistants etc",
|
|
items: [
|
|
"embedding/supported_embedding",
|
|
"image_generation",
|
|
"audio_transcription",
|
|
"text_to_speech",
|
|
"rerank",
|
|
"assistants",
|
|
"batches",
|
|
"realtime",
|
|
"fine_tuning",
|
|
{
|
|
type: "link",
|
|
label: "Use LiteLLM Proxy with Vertex, Bedrock SDK",
|
|
href: "/docs/pass_through/vertex_ai",
|
|
},
|
|
],
|
|
},
|
|
"routing",
|
|
"scheduler",
|
|
{
|
|
type: "category",
|
|
label: "🚅 LiteLLM Python SDK",
|
|
items: [
|
|
"set_keys",
|
|
"completion/token_usage",
|
|
"sdk_custom_pricing",
|
|
"embedding/async_embedding",
|
|
"embedding/moderation",
|
|
"budget_manager",
|
|
"caching/all_caches",
|
|
"migration",
|
|
{
|
|
type: "category",
|
|
label: "LangChain, LlamaIndex, Instructor Integration",
|
|
items: ["langchain/langchain", "tutorials/instructor"],
|
|
},
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Load Testing",
|
|
items: [
|
|
"benchmarks",
|
|
"load_test",
|
|
"load_test_advanced",
|
|
"load_test_sdk",
|
|
"load_test_rpm",
|
|
]
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Logging & Observability",
|
|
items: [
|
|
"observability/langfuse_integration",
|
|
"observability/gcs_bucket_integration",
|
|
"observability/langsmith_integration",
|
|
"observability/literalai_integration",
|
|
"observability/opentelemetry_integration",
|
|
"observability/logfire_integration",
|
|
"observability/argilla",
|
|
"observability/arize_integration",
|
|
"debugging/local_debugging",
|
|
"observability/raw_request_response",
|
|
"observability/custom_callback",
|
|
"observability/scrub_data",
|
|
"observability/braintrust",
|
|
"observability/sentry",
|
|
"observability/lago",
|
|
"observability/helicone_integration",
|
|
"observability/openmeter",
|
|
"observability/promptlayer_integration",
|
|
"observability/wandb_integration",
|
|
"observability/slack_integration",
|
|
"observability/athina_integration",
|
|
"observability/lunary_integration",
|
|
"observability/greenscale_integration",
|
|
"observability/supabase_integration",
|
|
`observability/telemetry`,
|
|
"observability/opik_integration",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Tutorials",
|
|
items: [
|
|
'tutorials/litellm_proxy_aporia',
|
|
'tutorials/azure_openai',
|
|
'tutorials/instructor',
|
|
"tutorials/gradio_integration",
|
|
"tutorials/huggingface_codellama",
|
|
"tutorials/huggingface_tutorial",
|
|
"tutorials/TogetherAI_liteLLM",
|
|
"tutorials/finetuned_chat_gpt",
|
|
"tutorials/text_completion",
|
|
"tutorials/first_playground",
|
|
"tutorials/model_fallbacks",
|
|
],
|
|
},
|
|
{
|
|
type: "category",
|
|
label: "Extras",
|
|
items: [
|
|
"extras/contributing",
|
|
"data_security",
|
|
"migration_policy",
|
|
"contributing",
|
|
"proxy/pii_masking",
|
|
"extras/code_quality",
|
|
"rules",
|
|
"proxy_server",
|
|
{
|
|
type: "category",
|
|
label: "❤️ 🚅 Projects built on LiteLLM",
|
|
link: {
|
|
type: "generated-index",
|
|
title: "Projects built on LiteLLM",
|
|
description:
|
|
"Learn how to deploy + call models from different providers on LiteLLM",
|
|
slug: "/project",
|
|
},
|
|
items: [
|
|
"projects/Docq.AI",
|
|
"projects/OpenInterpreter",
|
|
"projects/dbally",
|
|
"projects/FastREPL",
|
|
"projects/PROMPTMETHEUS",
|
|
"projects/Codium PR Agent",
|
|
"projects/Prompt2Model",
|
|
"projects/SalesGPT",
|
|
"projects/Quivr",
|
|
"projects/Langstream",
|
|
"projects/Otter",
|
|
"projects/GPT Migrate",
|
|
"projects/YiVal",
|
|
"projects/LiteLLM Proxy",
|
|
"projects/llm_cord",
|
|
],
|
|
},
|
|
],
|
|
},
|
|
"troubleshoot",
|
|
],
|
|
};
|
|
|
|
module.exports = sidebars;
|