diff --git a/docs/my-website/docs/proxy/custom_pricing.md b/docs/my-website/docs/proxy/custom_pricing.md new file mode 100644 index 000000000..10ae06667 --- /dev/null +++ b/docs/my-website/docs/proxy/custom_pricing.md @@ -0,0 +1,34 @@ +import Image from '@theme/IdealImage'; + +# Custom Pricing - Sagemaker, etc. + +Use this to register custom pricing (cost per token or cost per second) for models. + +## Quick Start + +Register custom pricing for sagemaker completion + embedding models. + +For cost per second pricing, you **just** need to register `input_cost_per_second`. + +**Step 1: Add pricing to config.yaml** +```yaml +model_list: + - model_name: sagemaker-completion-model + litellm_params: + model: sagemaker/berri-benchmarking-Llama-2-70b-chat-hf-4 + input_cost_per_second: 0.000420 + - model_name: sagemaker-embedding-model + litellm_params: + model: sagemaker/berri-benchmarking-gpt-j-6b-fp16 + input_cost_per_second: 0.000420 +``` + +**Step 2: Start proxy** + +```bash +litellm /path/to/config.yaml +``` + +**Step 3: View Spend Logs** + + \ No newline at end of file diff --git a/docs/my-website/img/spend_logs_table.png b/docs/my-website/img/spend_logs_table.png new file mode 100644 index 000000000..a0f259244 Binary files /dev/null and b/docs/my-website/img/spend_logs_table.png differ diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index 900e7bc5f..8e20426fd 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -139,6 +139,7 @@ const sidebars = { "items": [ "proxy/call_hooks", "proxy/rules", + "proxy/custom_pricing" ] }, "proxy/deploy",