diff --git a/dist/litellm-0.1.629-py3-none-any.whl b/dist/litellm-0.1.629-py3-none-any.whl new file mode 100644 index 000000000..b2ea2e4a5 Binary files /dev/null and b/dist/litellm-0.1.629-py3-none-any.whl differ diff --git a/dist/litellm-0.1.629.tar.gz b/dist/litellm-0.1.629.tar.gz new file mode 100644 index 000000000..c4a8301c1 Binary files /dev/null and b/dist/litellm-0.1.629.tar.gz differ diff --git a/docs/my-website/docs/budget_manager.md b/docs/my-website/docs/budget_manager.md index c3d1d8f75..059daba3c 100644 --- a/docs/my-website/docs/budget_manager.md +++ b/docs/my-website/docs/budget_manager.md @@ -5,10 +5,28 @@ import TabItem from '@theme/TabItem'; Don't want to get crazy bills because either while you're calling LLM APIs **or** while your users are calling them? use this. -LiteLLM exposes the `BudgetManager` class to help set budgets per user. BudgetManager creates a dictionary to manage the user budgets, where the key is user and the object is their current cost + model-specific costs. +LiteLLM exposes: +* `litellm.max_budget`: a global variable you can use to set the max budget (in USD) across all your litellm calls. If this budget is exceeded, it will raise a BudgetExceededError +* `BudgetManager`: A class to help set budgets per user. BudgetManager creates a dictionary to manage the user budgets, where the key is user and the object is their current cost + model-specific costs. ## quick start +```python +import litellm, os +from litellm import completion + +# set env variable +os.environ["OPENAI_API_KEY"] = "your-api-key" + +litellm.max_budget = 0.001 # sets a max budget of $0.001 + +messages = [{"role": "user", "content": "Hey, how's it going"}] +completion(model="gpt-4", messages=messages) +print(litellm._current_cost) +completion(model="gpt-4", messages=messages) +``` + +## User-based rate limiting Open In Colab