diff --git a/docs/my-website/docs/proxy/billing.md b/docs/my-website/docs/proxy/billing.md index 97e944f25..521475e0e 100644 --- a/docs/my-website/docs/proxy/billing.md +++ b/docs/my-website/docs/proxy/billing.md @@ -2,19 +2,40 @@ import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# 💰 Billing +# 💵 Billing Bill users for their usage. -Requirements: -- Setup a billing plan on Lago, for usage-based billing. We recommend following their Stripe tutorial - https://docs.getlago.com/templates/per-transaction/stripe#step-1-create-billable-metrics-for-transaction +**🚨 Requirements** +- [Setup Lago](https://docs.getlago.com/guide/self-hosted/docker#run-the-app), for usage-based billing. We recommend following [their Stripe tutorial](https://docs.getlago.com/templates/per-transaction/stripe#step-1-create-billable-metrics-for-transaction) Steps: - Connect the proxy to Lago - Set the id you want to bill for (customers, internal users, teams) - Start! -## 1. Connect proxy to Lago +## Quick Start + +Bill internal users for their usage + +### 1. Connect proxy to Lago + +Set 'lago' as a callback on your proxy config.yaml + +```yaml +model_name: + - model_name: fake-openai-endpoint + litellm_params: + model: openai/fake + api_key: fake-key + api_base: https://exampleopenaiendpoint-production.up.railway.app/ + +litellm_settings: + callbacks: ["lago"] # 👈 KEY CHANGE + +general_settings: + master_key: sk-1234 +``` Add your Lago keys to the environment @@ -22,56 +43,60 @@ Add your Lago keys to the environment export LAGO_API_BASE="http://localhost:3000" # self-host - https://docs.getlago.com/guide/self-hosted/docker#run-the-app export LAGO_API_KEY="3e29d607-de54-49aa-a019-ecf585729070" # Get key - https://docs.getlago.com/guide/self-hosted/docker#find-your-api-key export LAGO_API_EVENT_CODE="openai_tokens" # name of lago billing code +export LAGO_API_CHARGE_BY="user_id" # 👈 Charges 'user_id' attached to proxy key ``` -Set 'lago' as a callback on your proxy config.yaml +Start proxy -```yaml -... -litellm_settings: - callbacks: ["lago"] +```bash +litellm --config /path/to/config.yaml ``` -## 2. Set the id you want to bill for +### 2. Create Key for Internal User -For: -- Customers (id passed via 'user' param in /chat/completion call) = 'end_user_id' -- Internal Users (id set when [creating keys](https://docs.litellm.ai/docs/proxy/virtual_keys#advanced---spend-tracking)) = 'user_id' -- Teams (id set when [creating keys](https://docs.litellm.ai/docs/proxy/virtual_keys#advanced---spend-tracking)) = 'team_id' - -```yaml -export LAGO_API_CHARGE_BY="end_user_id" # 👈 Charge 'Customers'. Default is 'end_user_id'. +```bash +curl 'http://0.0.0.0:4000/key/generate' \ +--header 'Authorization: Bearer sk-1234' \ +--header 'Content-Type: application/json' \ +--data-raw '{"user_id": "my-unique-id"}' # 👈 Internal User's ID ``` -## 3. Start billing! +Response Object: +```bash +{ + "key": "sk-tXL0wt5-lOOVK9sfY2UacA", +} +``` + + +### 3. Start billing! - + -### **Curl** -```shell +```bash curl --location 'http://0.0.0.0:4000/chat/completions' \ --header 'Content-Type: application/json' \ +--header 'Authorization: Bearer sk-tXL0wt5-lOOVK9sfY2UacA' \ # 👈 User's Key --data ' { - "model": "gpt-3.5-turbo", + "model": "fake-openai-endpoint", "messages": [ { "role": "user", "content": "what llm are you" } ], - "user": "my_customer_id" # 👈 whatever your customer id is } ' ``` - -### **OpenAI Python SDK** + + ```python import openai client = openai.OpenAI( - api_key="anything", + api_key="sk-tXL0wt5-lOOVK9sfY2UacA", # 👈 User's Key base_url="http://0.0.0.0:4000" ) @@ -81,12 +106,12 @@ response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ "role": "user", "content": "this is a test request, write a short poem" } -], user="my_customer_id") # 👈 whatever your customer id is +]) print(response) ``` - -### **Langchain** + + ```python from langchain.chat_models import ChatOpenAI @@ -98,15 +123,12 @@ from langchain.prompts.chat import ( from langchain.schema import HumanMessage, SystemMessage import os -os.environ["OPENAI_API_KEY"] = "anything" +os.environ["OPENAI_API_KEY"] = "sk-tXL0wt5-lOOVK9sfY2UacA" # 👈 User's Key chat = ChatOpenAI( openai_api_base="http://0.0.0.0:4000", model = "gpt-3.5-turbo", temperature=0.1, - extra_body={ - "user": "my_customer_id" # 👈 whatever your customer id is - } ) messages = [ @@ -119,85 +141,6 @@ messages = [ ] response = chat(messages) -print(response) -``` - - - -1. Create a key for that user - -```bash -curl 'http://0.0.0.0:4000/key/generate' \ ---header 'Authorization: Bearer ' \ ---header 'Content-Type: application/json' \ ---data-raw '{"user_id": "my-unique-id"}' -``` - -Response Object: - -```bash -{ - "key": "sk-tXL0wt5-lOOVK9sfY2UacA", -} -``` - -2. Make API Calls with that Key - -```python -import openai -client = openai.OpenAI( - api_key="sk-tXL0wt5-lOOVK9sfY2UacA", # 👈 Generated key - base_url="http://0.0.0.0:4000" -) - -# request sent to model set on litellm proxy, `litellm --model` -response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ - { - "role": "user", - "content": "this is a test request, write a short poem" - } -]) - -print(response) -``` - - - - -1. Create a key for that team - -```bash -curl 'http://0.0.0.0:4000/key/generate' \ ---header 'Authorization: Bearer ' \ ---header 'Content-Type: application/json' \ ---data-raw '{"team_id": "my-unique-id"}' -``` - -Response Object: - -```bash -{ - "key": "sk-tXL0wt5-lOOVK9sfY2UacA", -} -``` - -2. Make API Calls with that Key - -```python -import openai -client = openai.OpenAI( - api_key="sk-tXL0wt5-lOOVK9sfY2UacA", # 👈 Generated key - base_url="http://0.0.0.0:4000" -) - -# request sent to model set on litellm proxy, `litellm --model` -response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ - { - "role": "user", - "content": "this is a test request, write a short poem" - } -]) - print(response) ``` @@ -227,3 +170,150 @@ This is what LiteLLM will log to Lagos } } ``` + +## Advanced - Bill Customers, Internal Teams + +For: +- Customers (id passed via 'user' param in /chat/completion call) = 'end_user_id' +- Internal Users (id set when [creating keys](https://docs.litellm.ai/docs/proxy/virtual_keys#advanced---spend-tracking)) = 'user_id' +- Teams (id set when [creating keys](https://docs.litellm.ai/docs/proxy/virtual_keys#advanced---spend-tracking)) = 'team_id' + + + + + + +1. Set 'LAGO_API_CHARGE_BY' to 'end_user_id' + + ```bash + export LAGO_API_CHARGE_BY="end_user_id" + ``` + +2. Test it! + + + + + ```shell + curl --location 'http://0.0.0.0:4000/chat/completions' \ + --header 'Content-Type: application/json' \ + --data ' { + "model": "gpt-3.5-turbo", + "messages": [ + { + "role": "user", + "content": "what llm are you" + } + ], + "user": "my_customer_id" # 👈 whatever your customer id is + } + ' + ``` + + + + ```python + import openai + client = openai.OpenAI( + api_key="anything", + base_url="http://0.0.0.0:4000" + ) + + # request sent to model set on litellm proxy, `litellm --model` + response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ + { + "role": "user", + "content": "this is a test request, write a short poem" + } + ], user="my_customer_id") # 👈 whatever your customer id is + + print(response) + ``` + + + + + ```python + from langchain.chat_models import ChatOpenAI + from langchain.prompts.chat import ( + ChatPromptTemplate, + HumanMessagePromptTemplate, + SystemMessagePromptTemplate, + ) + from langchain.schema import HumanMessage, SystemMessage + import os + + os.environ["OPENAI_API_KEY"] = "anything" + + chat = ChatOpenAI( + openai_api_base="http://0.0.0.0:4000", + model = "gpt-3.5-turbo", + temperature=0.1, + extra_body={ + "user": "my_customer_id" # 👈 whatever your customer id is + } + ) + + messages = [ + SystemMessage( + content="You are a helpful assistant that im using to make a test request to." + ), + HumanMessage( + content="test from litellm. tell me why it's amazing in 1 sentence" + ), + ] + response = chat(messages) + + print(response) + ``` + + + + + + + +1. Set 'LAGO_API_CHARGE_BY' to 'team_id' + +```bash +export LAGO_API_CHARGE_BY="team_id" +``` + +2. Create a key for that team + +```bash +curl 'http://0.0.0.0:4000/key/generate' \ +--header 'Authorization: Bearer ' \ +--header 'Content-Type: application/json' \ +--data-raw '{"team_id": "my-unique-id"}' +``` + +Response Object: + +```bash +{ + "key": "sk-tXL0wt5-lOOVK9sfY2UacA", +} +``` + +3. Make API Calls with that Key + +```python +import openai +client = openai.OpenAI( + api_key="sk-tXL0wt5-lOOVK9sfY2UacA", # 👈 Generated key + base_url="http://0.0.0.0:4000" +) + +# request sent to model set on litellm proxy, `litellm --model` +response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ + { + "role": "user", + "content": "this is a test request, write a short poem" + } +]) + +print(response) +``` + +