add to hosted debugger back to docs

This commit is contained in:
Krrish Dholakia 2023-09-07 21:11:35 -07:00
parent bea862827e
commit 6da73d06d3
3 changed files with 48 additions and 87 deletions

View file

@ -1,11 +1,15 @@
import Image from '@theme/IdealImage';
import QueryParamReader from '../../src/components/queryParamReader.js'
# Debug + Deploy LLMs [UI]
# [Beta] Monitor Logs in Production
LiteLLM offers a UI to:
* 1-Click Deploy LLMs - the client stores your api keys + model configurations
* Debug your Call Logs
:::note
This is in beta. Expect frequent updates, as we improve based on your feedback.
:::
LiteLLM provides an integration to let you monitor logs in production.
👉 Jump to our sample LiteLLM Dashboard: https://admin.litellm.ai/
@ -18,16 +22,49 @@ LiteLLM offers a UI to:
</a>
### 1. Make a normal `completion()` call
### 1. Get your LiteLLM Token
Go to [admin.litellm.ai](https://admin.litellm.ai/) and copy the code snippet with your unique token
<Image img={require('../../img/hosted_debugger_usage_page.png')} alt="Usage" />
### 2. Set up your environment
**Add it to your .env**
```
pip install litellm
import os
os.env["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with your unique token
```
<QueryParamReader/>
**Turn on LiteLLM Client**
```
import litellm
litellm.client = True
```
### 2. Check request state
All `completion()` calls print with a link to your session dashboard
### 3. Make a normal `completion()` call
```
import litellm
from litellm import completion
import os
# set env variables
os.environ["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with your unique token
os.environ["OPENAI_API_KEY"] = "openai key"
litellm.use_client = True # enable logging dashboard
messages = [{ "content": "Hello, how are you?","role": "user"}]
# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)
```
Your `completion()` call print with a link to your session dashboard (https://admin.litellm.ai/<your_unique_token>)
In the above case it would be: [`admin.litellm.ai/e24c4c06-d027-4c30-9e78-18bc3a50aebb`](https://admin.litellm.ai/e24c4c06-d027-4c30-9e78-18bc3a50aebb)
Click on your personal dashboard link. Here's how you can find it 👇
@ -51,81 +88,4 @@ Ah! So we can see that this request was made to a **Baseten** (see litellm_param
🎉 Congratulations! You've successfully debugger your first log!
:::
## Deploy your first LLM
LiteLLM also lets you to add a new model to your project - without touching code **or** using a proxy server.
### 1. Add new model
On the same debugger dashboard we just made, just go to the 'Add New LLM' Section:
* Select Provider
* Select your LLM
* Add your LLM Key
<Image img={require('../../img/add_model.png')} alt="Dashboard" />
This works with any model on - Replicate, Together_ai, Baseten, Anthropic, Cohere, AI21, OpenAI, Azure, VertexAI (Google Palm), OpenRouter
After adding your new LLM, LiteLLM securely stores your API key and model configs.
[👋 Tell us if you need to self-host **or** integrate with your key manager](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version?month=2023-08)
### 2. Test new model Using `completion()`
Once you've added your models LiteLLM completion calls will just work for those models + providers.
```python
import litellm
from litellm import completion
litellm.token = "80888ede-4881-4876-ab3f-765d47282e66" # use your token
messages = [{ "content": "Hello, how are you?" ,"role": "user"}]
# no need to set key, LiteLLM Client reads your set key
response = completion(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}])
```
### 3. [Bonus] Get available model list
Get a list of all models you've created through the Dashboard with 1 function call
```python
import litellm
litellm.token = "80888ede-4881-4876-ab3f-765d47282e66" # use your token
litellm.get_model_list()
```
## Persisting your dashboard
If you want to use the same dashboard for your project set
`litellm.token` in code or your .env as `LITELLM_TOKEN`
All generated dashboards come with a token
```python
import litellm
litellm.token = "80888ede-4881-4876-ab3f-765d47282e66"
```
## Additional Information
### LiteLLM Dashboard - Debug Logs
All your `completion()` and `embedding()` call logs are available on `admin.litellm.ai/<your-token>`
#### Debug Logs for `completion()` and `embedding()`
<Image img={require('../../img/lite_logs.png')} alt="Dashboard" />
#### Viewing Errors on debug logs
<Image img={require('../../img/lite_logs2.png')} alt="Dashboard" />
### Opt-Out of using LiteLLM Client
If you want to opt out of using LiteLLM client you can set
```python
litellm.use_client = True
```
:::

Binary file not shown.

After

Width:  |  Height:  |  Size: 329 KiB

View file

@ -50,6 +50,7 @@ const sidebars = {
"token_usage",
"exception_mapping",
'debugging/local_debugging',
'debugging/hosted_debugging',
{
type: 'category',
label: 'Tutorials',