forked from phoenix/litellm-mirror
v2
This commit is contained in:
ishaan-jaff
parent
5dd5acd2ee
commit
6569b435d1
8 changed files with 149 additions and 0 deletions
23
docs/advanced.md
Normal file
23
docs/advanced.md
Normal file
|
|
@ -0,0 +1,23 @@
|
||||||
|
# Advanced - liteLLM client
|
||||||
|
|
||||||
|
## Use liteLLM client to send Output Data to Posthog, Sentry etc
|
||||||
|
liteLLM allows you to create `completion_client` and `embedding_client` to send successfull / error LLM API call data to Posthog, Sentry, Slack etc
|
||||||
|
|
||||||
|
### Quick Start
|
||||||
|
```python
|
||||||
|
from main import litellm_client
|
||||||
|
import os
|
||||||
|
|
||||||
|
## set env variables
|
||||||
|
os.environ['SENTRY_API_URL'] = ""
|
||||||
|
os.environ['POSTHOG_API_KEY'], os.environ['POSTHOG_API_URL'] = "api-key", "api-url"
|
||||||
|
|
||||||
|
# init liteLLM client
|
||||||
|
client = litellm_client(success_callback=["posthog"], failure_callback=["sentry", "posthog"])
|
||||||
|
completion = client.completion
|
||||||
|
embedding = client.embedding
|
||||||
|
|
||||||
|
response = completion(model="gpt-3.5-turbo", messages=messages)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
11
docs/client_integrations.md
Normal file
11
docs/client_integrations.md
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
# Data Logging Integrations
|
||||||
|
|
||||||
|
| Integration | Required OS Variables | How to Use with litellm Client |
|
||||||
|
|-----------------|--------------------------------------------|-------------------------------------------|
|
||||||
|
| Sentry | `SENTRY_API_URL` | `client = litellm_client(success_callback=["sentry"], failure_callback=["sentry"])` |
|
||||||
|
| Posthog | `POSTHOG_API_KEY`,<br>`POSTHOG_API_URL` | `client = litellm_client(success_callback=["posthog"], failure_callback=["posthog"])` |
|
||||||
|
| Slack | `SLACK_API_TOKEN`,<br>`SLACK_API_SECRET`,<br>`SLACK_API_CHANNEL` | `client = litellm_client(success_callback=["slack"], failure_callback=["slack"])` |
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
6
docs/contact.md
Normal file
6
docs/contact.md
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
# Contact Us
|
||||||
|
|
||||||
|
[](https://discord.gg/wuPM9dRgDw)
|
||||||
|
|
||||||
|
* [Meet with us 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
|
||||||
|
* Contact us at ishaan@berri.ai / krrish@berri.ai
|
||||||
39
docs/index.md
Normal file
39
docs/index.md
Normal file
|
|
@ -0,0 +1,39 @@
|
||||||
|
# *🚅 litellm*
|
||||||
|
a light 100 line package to simplify calling OpenAI, Azure, Cohere, Anthropic APIs
|
||||||
|
|
||||||
|
###### litellm manages:
|
||||||
|
* Calling all LLM APIs using the OpenAI format - `completion(model, messages)`
|
||||||
|
* Consistent output for all LLM APIs, text responses will always be available at `['choices'][0]['message']['content']`
|
||||||
|
* **[Advanced]** Automatically logging your output to Sentry, Posthog, Slack [see liteLLM Client](/docs/advanced.md)
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
Go directly to code: [Getting Started Notebook](https://colab.research.google.com/drive/1gR3pY-JzDZahzpVdbGBtrNGDBmzUNJaJ?usp=sharing)
|
||||||
|
### Installation
|
||||||
|
```
|
||||||
|
pip install litellm
|
||||||
|
```
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
```python
|
||||||
|
from litellm import completion
|
||||||
|
|
||||||
|
## set ENV variables
|
||||||
|
os.environ["OPENAI_API_KEY"] = "openai key"
|
||||||
|
os.environ["COHERE_API_KEY"] = "cohere key"
|
||||||
|
|
||||||
|
messages = [{ "content": "Hello, how are you?","role": "user"}]
|
||||||
|
|
||||||
|
# openai call
|
||||||
|
response = completion(model="gpt-3.5-turbo", messages=messages)
|
||||||
|
|
||||||
|
# cohere call
|
||||||
|
response = completion("command-nightly", messages)
|
||||||
|
```
|
||||||
|
Need Help / Support : [see troubleshooting](/docs/troubleshoot.md)
|
||||||
|
|
||||||
|
## Why did we build liteLLM
|
||||||
|
- **Need for simplicity**: Our code started to get extremely complicated managing & translating calls between Azure, OpenAI, Cohere
|
||||||
|
|
||||||
|
## Support
|
||||||
|
* [Meet with us 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version)
|
||||||
|
* Contact us at ishaan@berri.ai / krrish@berri.ai
|
||||||
41
docs/supported.md
Normal file
41
docs/supported.md
Normal file
|
|
@ -0,0 +1,41 @@
|
||||||
|
## Generation/Completion/Chat Completion Models
|
||||||
|
|
||||||
|
### OpenAI Chat Completion Models
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|------------------|----------------------------------------|--------------------------------------|
|
||||||
|
| gpt-3.5-turbo | `completion('gpt-3.5-turbo', messages)` | `os.environ['OPENAI_API_KEY']` |
|
||||||
|
| gpt-4 | `completion('gpt-4', messages)` | `os.environ['OPENAI_API_KEY']` |
|
||||||
|
|
||||||
|
## Azure OpenAI Chat Completion Models
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|------------------|-----------------------------------------|-------------------------------------------|
|
||||||
|
| gpt-3.5-turbo | `completion('gpt-3.5-turbo', messages, azure=True)` | `os.environ['AZURE_API_KEY']`,<br>`os.environ['AZURE_API_BASE']`,<br>`os.environ['AZURE_API_VERSION']` |
|
||||||
|
| gpt-4 | `completion('gpt-4', messages, azure=True)` | `os.environ['AZURE_API_KEY']`,<br>`os.environ['AZURE_API_BASE']`,<br>`os.environ['AZURE_API_VERSION']` |
|
||||||
|
|
||||||
|
### OpenAI Text Completion Models
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|------------------|--------------------------------------------|--------------------------------------|
|
||||||
|
| text-davinci-003 | `completion('text-davinci-003', messages)` | `os.environ['OPENAI_API_KEY']` |
|
||||||
|
|
||||||
|
### Cohere Models
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|------------------|--------------------------------------------|--------------------------------------|
|
||||||
|
| command-nightly | `completion('command-nightly', messages)` | `os.environ['COHERE_API_KEY']` |
|
||||||
|
|
||||||
|
### OpenRouter Models
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|----------------------------------|----------------------------------------------------------------------|---------------------------------------------------------------------------|
|
||||||
|
| google/palm-2-codechat-bison | `completion('google/palm-2-codechat-bison', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| google/palm-2-chat-bison | `completion('google/palm-2-chat-bison', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| openai/gpt-3.5-turbo | `completion('openai/gpt-3.5-turbo', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| openai/gpt-3.5-turbo-16k | `completion('openai/gpt-3.5-turbo-16k', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| openai/gpt-4-32k | `completion('openai/gpt-4-32k', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| anthropic/claude-2 | `completion('anthropic/claude-2', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| anthropic/claude-instant-v1 | `completion('anthropic/claude-instant-v1', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| meta-llama/llama-2-13b-chat | `completion('meta-llama/llama-2-13b-chat', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
|
| meta-llama/llama-2-70b-chat | `completion('meta-llama/llama-2-70b-chat', messages)` | `os.environ['OPENROUTER_API_KEY']`,<br>`os.environ['OR_SITE_URL']`,<br>`os.environ['OR_APP_NAME']` |
|
||||||
5
docs/supported_embedding.md
Normal file
5
docs/supported_embedding.md
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
## Embedding Models
|
||||||
|
|
||||||
|
| Model Name | Function Call | Required OS Variables |
|
||||||
|
|----------------------|---------------------------------------------|--------------------------------------|
|
||||||
|
| text-embedding-ada-002 | `embedding('text-embedding-ada-002', input)` | `os.environ['OPENAI_API_KEY']` |
|
||||||
9
docs/troubleshoot.md
Normal file
9
docs/troubleshoot.md
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
## Stable Version
|
||||||
|
|
||||||
|
If you're running into problems with installation / Usage
|
||||||
|
Use the stable version of litellm
|
||||||
|
|
||||||
|
```
|
||||||
|
pip install litellm==0.1.1
|
||||||
|
```
|
||||||
|
|
||||||
15
mkdocs.yml
Normal file
15
mkdocs.yml
Normal file
|
|
@ -0,0 +1,15 @@
|
||||||
|
site_name: liteLLM
|
||||||
|
nav:
|
||||||
|
- ⚡ Getting Started:
|
||||||
|
- Installation & Quick Start: index.md
|
||||||
|
- 🤖 Supported LLM APIs:
|
||||||
|
- Supported Completion & Chat APIs: supported.md
|
||||||
|
- Supported Embedding APIs: supported_embedding.md
|
||||||
|
- 💾 liteLLM Client - Logging Output:
|
||||||
|
- Quick Start: advanced.md
|
||||||
|
- Output Integrations: client_integrations.md
|
||||||
|
- 💡 Support:
|
||||||
|
- Troubleshooting & Help: troubleshoot.md
|
||||||
|
- Contact Us: contact.md
|
||||||
|
|
||||||
|
theme: readthedocs
|
||||||
Loading…
Add table
Add a link
Reference in a new issue