From 48c09505dfa152b74217b4f14c6a83340b4d50ee Mon Sep 17 00:00:00 2001 From: ishaan-jaff Date: Fri, 11 Aug 2023 16:50:43 -0700 Subject: [PATCH] with new README --- cookbook/proxy-server/readme.md | 48 ++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 7 deletions(-) diff --git a/cookbook/proxy-server/readme.md b/cookbook/proxy-server/readme.md index 12f3c65bd..d6f5ab124 100644 --- a/cookbook/proxy-server/readme.md +++ b/cookbook/proxy-server/readme.md @@ -6,9 +6,9 @@ ![Downloads](https://img.shields.io/pypi/dm/litellm) [![litellm](https://img.shields.io/badge/%20%F0%9F%9A%85%20liteLLM-OpenAI%7CAzure%7CAnthropic%7CPalm%7CCohere%7CReplicate%7CHugging%20Face-blue?color=green)](https://github.com/BerriAI/litellm) -[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/_YF4Qj?referralCode=t3ukrU) +[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/DYqQAW?referralCode=t3ukrU) -# What does liteLLM proxy do +## What does liteLLM proxy do - Make `/chat/completions` requests for 50+ LLM models **Azure, OpenAI, Replicate, Anthropic, Hugging Face** Example: for `model` use `claude-2`, `gpt-3.5`, `gpt-4`, `command-nightly`, `stabilityai/stablecode-completion-alpha-3b-4k` @@ -24,12 +24,12 @@ } ``` - **Consistent Input/Output** Format - - Call all models using the OpenAI format - completion(model, messages) - - Text responses will always be available at ['choices'][0]['message']['content'] + - Call all models using the OpenAI format - `completion(model, messages)` + - Text responses will always be available at `['choices'][0]['message']['content']` - **Error Handling** Using Model Fallbacks (if `GPT-4` fails, try `llama2`) - **Logging** - Log Requests, Responses and Errors to `Supabase`, `Posthog`, `Mixpanel`, `Sentry`, `Helicone` (Any of the supported providers here: https://litellm.readthedocs.io/en/latest/advanced/ - Example: Logs sent to Supabase + **Example: Logs sent to Supabase** Screenshot 2023-08-11 at 4 02 46 PM - **Token Usage & Spend** - Track Input + Completion tokens used + Spend/model @@ -118,7 +118,41 @@ All responses from the server are returned in the following format (for all LLM } ``` - - +## Installation & Usage +### Running Locally +1. Clone liteLLM repository to your local machine: + ``` + git clone https://github.com/BerriAI/liteLLM-proxy + ``` +2. Install the required dependencies using pip + ``` + pip install requirements.txt + ``` +3. Set your LLM API keys + ``` + os.environ['OPENAI_API_KEY]` = "YOUR_API_KEY" + or + set OPENAI_API_KEY in your .env file + ``` +4. Run the server: + ``` + python main.py + ``` + + + +Deploying +1. Quick Start: Deploy on Railway + + [![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/DYqQAW?referralCode=t3ukrU) + +2. `GCP`, `AWS`, `Azure` +This project includes a `Dockerfile` allowing you to build and deploy a Docker Project on your providers + +# Support / Talk with founders +- [Our calendar 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) +- [Community Discord 💭](https://discord.gg/wuPM9dRgDw) +- Our numbers 📞 +1 (770) 8783-106 / +1 (412) 618-6238 +- Our emails ✉️ ishaan@berri.ai / krrish@berri.ai