docs: add languages to code blocks

This commit is contained in:
Jordan Tucker 2023-09-12 15:02:11 -05:00
parent e6d48d91ce
commit ee618444c9
21 changed files with 47 additions and 49 deletions

BIN
docs/.DS_Store vendored

Binary file not shown.

View file

@ -31,7 +31,7 @@ BudgetManager creates a dictionary to manage the user budgets, where the key is
### get model-breakdown per user ### get model-breakdown per user
``` ```python
user = "1234" user = "1234"
# ... # ...
budget_manager.get_model_cost(user=user) # {"gpt-3.5-turbo-0613": 7.3e-05} budget_manager.get_model_cost(user=user) # {"gpt-3.5-turbo-0613": 7.3e-05}

View file

@ -89,7 +89,7 @@ response3 = completion(model="gpt-3.5-turbo", messages=messages, temperature=0.1
For resposes that were returned as cache hit, the response includes a param `cache` = True For resposes that were returned as cache hit, the response includes a param `cache` = True
Example response with cache hit Example response with cache hit
``` ```python
{ {
'cache': True, 'cache': True,
'id': 'chatcmpl-7wggdzd6OXhgE2YhcLJHJNZsEWzZ2', 'id': 'chatcmpl-7wggdzd6OXhgE2YhcLJHJNZsEWzZ2',

View file

@ -39,7 +39,7 @@ for chunk in response:
## (Non-streaming) Mock Response Object ## (Non-streaming) Mock Response Object
``` ```json
{ {
"choices": [ "choices": [
{ {

View file

@ -6,7 +6,7 @@ LiteLLM simplifies this by letting you pass in a model alias mapping.
# expected format # expected format
``` ```python
litellm.model_alias_map = { litellm.model_alias_map = {
# a dictionary containing a mapping of the alias string to the actual litellm model name string # a dictionary containing a mapping of the alias string to the actual litellm model name string
"model_alias": "litellm_model_name" "model_alias": "litellm_model_name"
@ -16,7 +16,7 @@ litellm.model_alias_map = {
# usage # usage
### Relevant Code ### Relevant Code
``` ```python
model_alias_map = { model_alias_map = {
"GPT-3.5": "gpt-3.5-turbo-16k", "GPT-3.5": "gpt-3.5-turbo-16k",
"llama2": "replicate/llama-2-70b-chat:2796ee9483c3fd7aa2e171d38f4ca12251a30609463dcfd4cd76703f22e96cdf" "llama2": "replicate/llama-2-70b-chat:2796ee9483c3fd7aa2e171d38f4ca12251a30609463dcfd4cd76703f22e96cdf"
@ -26,7 +26,7 @@ litellm.model_alias_map = model_alias_map
``` ```
### Complete Code ### Complete Code
``` ```python
import litellm import litellm
from litellm import completion from litellm import completion

View file

@ -14,10 +14,9 @@ for chunk in response:
``` ```
## Async Completion ## Async Completion
Asynchronous Completion with LiteLLM Asynchronous Completion with LiteLLM. LiteLLM provides an asynchronous version of the completion function called `acompletion`
LiteLLM provides an asynchronous version of the completion function called `acompletion`
### Usage ### Usage
``` ```python
from litellm import acompletion from litellm import acompletion
import asyncio import asyncio
@ -37,7 +36,7 @@ We've implemented an `__anext__()` function in the streaming object returned. Th
### Usage ### Usage
Here's an example of using it with openai. But this Here's an example of using it with openai. But this
``` ```python
from litellm import completion from litellm import completion
import asyncio import asyncio

View file

@ -32,7 +32,7 @@ Go to [admin.litellm.ai](https://admin.litellm.ai/) and copy the code snippet wi
**Add it to your .env** **Add it to your .env**
``` ```python
import os import os
os.env["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with your unique token os.env["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with your unique token
@ -40,13 +40,13 @@ os.env["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with
``` ```
**Turn on LiteLLM Client** **Turn on LiteLLM Client**
``` ```python
import litellm import litellm
litellm.client = True litellm.client = True
``` ```
### 3. Make a normal `completion()` call ### 3. Make a normal `completion()` call
``` ```python
import litellm import litellm
from litellm import completion from litellm import completion
import os import os

View file

@ -4,7 +4,7 @@ There's 2 ways to do local debugging - `litellm.set_verbose=True` and by passing
## Set Verbose ## Set Verbose
This is good for getting print statements for everything litellm is doing. This is good for getting print statements for everything litellm is doing.
``` ```python
from litellm import completion from litellm import completion
litellm.set_verbose=True # 👈 this is the 1-line change you need to make litellm.set_verbose=True # 👈 this is the 1-line change you need to make
@ -31,13 +31,13 @@ In that case, LiteLLM allows you to pass in a custom logging function to see / m
Your custom function Your custom function
``` ```python
def my_custom_logging_fn(model_call_dict): def my_custom_logging_fn(model_call_dict):
print(f"model call details: {model_call_dict}") print(f"model call details: {model_call_dict}")
``` ```
### Complete Example ### Complete Example
``` ```python
from litellm import completion from litellm import completion
def my_custom_logging_fn(model_call_dict): def my_custom_logging_fn(model_call_dict):

View file

@ -12,7 +12,7 @@ Integrates with [Infisical's Secret Manager](https://infisical.com/) for secure
### Usage ### Usage
liteLLM manages reading in your LLM API secrets/env variables from Infisical for you liteLLM manages reading in your LLM API secrets/env variables from Infisical for you
``` ```python
import litellm import litellm
from infisical import InfisicalClient from infisical import InfisicalClient

View file

@ -8,7 +8,7 @@ In this case, we want to log requests to Helicone when a request succeeds.
### Approach 1: Use Callbacks ### Approach 1: Use Callbacks
Use just 1 line of code, to instantly log your responses **across all providers** with helicone: Use just 1 line of code, to instantly log your responses **across all providers** with helicone:
``` ```python
litellm.success_callback=["helicone"] litellm.success_callback=["helicone"]
``` ```
@ -39,7 +39,7 @@ If you want to use Helicone to proxy your OpenAI/Azure requests, then you can -
- Pass in helicone request headers via: `litellm.headers` - Pass in helicone request headers via: `litellm.headers`
Complete Code Complete Code
``` ```python
import litellm import litellm
from litellm import completion from litellm import completion

View file

@ -16,10 +16,9 @@ First, sign up to get an app ID on the [LLMonitor dashboard](https://llmonitor.c
Use just 2 lines of code, to instantly log your responses **across all providers** with llmonitor: Use just 2 lines of code, to instantly log your responses **across all providers** with llmonitor:
``` ```python
litellm.success_callback = ["llmonitor"] litellm.success_callback = ["llmonitor"]
litellm.failure_callback = ["llmonitor"] litellm.failure_callback = ["llmonitor"]
``` ```
Complete code Complete code

View file

@ -35,7 +35,7 @@ create table
### Use Callbacks ### Use Callbacks
Use just 2 lines of code, to instantly see costs and log your responses **across all providers** with Supabase: Use just 2 lines of code, to instantly see costs and log your responses **across all providers** with Supabase:
``` ```python
litellm.success_callback=["supabase"] litellm.success_callback=["supabase"]
litellm.failure_callback=["supabase"] litellm.failure_callback=["supabase"]
``` ```

View file

@ -13,7 +13,7 @@ While Traceloop is still in beta, [ping them](nir@traceloop.com) and mention you
Then, install the Traceloop SDK: Then, install the Traceloop SDK:
```bash ```
pip install traceloop pip install traceloop
``` ```

View file

@ -2,7 +2,7 @@
LiteLLM supports Claude-1, 1.2 and Claude-2. LiteLLM supports Claude-1, 1.2 and Claude-2.
### API KEYS ### API KEYS
``` ```python
import os import os
os.environ["ANTHROPIC_API_KEY"] = "" os.environ["ANTHROPIC_API_KEY"] = ""

View file

@ -2,7 +2,7 @@
LiteLLM supports Azure Chat + Embedding calls. LiteLLM supports Azure Chat + Embedding calls.
### API KEYS ### API KEYS
``` ```python
import os import os
os.environ["AZURE_API_KEY"] = "" os.environ["AZURE_API_KEY"] = ""

View file

@ -49,7 +49,7 @@ resp = requests.post(
``` ```
Outputs from your custom LLM api bases should follow this format: Outputs from your custom LLM api bases should follow this format:
``` ```python
{ {
'data': [ 'data': [
{ {

View file

@ -96,7 +96,7 @@ Model name - `WizardLM/WizardCoder-Python-34B-V1.0`
Model id - `https://ji16r2iys9a8rjk2.us-east-1.aws.endpoints.huggingface.cloud` Model id - `https://ji16r2iys9a8rjk2.us-east-1.aws.endpoints.huggingface.cloud`
``` ```python
import os import os
from litellm import completion from litellm import completion
@ -115,7 +115,7 @@ print(response)
Same as the OpenAI format, but also includes logprobs. [See the code](https://github.com/BerriAI/litellm/blob/b4b2dbf005142e0a483d46a07a88a19814899403/litellm/llms/huggingface_restapi.py#L115) Same as the OpenAI format, but also includes logprobs. [See the code](https://github.com/BerriAI/litellm/blob/b4b2dbf005142e0a483d46a07a88a19814899403/litellm/llms/huggingface_restapi.py#L115)
``` ```json
{ {
"choices": [ "choices": [
{ {

View file

@ -71,7 +71,7 @@ The accepted template format is: [Reference](https://huggingface.co/OpenAssistan
``` ```
Let's register our custom prompt template: [Implementation Code](https://github.com/BerriAI/litellm/blob/64f3d3c56ef02ac5544983efc78293de31c1c201/litellm/llms/prompt_templates/factory.py#L77) Let's register our custom prompt template: [Implementation Code](https://github.com/BerriAI/litellm/blob/64f3d3c56ef02ac5544983efc78293de31c1c201/litellm/llms/prompt_templates/factory.py#L77)
``` ```python
import litellm import litellm
litellm.register_prompt_template( litellm.register_prompt_template(
@ -84,7 +84,7 @@ litellm.register_prompt_template(
Let's use it! Let's use it!
``` ```python
from litellm import completion from litellm import completion
# set env variable # set env variable
@ -97,7 +97,7 @@ completion(model="together_ai/OpenAssistant/llama2-70b-oasst-sft-v10", messages=
**Complete Code** **Complete Code**
``` ```python
import litellm import litellm
from litellm import completion from litellm import completion
@ -119,7 +119,7 @@ print(response)
``` ```
**Output** **Output**
``` ```json
{ {
"choices": [ "choices": [
{ {

View file

@ -43,7 +43,7 @@ print(response)
### Batch Completion ### Batch Completion
``` ```python
from litellm import batch_completion from litellm import batch_completion
model_name = "facebook/opt-125m" model_name = "facebook/opt-125m"

View file

@ -64,7 +64,7 @@ The accepted template format is: [Reference](https://huggingface.co/OpenAssistan
``` ```
Let's register our custom prompt template: [Implementation Code](https://github.com/BerriAI/litellm/blob/64f3d3c56ef02ac5544983efc78293de31c1c201/litellm/llms/prompt_templates/factory.py#L77) Let's register our custom prompt template: [Implementation Code](https://github.com/BerriAI/litellm/blob/64f3d3c56ef02ac5544983efc78293de31c1c201/litellm/llms/prompt_templates/factory.py#L77)
``` ```python
import litellm import litellm
litellm.register_prompt_template( litellm.register_prompt_template(
@ -77,7 +77,7 @@ litellm.register_prompt_template(
Let's use it! Let's use it!
``` ```python
from litellm import completion from litellm import completion
# set env variable # set env variable
@ -90,7 +90,7 @@ completion(model="together_ai/OpenAssistant/llama2-70b-oasst-sft-v10", messages=
**Complete Code** **Complete Code**
``` ```python
import litellm import litellm
from litellm import completion from litellm import completion
@ -112,7 +112,7 @@ print(response)
``` ```
**Output** **Output**
``` ```json
{ {
"choices": [ "choices": [
{ {

View file

@ -16,7 +16,7 @@ In this case, let's try and call 3 models:
Here's the complete example: Here's the complete example:
``` ```python
from litellm import completion from litellm import completion
model = "deepset/deberta-v3-large-squad2" model = "deepset/deberta-v3-large-squad2"
@ -36,7 +36,7 @@ What's happening?
We've deployed `meta-llama/Llama-2-7b-hf` behind a public endpoint - `https://ag3dkq4zui5nu8g3.us-east-1.aws.endpoints.huggingface.cloud`. We've deployed `meta-llama/Llama-2-7b-hf` behind a public endpoint - `https://ag3dkq4zui5nu8g3.us-east-1.aws.endpoints.huggingface.cloud`.
Let's try it out: Let's try it out:
``` ```python
from litellm import completion from litellm import completion
model = "meta-llama/Llama-2-7b-hf" model = "meta-llama/Llama-2-7b-hf"
@ -60,15 +60,15 @@ Either via environment variables, by setting it as a package variable or when ca
**Setting via environment variables** **Setting via environment variables**
Here's the 1 line of code you need to add Here's the 1 line of code you need to add
``` ```python
os.environ["HF_TOKEN] = "..." os.environ["HF_TOKEN"] = "..."
``` ```
Here's the full code: Here's the full code:
``` ```python
from litellm import completion from litellm import completion
os.environ["HF_TOKEN] = "..." os.environ["HF_TOKEN"] = "..."
model = "meta-llama/Llama-2-7b-hf" model = "meta-llama/Llama-2-7b-hf"
messages = [{"role": "user", "content": "Hey, how's it going?"}] # LiteLLM follows the OpenAI format messages = [{"role": "user", "content": "Hey, how's it going?"}] # LiteLLM follows the OpenAI format
@ -80,12 +80,12 @@ completion(model=model, messages=messages, custom_llm_provider="huggingface", ap
**Setting it as package variable** **Setting it as package variable**
Here's the 1 line of code you need to add Here's the 1 line of code you need to add
``` ```python
litellm.huggingface_key = "..." litellm.huggingface_key = "..."
``` ```
Here's the full code: Here's the full code:
``` ```python
import litellm import litellm
from litellm import completion from litellm import completion
@ -100,13 +100,13 @@ completion(model=model, messages=messages, custom_llm_provider="huggingface", ap
``` ```
**Passed in during completion call** **Passed in during completion call**
``` ```python
completion(..., api_key="...") completion(..., api_key="...")
``` ```
Here's the full code: Here's the full code:
``` ```python
from litellm import completion from litellm import completion
model = "meta-llama/Llama-2-7b-hf" model = "meta-llama/Llama-2-7b-hf"