Fix typos

This commit is contained in:
Dimitri Papadopoulos 2025-04-23 13:53:00 +02:00
parent f5996b2f6b
commit 2ce8c4baa7
No known key found for this signature in database
GPG key ID: 95998121D9D25F5D
24 changed files with 57 additions and 57 deletions

View file

@ -3,7 +3,7 @@ import TabItem from '@theme/TabItem';
# Using Audio Models # Using Audio Models
How to send / receieve audio to a `/chat/completions` endpoint How to send / receive audio to a `/chat/completions` endpoint
## Audio Output from a model ## Audio Output from a model

View file

@ -3,7 +3,7 @@ import TabItem from '@theme/TabItem';
# Using PDF Input # Using PDF Input
How to send / receieve pdf's (other document types) to a `/chat/completions` endpoint How to send / receive pdf's (other document types) to a `/chat/completions` endpoint
Works for: Works for:
- Vertex AI models (Gemini + Anthropic) - Vertex AI models (Gemini + Anthropic)

View file

@ -194,7 +194,7 @@ Expected Response
## Explicitly specify image type ## Explicitly specify image type
If you have images without a mime-type, or if litellm is incorrectly inferring the mime type of your image (e.g. calling `gs://` url's with vertex ai), you can set this explicity via the `format` param. If you have images without a mime-type, or if litellm is incorrectly inferring the mime type of your image (e.g. calling `gs://` url's with vertex ai), you can set this explicitly via the `format` param.
```python ```python
"image_url": { "image_url": {

View file

@ -53,7 +53,7 @@ response = completion(
## Additional information in metadata ## Additional information in metadata
You can send any additional information to Greenscale by using the `metadata` field in completion and `greenscale_` prefix. This can be useful for sending metadata about the request, such as the project and application name, customer_id, enviornment, or any other information you want to track usage. `greenscale_project` and `greenscale_application` are required fields. You can send any additional information to Greenscale by using the `metadata` field in completion and `greenscale_` prefix. This can be useful for sending metadata about the request, such as the project and application name, customer_id, environment, or any other information you want to track usage. `greenscale_project` and `greenscale_application` are required fields.
```python ```python
#openai call with additional metadata #openai call with additional metadata

View file

@ -185,7 +185,7 @@ curl --location --request POST 'http://0.0.0.0:4000/chat/completions' \
* `trace_release` - Release for the trace, defaults to `None` * `trace_release` - Release for the trace, defaults to `None`
* `trace_metadata` - Metadata for the trace, defaults to `None` * `trace_metadata` - Metadata for the trace, defaults to `None`
* `trace_user_id` - User identifier for the trace, defaults to completion argument `user` * `trace_user_id` - User identifier for the trace, defaults to completion argument `user`
* `tags` - Tags for the trace, defeaults to `None` * `tags` - Tags for the trace, defaults to `None`
##### Updatable Parameters on Continuation ##### Updatable Parameters on Continuation

View file

@ -222,7 +222,7 @@ curl http://localhost:4000/vertex-ai/v1/projects/${PROJECT_ID}/locations/us-cent
LiteLLM Proxy Server supports two methods of authentication to Vertex AI: LiteLLM Proxy Server supports two methods of authentication to Vertex AI:
1. Pass Vertex Credetials client side to proxy server 1. Pass Vertex Credentials client side to proxy server
2. Set Vertex AI credentials on proxy server 2. Set Vertex AI credentials on proxy server

View file

@ -1095,7 +1095,7 @@ response = completion(
print(response.choices[0]) print(response.choices[0])
``` ```
</TabItem> </TabItem>
<TabItem value="proxy" lable="PROXY"> <TabItem value="proxy" label="PROXY">
1. Add model to config 1. Add model to config

View file

@ -483,7 +483,7 @@ response.stream_to_file(speech_file_path)
This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make `litellm.completion()` calls This is a walkthrough on how to use Azure Active Directory Tokens - Microsoft Entra ID to make `litellm.completion()` calls
Step 1 - Download Azure CLI Step 1 - Download Azure CLI
Installation instructons: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli Installation instructions: https://learn.microsoft.com/en-us/cli/azure/install-azure-cli
```shell ```shell
brew update && brew install azure-cli brew update && brew install azure-cli
``` ```

View file

@ -692,7 +692,7 @@ curl http://0.0.0.0:4000/v1/chat/completions \
### **Context Caching** ### **Context Caching**
Use Vertex AI context caching is supported by calling provider api directly. (Unified Endpoint support comin soon.). Use Vertex AI context caching is supported by calling provider api directly. (Unified Endpoint support coming soon.).
[**Go straight to provider**](../pass_through/vertex_ai.md#context-caching) [**Go straight to provider**](../pass_through/vertex_ai.md#context-caching)
@ -910,7 +910,7 @@ export VERTEXAI_PROJECT="my-test-project" # ONLY use if model project is differe
## Specifying Safety Settings ## Specifying Safety Settings
In certain use-cases you may need to make calls to the models and pass [safety settigns](https://ai.google.dev/docs/safety_setting_gemini) different from the defaults. To do so, simple pass the `safety_settings` argument to `completion` or `acompletion`. For example: In certain use-cases you may need to make calls to the models and pass [safety settings](https://ai.google.dev/docs/safety_setting_gemini) different from the defaults. To do so, simple pass the `safety_settings` argument to `completion` or `acompletion`. For example:
### Set per model/request ### Set per model/request
@ -2050,7 +2050,7 @@ response = completion(
print(response.choices[0]) print(response.choices[0])
``` ```
</TabItem> </TabItem>
<TabItem value="proxy" lable="PROXY"> <TabItem value="proxy" label="PROXY">
1. Add model to config 1. Add model to config

View file

@ -243,12 +243,12 @@ We allow you to pass a local image or a an http/https url of your image
Set `UI_LOGO_PATH` on your env. We recommend using a hosted image, it's a lot easier to set up and configure / debug Set `UI_LOGO_PATH` on your env. We recommend using a hosted image, it's a lot easier to set up and configure / debug
Exaple setting Hosted image Example setting Hosted image
```shell ```shell
UI_LOGO_PATH="https://litellm-logo-aws-marketplace.s3.us-west-2.amazonaws.com/berriai-logo-github.png" UI_LOGO_PATH="https://litellm-logo-aws-marketplace.s3.us-west-2.amazonaws.com/berriai-logo-github.png"
``` ```
Exaple setting a local image (on your container) Example setting a local image (on your container)
```shell ```shell
UI_LOGO_PATH="ui_images/logo.jpg" UI_LOGO_PATH="ui_images/logo.jpg"
``` ```

View file

@ -213,7 +213,7 @@ model_list:
general_settings: general_settings:
master_key: sk-1234 master_key: sk-1234
alerting: ["slack"] alerting: ["slack"]
alerting_threshold: 0.0001 # (Seconds) set an artifically low threshold for testing alerting alerting_threshold: 0.0001 # (Seconds) set an artificially low threshold for testing alerting
alert_to_webhook_url: { alert_to_webhook_url: {
"llm_exceptions": "https://hooks.slack.com/services/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", "llm_exceptions": "https://hooks.slack.com/services/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",
"llm_too_slow": "https://hooks.slack.com/services/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", "llm_too_slow": "https://hooks.slack.com/services/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",
@ -247,7 +247,7 @@ model_list:
general_settings: general_settings:
master_key: sk-1234 master_key: sk-1234
alerting: ["slack"] alerting: ["slack"]
alerting_threshold: 0.0001 # (Seconds) set an artifically low threshold for testing alerting alerting_threshold: 0.0001 # (Seconds) set an artificially low threshold for testing alerting
alert_to_webhook_url: { alert_to_webhook_url: {
"llm_exceptions": ["os.environ/SLACK_WEBHOOK_URL", "os.environ/SLACK_WEBHOOK_URL_2"], "llm_exceptions": ["os.environ/SLACK_WEBHOOK_URL", "os.environ/SLACK_WEBHOOK_URL_2"],
"llm_too_slow": ["https://webhook.site/7843a980-a494-4967-80fb-d502dbc16886", "https://webhook.site/28cfb179-f4fb-4408-8129-729ff55cf213"], "llm_too_slow": ["https://webhook.site/7843a980-a494-4967-80fb-d502dbc16886", "https://webhook.site/28cfb179-f4fb-4408-8129-729ff55cf213"],
@ -425,7 +425,7 @@ curl -X GET --location 'http://0.0.0.0:4000/health/services?service=webhook' \
- `projected_exceeded_date` *str or null*: The date when the budget is projected to be exceeded, returned when 'soft_budget' is set for key (optional). - `projected_exceeded_date` *str or null*: The date when the budget is projected to be exceeded, returned when 'soft_budget' is set for key (optional).
- `projected_spend` *float or null*: The projected spend amount, returned when 'soft_budget' is set for key (optional). - `projected_spend` *float or null*: The projected spend amount, returned when 'soft_budget' is set for key (optional).
- `event` *Literal["budget_crossed", "threshold_crossed", "projected_limit_exceeded"]*: The type of event that triggered the webhook. Possible values are: - `event` *Literal["budget_crossed", "threshold_crossed", "projected_limit_exceeded"]*: The type of event that triggered the webhook. Possible values are:
* "spend_tracked": Emitted whenver spend is tracked for a customer id. * "spend_tracked": Emitted whenever spend is tracked for a customer id.
* "budget_crossed": Indicates that the spend has exceeded the max budget. * "budget_crossed": Indicates that the spend has exceeded the max budget.
* "threshold_crossed": Indicates that spend has crossed a threshold (currently sent when 85% and 95% of budget is reached). * "threshold_crossed": Indicates that spend has crossed a threshold (currently sent when 85% and 95% of budget is reached).
* "projected_limit_exceeded": For "key" only - Indicates that the projected spend is expected to exceed the soft budget threshold. * "projected_limit_exceeded": For "key" only - Indicates that the projected spend is expected to exceed the soft budget threshold.
@ -480,7 +480,7 @@ LLM-related Alerts
| `cooldown_deployment` | Alerts when a deployment is put into cooldown | ✅ | | `cooldown_deployment` | Alerts when a deployment is put into cooldown | ✅ |
| `new_model_added` | Notifications when a new model is added to litellm proxy through /model/new| ✅ | | `new_model_added` | Notifications when a new model is added to litellm proxy through /model/new| ✅ |
| `outage_alerts` | Alerts when a specific LLM deployment is facing an outage | ✅ | | `outage_alerts` | Alerts when a specific LLM deployment is facing an outage | ✅ |
| `region_outage_alerts` | Alerts when a specfic LLM region is facing an outage. Example us-east-1 | ✅ | | `region_outage_alerts` | Alerts when a specific LLM region is facing an outage. Example us-east-1 | ✅ |
Budget and Spend Alerts Budget and Spend Alerts

View file

@ -56,7 +56,7 @@ model_list:
model: azure/<your_deployment_name> model: azure/<your_deployment_name>
api_key: os.environ/AZURE_API_KEY api_key: os.environ/AZURE_API_KEY
api_base: os.environ/AZURE_API_BASE api_base: os.environ/AZURE_API_BASE
api_version: os.envrion/AZURE_API_VERSION api_version: os.environ/AZURE_API_VERSION
model_info: model_info:
input_cost_per_token: 0.000421 # 👈 ONLY to track cost per token input_cost_per_token: 0.000421 # 👈 ONLY to track cost per token
output_cost_per_token: 0.000520 # 👈 ONLY to track cost per token output_cost_per_token: 0.000520 # 👈 ONLY to track cost per token

View file

@ -19,7 +19,7 @@ LiteLLM writes `UPDATE` and `UPSERT` queries to the DB. When using 10+ instances
### Stage 1. Each instance writes updates to redis ### Stage 1. Each instance writes updates to redis
Each instance will accumlate the spend updates for a key, user, team, etc and write the updates to a redis queue. Each instance will accumulate the spend updates for a key, user, team, etc and write the updates to a redis queue.
<Image img={require('../../img/deadlock_fix_1.png')} style={{ width: '900px', height: 'auto' }} /> <Image img={require('../../img/deadlock_fix_1.png')} style={{ width: '900px', height: 'auto' }} />
<p style={{textAlign: 'left', color: '#666'}}> <p style={{textAlign: 'left', color: '#666'}}>

View file

@ -22,7 +22,7 @@ echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
# Add the litellm salt key - you cannot change this after adding a model # Add the litellm salt key - you cannot change this after adding a model
# It is used to encrypt / decrypt your LLM API Key credentials # It is used to encrypt / decrypt your LLM API Key credentials
# We recommned - https://1password.com/password-generator/ # We recommend - https://1password.com/password-generator/
# password generator to get a random hash for litellm salt key # password generator to get a random hash for litellm salt key
echo 'LITELLM_SALT_KEY="sk-1234"' >> .env echo 'LITELLM_SALT_KEY="sk-1234"' >> .env
@ -125,7 +125,7 @@ CMD ["--port", "4000", "--config", "config.yaml", "--detailed_debug"]
### Build from litellm `pip` package ### Build from litellm `pip` package
Follow these instructons to build a docker container from the litellm pip package. If your company has a strict requirement around security / building images you can follow these steps. Follow these instructions to build a docker container from the litellm pip package. If your company has a strict requirement around security / building images you can follow these steps.
Dockerfile Dockerfile
@ -999,7 +999,7 @@ services:
- "4000:4000" # Map the container port to the host, change the host port if necessary - "4000:4000" # Map the container port to the host, change the host port if necessary
volumes: volumes:
- ./litellm-config.yaml:/app/config.yaml # Mount the local configuration file - ./litellm-config.yaml:/app/config.yaml # Mount the local configuration file
# You can change the port or number of workers as per your requirements or pass any new supported CLI augument. Make sure the port passed here matches with the container port defined above in `ports` value # You can change the port or number of workers as per your requirements or pass any new supported CLI argument. Make sure the port passed here matches with the container port defined above in `ports` value
command: [ "--config", "/app/config.yaml", "--port", "4000", "--num_workers", "8" ] command: [ "--config", "/app/config.yaml", "--port", "4000", "--num_workers", "8" ]
# ...rest of your docker-compose config if any # ...rest of your docker-compose config if any

View file

@ -691,7 +691,7 @@ curl --request POST \
<TabItem value="admin_only_routes" label="Test `admin_only_routes`"> <TabItem value="admin_only_routes" label="Test `admin_only_routes`">
**Successfull Request** **Successful Request**
```shell ```shell
curl --location 'http://0.0.0.0:4000/key/generate' \ curl --location 'http://0.0.0.0:4000/key/generate' \
@ -729,7 +729,7 @@ curl --location 'http://0.0.0.0:4000/key/generate' \
<TabItem value="allowed_routes" label="Test `allowed_routes`"> <TabItem value="allowed_routes" label="Test `allowed_routes`">
**Successfull Request** **Successful Request**
```shell ```shell
curl http://localhost:4000/chat/completions \ curl http://localhost:4000/chat/completions \

View file

@ -164,7 +164,7 @@ curl -i http://localhost:4000/v1/chat/completions \
**Expected response** **Expected response**
Your response headers will incude `x-litellm-applied-guardrails` with the guardrail applied Your response headers will include `x-litellm-applied-guardrails` with the guardrail applied
``` ```
x-litellm-applied-guardrails: aporia-pre-guard x-litellm-applied-guardrails: aporia-pre-guard

View file

@ -277,7 +277,7 @@ Found under `kwargs["standard_logging_object"]`. This is a standard payload, log
## Langfuse ## Langfuse
We will use the `--config` to set `litellm.success_callback = ["langfuse"]` this will log all successfull LLM calls to langfuse. Make sure to set `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` in your environment We will use the `--config` to set `litellm.success_callback = ["langfuse"]` this will log all successful LLM calls to langfuse. Make sure to set `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY` in your environment
**Step 1** Install langfuse **Step 1** Install langfuse
@ -535,11 +535,11 @@ print(response)
Use this if you want to control which LiteLLM-specific fields are logged as tags by the LiteLLM proxy. By default LiteLLM Proxy logs no LiteLLM-specific fields Use this if you want to control which LiteLLM-specific fields are logged as tags by the LiteLLM proxy. By default LiteLLM Proxy logs no LiteLLM-specific fields
| LiteLLM specific field | Description | Example Value | | LiteLLM specific field | Description | Example Value |
|------------------------|-------------------------------------------------------|------------------------------------------------| |---------------------------|-----------------------------------------------------------------------------------------|------------------------------------------------|
| `cache_hit` | Indicates whether a cache hit occured (True) or not (False) | `true`, `false` | | `cache_hit` | Indicates whether a cache hit occurred (True) or not (False) | `true`, `false` |
| `cache_key` | The Cache key used for this request | `d2b758c****`| | `cache_key` | The Cache key used for this request | `d2b758c****` |
| `proxy_base_url` | The base URL for the proxy server, the value of env var `PROXY_BASE_URL` on your server | `https://proxy.example.com`| | `proxy_base_url` | The base URL for the proxy server, the value of env var `PROXY_BASE_URL` on your server | `https://proxy.example.com` |
| `user_api_key_alias` | An alias for the LiteLLM Virtual Key.| `prod-app1` | | `user_api_key_alias` | An alias for the LiteLLM Virtual Key. | `prod-app1` |
| `user_api_key_user_id` | The unique ID associated with a user's API key. | `user_123`, `user_456` | | `user_api_key_user_id` | The unique ID associated with a user's API key. | `user_123`, `user_456` |
| `user_api_key_user_email` | The email associated with a user's API key. | `user@example.com`, `admin@example.com` | | `user_api_key_user_email` | The email associated with a user's API key. | `user@example.com`, `admin@example.com` |
| `user_api_key_team_alias` | An alias for a team associated with an API key. | `team_alpha`, `dev_team` | | `user_api_key_team_alias` | An alias for a team associated with an API key. | `team_alpha`, `dev_team` |
@ -1190,7 +1190,7 @@ We will use the `--config` to set
- `litellm.success_callback = ["s3"]` - `litellm.success_callback = ["s3"]`
This will log all successfull LLM calls to s3 Bucket This will log all successful LLM calls to s3 Bucket
**Step 1** Set AWS Credentials in .env **Step 1** Set AWS Credentials in .env
@ -1279,7 +1279,7 @@ Log LLM Logs to [Azure Data Lake Storage](https://learn.microsoft.com/en-us/azur
| Property | Details | | Property | Details |
|----------|---------| |----------|---------|
| Description | Log LLM Input/Output to Azure Blob Storag (Bucket) | | Description | Log LLM Input/Output to Azure Blob Storage (Bucket) |
| Azure Docs on Data Lake Storage | [Azure Data Lake Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-introduction) | | Azure Docs on Data Lake Storage | [Azure Data Lake Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-introduction) |
@ -1360,7 +1360,7 @@ LiteLLM Supports logging to the following Datdog Integrations:
<Tabs> <Tabs>
<TabItem value="datadog" label="Datadog Logs"> <TabItem value="datadog" label="Datadog Logs">
We will use the `--config` to set `litellm.callbacks = ["datadog"]` this will log all successfull LLM calls to DataDog We will use the `--config` to set `litellm.callbacks = ["datadog"]` this will log all successful LLM calls to DataDog
**Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback` **Step 1**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
@ -1636,7 +1636,7 @@ class MyCustomHandler(CustomLogger):
litellm_params = kwargs.get("litellm_params", {}) litellm_params = kwargs.get("litellm_params", {})
metadata = litellm_params.get("metadata", {}) # headers passed to LiteLLM proxy, can be found here metadata = litellm_params.get("metadata", {}) # headers passed to LiteLLM proxy, can be found here
# Acess Exceptions & Traceback # Access Exceptions & Traceback
exception_event = kwargs.get("exception", None) exception_event = kwargs.get("exception", None)
traceback_event = kwargs.get("traceback_exception", None) traceback_event = kwargs.get("traceback_exception", None)
@ -2205,7 +2205,7 @@ We will use the `--config` to set
- `litellm.success_callback = ["dynamodb"]` - `litellm.success_callback = ["dynamodb"]`
- `litellm.dynamodb_table_name = "your-table-name"` - `litellm.dynamodb_table_name = "your-table-name"`
This will log all successfull LLM calls to DynamoDB This will log all successful LLM calls to DynamoDB
**Step 1** Set AWS Credentials in .env **Step 1** Set AWS Credentials in .env
@ -2370,7 +2370,7 @@ litellm --test
[Athina](https://athina.ai/) allows you to log LLM Input/Output for monitoring, analytics, and observability. [Athina](https://athina.ai/) allows you to log LLM Input/Output for monitoring, analytics, and observability.
We will use the `--config` to set `litellm.success_callback = ["athina"]` this will log all successfull LLM calls to athina We will use the `--config` to set `litellm.success_callback = ["athina"]` this will log all successful LLM calls to athina
**Step 1** Set Athina API key **Step 1** Set Athina API key

View file

@ -61,7 +61,7 @@ CMD ["--port", "4000", "--config", "./proxy_server_config.yaml"]
## 3. Use Redis 'port','host', 'password'. NOT 'redis_url' ## 3. Use Redis 'port','host', 'password'. NOT 'redis_url'
If you decide to use Redis, DO NOT use 'redis_url'. We recommend usig redis port, host, and password params. If you decide to use Redis, DO NOT use 'redis_url'. We recommend using redis port, host, and password params.
`redis_url`is 80 RPS slower `redis_url`is 80 RPS slower
@ -169,7 +169,7 @@ If you plan on using the DB, set a salt key for encrypting/decrypting variables
Do not change this after adding a model. It is used to encrypt / decrypt your LLM API Key credentials Do not change this after adding a model. It is used to encrypt / decrypt your LLM API Key credentials
We recommned - https://1password.com/password-generator/ password generator to get a random hash for litellm salt key. We recommend - https://1password.com/password-generator/ password generator to get a random hash for litellm salt key.
```bash ```bash
export LITELLM_SALT_KEY="sk-1234" export LITELLM_SALT_KEY="sk-1234"

View file

@ -3,7 +3,7 @@
Set temporary budget increase for a LiteLLM Virtual Key. Use this if you get asked to increase the budget for a key temporarily. Set temporary budget increase for a LiteLLM Virtual Key. Use this if you get asked to increase the budget for a key temporarily.
| Heirarchy | Supported | | Hierarchy | Supported |
|-----------|-----------| |-----------|-----------|
| LiteLLM Virtual Key | ✅ | | LiteLLM Virtual Key | ✅ |
| User | ❌ | | User | ❌ |

View file

@ -4,7 +4,7 @@ import TabItem from '@theme/TabItem';
# Adding LLM Credentials # Adding LLM Credentials
You can add LLM provider credentials on the UI. Once you add credentials you can re-use them when adding new models You can add LLM provider credentials on the UI. Once you add credentials you can reuse them when adding new models
## Add a credential + model ## Add a credential + model

View file

@ -23,7 +23,7 @@ Requirements:
- ** Set on config.yaml** set your master key under `general_settings:master_key`, example below - ** Set on config.yaml** set your master key under `general_settings:master_key`, example below
- ** Set env variable** set `LITELLM_MASTER_KEY` - ** Set env variable** set `LITELLM_MASTER_KEY`
(the proxy Dockerfile checks if the `DATABASE_URL` is set and then intializes the DB connection) (the proxy Dockerfile checks if the `DATABASE_URL` is set and then initializes the DB connection)
```shell ```shell
export DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname> export DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname>
@ -333,7 +333,7 @@ curl http://localhost:4000/v1/chat/completions \
**Expected Response** **Expected Response**
Expect to see a successfull response from the litellm proxy since the key passed in `X-Litellm-Key` is valid Expect to see a successful response from the litellm proxy since the key passed in `X-Litellm-Key` is valid
```shell ```shell
{"id":"chatcmpl-f9b2b79a7c30477ab93cd0e717d1773e","choices":[{"finish_reason":"stop","index":0,"message":{"content":"\n\nHello there, how may I assist you today?","role":"assistant","tool_calls":null,"function_call":null}}],"created":1677652288,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":"fp_44709d6fcb","usage":{"completion_tokens":12,"prompt_tokens":9,"total_tokens":21} {"id":"chatcmpl-f9b2b79a7c30477ab93cd0e717d1773e","choices":[{"finish_reason":"stop","index":0,"message":{"content":"\n\nHello there, how may I assist you today?","role":"assistant","tool_calls":null,"function_call":null}}],"created":1677652288,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":"fp_44709d6fcb","usage":{"completion_tokens":12,"prompt_tokens":9,"total_tokens":21}
``` ```

View file

@ -994,16 +994,16 @@ litellm --health
## Logging Proxy Input/Output - OpenTelemetry ## Logging Proxy Input/Output - OpenTelemetry
### Step 1 Start OpenTelemetry Collecter Docker Container ### Step 1 Start OpenTelemetry Collector Docker Container
This container sends logs to your selected destination This container sends logs to your selected destination
#### Install OpenTelemetry Collecter Docker Image #### Install OpenTelemetry Collector Docker Image
```shell ```shell
docker pull otel/opentelemetry-collector:0.90.0 docker pull otel/opentelemetry-collector:0.90.0
docker run -p 127.0.0.1:4317:4317 -p 127.0.0.1:55679:55679 otel/opentelemetry-collector:0.90.0 docker run -p 127.0.0.1:4317:4317 -p 127.0.0.1:55679:55679 otel/opentelemetry-collector:0.90.0
``` ```
#### Set Destination paths on OpenTelemetry Collecter #### Set Destination paths on OpenTelemetry Collector
Here's the OpenTelemetry yaml config to use with Elastic Search Here's the OpenTelemetry yaml config to use with Elastic Search
```yaml ```yaml
@ -1077,7 +1077,7 @@ general_settings:
LiteLLM will read the `OTEL_ENDPOINT` environment variable to send data to your OTEL collector LiteLLM will read the `OTEL_ENDPOINT` environment variable to send data to your OTEL collector
```python ```python
os.environ['OTEL_ENDPOINT'] # defauls to 127.0.0.1:4317 if not provided os.environ['OTEL_ENDPOINT'] # defaults to 127.0.0.1:4317 if not provided
``` ```
#### Start LiteLLM Proxy #### Start LiteLLM Proxy
@ -1101,8 +1101,8 @@ curl --location 'http://0.0.0.0:4000/chat/completions' \
``` ```
#### Test & View Logs on OpenTelemetry Collecter #### Test & View Logs on OpenTelemetry Collector
On successfull logging you should be able to see this log on your `OpenTelemetry Collecter` Docker Container On successful logging you should be able to see this log on your `OpenTelemetry Collector` Docker Container
```shell ```shell
Events: Events:
SpanEvent #0 SpanEvent #0
@ -1149,7 +1149,7 @@ Here's the log view on Elastic Search. You can see the request `input`, `output`
<Image img={require('../img/elastic_otel.png')} /> <Image img={require('../img/elastic_otel.png')} />
## Logging Proxy Input/Output - Langfuse ## Logging Proxy Input/Output - Langfuse
We will use the `--config` to set `litellm.success_callback = ["langfuse"]` this will log all successfull LLM calls to langfuse We will use the `--config` to set `litellm.success_callback = ["langfuse"]` this will log all successful LLM calls to langfuse
**Step 1** Install langfuse **Step 1** Install langfuse

View file

@ -117,7 +117,7 @@ response = completion("command-nightly", messages)
""" """
# qustions/logs you want to run the LLM on # questions/logs you want to run the LLM on
questions = [ questions = [
"what is litellm?", "what is litellm?",
"why should I use LiteLLM", "why should I use LiteLLM",

View file

@ -30,7 +30,7 @@ def inference(message, history):
yield partial_message yield partial_message
except Exception as e: except Exception as e:
print("Exception encountered:", str(e)) print("Exception encountered:", str(e))
yield f"An Error occured please 'Clear' the error and try your question again" yield f"An Error occurred please 'Clear' the error and try your question again"
``` ```
### Define Chat Interface ### Define Chat Interface