diff --git a/docs/my-website/docs/observability/raw_request_response.md b/docs/my-website/docs/observability/raw_request_response.md index dddf75e98..71305dae6 100644 --- a/docs/my-website/docs/observability/raw_request_response.md +++ b/docs/my-website/docs/observability/raw_request_response.md @@ -1,10 +1,16 @@ import Image from '@theme/IdealImage'; +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; # Raw Request/Response Logging + +## Logging See the raw request/response sent by LiteLLM in your logging provider (OTEL/Langfuse/etc.). -**on SDK** + + + ```python # pip install langfuse import litellm @@ -34,13 +40,85 @@ response = litellm.completion( ) ``` -**on Proxy** + + + + ```yaml litellm_settings: log_raw_request_response: True ``` + + + + **Expected Log** - \ No newline at end of file + + + +## Return Raw Response Headers + +Return raw response headers from llm provider. + +Currently only supported for openai. + + + + +```python +import litellm +import os + +litellm.return_response_headers = True + +## set ENV variables +os.environ["OPENAI_API_KEY"] = "your-api-key" + +response = litellm.completion( + model="gpt-3.5-turbo", + messages=[{ "content": "Hello, how are you?","role": "user"}] +) + +print(response._hidden_params) +``` + + + + +1. Setup config.yaml + +```yaml +model_list: + - model_name: gpt-3.5-turbo + litellm_params: + model: gpt-3.5-turbo + api_key: os.environ/GROQ_API_KEY + +litellm_settings: + return_response_headers: true +``` + +2. Test it! + +```bash +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ +-H 'Content-Type: application/json' \ +-H 'Authorization: Bearer sk-1234' \ +-D '{ + "model": "gpt-3.5-turbo", + "messages": [ + { "role": "system", "content": "Use your tools smartly"}, + { "role": "user", "content": "What time is it now? Use your tool"} + ] +}' +``` + + + + +**Expected Response** + + \ No newline at end of file diff --git a/docs/my-website/img/raw_response_headers.png b/docs/my-website/img/raw_response_headers.png new file mode 100644 index 000000000..d6595c807 Binary files /dev/null and b/docs/my-website/img/raw_response_headers.png differ