From b8fa4a48d9c80cbcba1e8ed7713ecbea06b4f75a Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Thu, 1 Aug 2024 09:12:40 -0700 Subject: [PATCH] docs add info on forward_traceparent_to_llm_provider --- docs/my-website/docs/proxy/logging.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/my-website/docs/proxy/logging.md b/docs/my-website/docs/proxy/logging.md index 34e153750..e61051bb5 100644 --- a/docs/my-website/docs/proxy/logging.md +++ b/docs/my-website/docs/proxy/logging.md @@ -714,6 +714,24 @@ Search for Trace=`80e1afed08e019fc1110464cfa66635c` on your OTEL Collector +### Forwarding `Traceparent HTTP Header` to LLM APIs + +Use this if you want to forward the traceparent headers to your self hosted LLMs like vLLM + +Set `forward_traceparent_to_llm_provider: True` in your `config.yaml`. This will forward the `traceparent` header to your LLM API + +:::warning + +Only use this for self hosted LLMs, this can cause Bedrock, VertexAI calls to fail + +::: + +```yaml +litellm_settings: + forward_traceparent_to_llm_provider: True + +``` + ## Custom Callback Class [Async] Use this when you want to run custom callbacks in `python`