phoenix/litellm-mirror
0
0
Fork 1
mirror of https://github.com/BerriAI/litellm.git synced 2025-04-27 11:43:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add anthropic thinking + reasoning content support (#8778)

Browse source 
* feat(anthropic/chat/transformation.py): add anthropic thinking param support

* feat(anthropic/chat/transformation.py): support returning thinking content for anthropic on streaming responses

* feat(anthropic/chat/transformation.py): return list of thinking blocks (include block signature)

allows usage in tool call responses

* fix(types/utils.py): extract and map reasoning_content from anthropic as content str

* test: add testing to ensure thinking_blocks are returned at the root

* fix(anthropic/chat/handler.py): return thinking blocks on streaming - include signature

* feat(factory.py): handle anthropic thinking blocks translation if in assistant response

* test: handle openai internal instability

* test: handle openai audio instability

* ci: pin anthropic dep

* test: handle openai audio instability

* fix: fix linting error

* refactor(anthropic/chat/transformation.py): refactor function to remain <50 LOC

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error

* fix: fix linting error
This commit is contained in:
Krish Dholakia 2025-02-24 21:54:30 -08:00 • committed by GitHub
parent 18c3d1221e
commit d5324a4ee4
16 changed files with 332 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

20
tests/local_testing/test_streaming.py
View file

@ -4065,20 +4065,32 @@ def test_mock_response_iterator_tool_use():
assert response_chunk["tool_use"] is not None
def test_deepseek_reasoning_content_completion():
@pytest.mark.parametrize(
"model",
[
# "deepseek/deepseek-reasoner",
"anthropic/claude-3-7-sonnet-20250219",
],
)
def test_deepseek_reasoning_content_completion(model):
# litellm.set_verbose = True
try:
# litellm._turn_on_debug()
resp = litellm.completion(
model="deepseek/deepseek-reasoner",
model=model,
messages=[{"role": "user", "content": "Tell me a joke."}],
stream=True,
thinking={"type": "enabled", "budget_tokens": 1024},
timeout=5,
)
reasoning_content_exists = False
for chunk in resp:
print(f"chunk: {chunk}")
if chunk.choices[0].delta.reasoning_content is not None:
print(f"chunk 2: {chunk}")
if (
hasattr(chunk.choices[0].delta, "reasoning_content")
and chunk.choices[0].delta.reasoning_content is not None
):
reasoning_content_exists = True
break
assert reasoning_content_exists