fix: loosen tool call checks in inference store (#2420)

# What does this PR do?

This loosens up the tool call function name and arguments checks in
`tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls`
because the small models we use in CI cannot reliably get the tool call
function name or arguments exactly right.

Closes #2345


## Test Plan

I ran this flaking test in a loop, let it run many dozens of times, and
didn't observe any flakes after the changes. Previously it flaked quite
regularly.

```
while uv run pytest -s -v \
    'tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[llama_stack_client-txt=3B-False]' \
    --stack-config=http://localhost:8321 \
    --text-model="meta-llama/Llama-3.2-3B-Instruct" \
    --embedding-model=all-MiniLM-L6-v2; do; sleep 0.1; done
```

Signed-off-by: Ben Browning <bbrownin@redhat.com>
This commit is contained in:
Ben Browning 2025-06-10 08:45:55 -04:00 committed by GitHub
parent 5639ad7466
commit ee57e58f29
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -372,10 +372,14 @@ def test_inference_store_tool_calls(compat_client, client_with_models, text_mode
) )
assert input_content == message, retrieved_response assert input_content == message, retrieved_response
tool_calls = retrieved_response.choices[0].message.tool_calls tool_calls = retrieved_response.choices[0].message.tool_calls
# sometimes model doesn't ouptut tool calls, but we still want to test that the tool was called # sometimes model doesn't output tool calls, but we still want to test that the tool was called
if tool_calls: if tool_calls:
# because we test with small models, just check that we retrieved
# a tool call with a name and arguments string, but ignore contents
assert len(tool_calls) == 1 assert len(tool_calls) == 1
assert tool_calls[0].function.name == "get_weather" assert tool_calls[0].function.name
assert "tokyo" in tool_calls[0].function.arguments.lower() assert tool_calls[0].function.arguments
else: else:
# failed tool call parses show up as a message with content, so ensure
# that the retrieve response content matches the original request
assert retrieved_response.choices[0].message.content == content assert retrieved_response.choices[0].message.content == content