mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-06-27 18:50:41 +00:00
5b8e75b392
# What does this PR do? Some of our multi-turn verification tests were failing because I had accidentally marked content as a required field in the OpenAI chat completion request assistant messages, but it's actually optional. It is required for messages from other roles, but assistant is explicitly allowed to be optional. Similarly, the assistant message tool_calls field should default to None instead of an empty list. These two changes get the openai-llama-stack verification test back to 100% passing, just like it passes 100% when not behind Llama Stack. They also increase the pass rate of some of the other providers in the verification test, but don't get them to 100%. ## Test Plan I started a Llama Stack server setup to run all the verification tests (requires OPENAI_API_KEY env variable) ``` llama stack run --image-type venv tests/verifications/openai-api-verification-run.yaml ``` Then, I manually ran the verification tests to see which were failing, fix them, and ran them again after these changes to ensure they were all passing. ``` python -m pytest -s -v tests/verifications/openai_api/test_chat_completion.py --provider=openai-llama-stack ``` Signed-off-by: Ben Browning <bbrownin@redhat.com> |
||
|---|---|---|
| .. | ||
| agents | ||
| batch_inference | ||
| benchmarks | ||
| common | ||
| datasetio | ||
| datasets | ||
| eval | ||
| files | ||
| inference | ||
| inspect | ||
| models | ||
| post_training | ||
| providers | ||
| safety | ||
| scoring | ||
| scoring_functions | ||
| shields | ||
| synthetic_data_generation | ||
| telemetry | ||
| tools | ||
| vector_dbs | ||
| vector_io | ||
| __init__.py | ||
| datatypes.py | ||
| resource.py | ||
| version.py | ||