mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-31 06:19:59 +00:00
c014571258
This fixes an issue where, with some models (ie the Llama 4 models), together.ai is sending a final usage chunk for streaming responses even if the user didn't ask to include usage. With this change, the OpenAI API verification tests now pass 100% when using Llama Stack as your API server and together.ai as the backend provider. As part of this, I also cleaned up the streaming/non-streaming return types of the `openai_chat_completion` method to keep type checking happy. Signed-off-by: Ben Browning <bbrownin@redhat.com> |
||
|---|---|---|
| .. | ||
| agents | ||
| datasetio | ||
| inference | ||
| post_training | ||
| safety | ||
| tool_runtime | ||
| vector_io | ||
| __init__.py | ||