mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-22 18:42:26 +00:00
805abf573f
Some checks failed
SqlStore Integration Tests / test-postgres (3.12) (push) Failing after 0s
Integration Auth Tests / test-matrix (oauth2_token) (push) Failing after 1s
Test External Providers Installed via Module / test-external-providers-from-module (venv) (push) Has been skipped
Integration Tests (Replay) / generate-matrix (push) Successful in 3s
SqlStore Integration Tests / test-postgres (3.13) (push) Failing after 3s
API Conformance Tests / check-schema-compatibility (push) Successful in 15s
Python Package Build Test / build (3.12) (push) Successful in 17s
Python Package Build Test / build (3.13) (push) Successful in 18s
Test External API and Providers / test-external (venv) (push) Failing after 28s
Vector IO Integration Tests / test-matrix (push) Failing after 43s
UI Tests / ui-tests (22) (push) Successful in 52s
Unit Tests / unit-tests (3.13) (push) Failing after 1m45s
Unit Tests / unit-tests (3.12) (push) Failing after 1m58s
Pre-commit / pre-commit (22) (push) Successful in 3m9s
Integration Tests (Replay) / Integration Tests (, , , client=, ) (push) Failing after 4m5s
# Problem As an Application Developer, I want to use the include parameter with the value message.output_text.logprobs, so that I can receive log probabilities for output tokens to assess the model's confidence in its response. # What does this PR do? - Updates the include parameter in various resource definitions - Updates the inline provider to return logprobs when "message.output_text.logprobs" is passed in the include parameter - Converts the logprobs returned by the inference provider from chat completion format to responses format Closes #[4260](https://github.com/llamastack/llama-stack/issues/4260) ## Test Plan - Created a script to explore OpenAI behavior: https://github.com/s-akhtar-baig/llama-stack-examples/blob/main/responses/src/include.py - Added integration tests and new recordings --------- Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu> Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
24 lines
3.2 KiB
JSON
Generated
24 lines
3.2 KiB
JSON
Generated
{
|
|
"test_id": "tests/integration/responses/test_basic_responses.py::test_include_logprobs_with_web_search[txt=openai/gpt-4o]",
|
|
"request": {
|
|
"test_id": "tests/integration/responses/test_basic_responses.py::test_include_logprobs_with_web_search[txt=openai/gpt-4o]",
|
|
"provider": "tavily",
|
|
"tool_name": "web_search",
|
|
"kwargs": {
|
|
"query": "positive news October 25 2023"
|
|
}
|
|
},
|
|
"response": {
|
|
"body": {
|
|
"__type__": "llama_stack_api.tools.ToolInvocationResult",
|
|
"__data__": {
|
|
"content": "{\"query\": \"positive news October 25 2023\", \"top_k\": [{\"url\": \"https://www.positive.news/society/what-went-right-in-2023-the-top-25-good-news-stories-of-the-year/\", \"title\": \"What went right in 2023: the top 25 good news stories of the year\", \"content\": \"The 'golden age of medicine' arrived, animals came back from the brink, the renewables juggernaut gathered pace, climate reparations became reality and\", \"score\": 0.9952448, \"raw_content\": null}, {\"url\": \"https://www.pbs.org/newshour/show/october-25-2023-pbs-newshour-full-episode\", \"title\": \"October 25, 2023 \\u2013 PBS NewsHour full episode | PBS News\", \"content\": \"PBS Full Episode Image 1: Increased National Guard Presence in Washington Image 11 PBS News October 25, 2023 PBS News Hour full episode * Image 13: Palestinians work to rescue people at the site of an Israeli strike on a residential building in Gaza City Israel may delay Gaza ground invasion for hostage negotiations and aid for Palestinians 3 min * Image 14: Former U.S. President Trump's civil fraud trial continues, in New York News Wrap: Trump fined $10,000 for violating a gag order in civil fraud trial 4 min Subscribe to Here\\u2019s the Deal, our politics newsletter for analysis you won\\u2019t find anywhere else. 1. Read Nov 29Maduro\\u2019s government rejects Trump\\u2019s claim of closing Venezuelan airspace PBS News\", \"score\": 0.993255, \"raw_content\": null}, {\"url\": \"https://www.nbcnews.com/nightly-news-netcast/video/nightly-news-full-broadcast-october-25th-196425797597\", \"title\": \"Nightly News Full Broadcast (October 25th)\", \"content\": \"Rep. Mike Johnson was elected Speaker of the House; President Biden says Israel has a 'right and responsibility' to target Hamas, but must protect civilians.\", \"score\": 0.99174845, \"raw_content\": null}, {\"url\": \"https://en.wikipedia.org/wiki/Portal:Current_events/2023_October_25\", \"title\": \"Portal:Current events/2023 October 25 - Wikipedia\", \"content\": \"The Gaza Health Ministry announces that at least 704 people have died in the past day. \\u00b7 At least 16 people are killed and several others are injured in Israeli\", \"score\": 0.98975134, \"raw_content\": null}, {\"url\": \"https://www.theguardian.com/theguardian/2023/oct/25\", \"title\": \"Wednesday 25 October 2023 - The Guardian\", \"content\": \"UK news \\u00b7 Gaza diary part eight: 'I am terrified that I'm getting used to what is happening' \\u00b7 'The West Bank will ignite': Israel's operations\", \"score\": 0.98926485, \"raw_content\": null}]}",
|
|
"error_message": null,
|
|
"error_code": null,
|
|
"metadata": null
|
|
}
|
|
},
|
|
"is_streaming": false
|
|
},
|
|
"id_normalization_mapping": {}
|
|
}
|