fix: add tool-calling example with tool response

llama_stack/models/llama/llama4/prompt_format.md

@@ -139,12 +139,9 @@ You are an expert in composing functions. You are given a question and a set of
             Based on the question, you will need to make one or more function/tool calls to achieve the purpose.
             If none of the function can be used, point it out. If the given question lacks the parameters required by the function,
             also point it out. You should only return the function call in tools call sections.
-
             If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)]
             You SHOULD NOT include any other text in the response.
-
             Here is a list of functions in JSON format that you can invoke.
-
             [
                 {
                     "name": "get_weather",
@@ -221,9 +218,7 @@ Here is a list of functions in JSON format that you can invoke:
                     }
                 }
             ]
-
             Should you decide to return the function call(s), put them in the format of [func1(params_name=params_value, params_name2=params_value2...), func2(params)]
-
             You SHOULD NOT include any other text in the response.<|eot|><|header_start|>assistant<|header_end|>
 
 
@@ -264,7 +259,7 @@ Reminder:
             - Function calls MUST follow the specified format, start with <function= and end with </function>
             - Required parameters MUST be specified
             - Only call one function at a time
-- Put the entire function call reply on one line<|eot_id|><|eot|><|header_start|>user<|header_end|>
+            - Put the entire function call reply on one line<|eot|><|header_start|>user<|header_end|>
 
 Use tools to get latest trending songs<|eot|><|header_start|>assistant<|header_end|>
 
@@ -273,5 +268,54 @@ Use tools to get latest trending songs<|eot|><|header_start|>assistant<|header_e
 
 ##### Model Response Format
 ```
-<function=trending_songs>{"n": "10"}</function><|eot|>
+<function=trending_songs>{"n": 10}</function><|eot|>
 ```
+
+
+
+
+## Parsing tool outputs
+
+This example shows how the model interprets the output of a tool call and synthesizes it into a response.
+
+##### Input Prompt Format
+```
+<|begin_of_text|><|header_start|>system<|header_end|>
+
+You are an expert assistant who can answer general questions or invoke tools when necessary. You have access to the following functions:
+Use the function 'trending_songs' to 'Returns the trending songs on a Music site':
+{"name": "trending_songs", "description": "Returns the trending songs on a Music site", "parameters": {"genre": {"description": "The genre of the songs to return", "param_type": "str", "required": false}, "n": {"description": "The number of songs to return", "param_type": "int", "required": true}}}
+
+Think very carefully before calling functions.
+If you choose to call a function ONLY reply in the following format with no prefix or suffix:
+
+<function=example_function_name>{"example_name": "example_value"}</function>
+Reminder:
+- Function calls MUST follow the specified format, start with <function= and end with </function>
+- Required parameters MUST be specified
+- Only call one function at a time
+- Put the entire function call reply on one line
+- In addition to tool calls, you should also augment your responses by using the tool outputs.<|eot|><|header_start|>user<|header_end|>
+
+Get the top 2 latest trending songs<|eot|><|header_start|>assistant<|header_end|>
+
+<function=trending_songs>{"n": "2"}</function><|eot|><|header_start|>ipython<|header_end|>
+
+[{"name": "Song 1", "artist": "Artist 1", "genre": "Genre 1"}, {"name": "Song 2", "artist": "Artist 2", "genre": "Genre 2"}]<|eom|><|header_start|>assistant<|header_end|>
+
+
+```
+
+##### Model Response Format
+```
+The top 2 latest trending songs are:
+
+1. Song 1 by Artist 1 (Genre 1)
+2. Song 2 by Artist 2 (Genre 2)<|eot|>
+```
+
+
+##### Notes
+
+- Tool outputs should be passed back to the model in the `tool` role, which uses the `<|ipython|>` tag.
+- The model parses the tool output contents until it encounters the `<|eom|>` tag. It uses this to synthesize an appropriate response to the query.

llama_stack/models/llama/llama4/prompts.py

@@ -188,12 +188,9 @@ def usecases(base_model: bool = False) -> List[UseCase | str]:
             Based on the question, you will need to make one or more function/tool calls to achieve the purpose.
             If none of the function can be used, point it out. If the given question lacks the parameters required by the function,
             also point it out. You should only return the function call in tools call sections.
-
             If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)]
             You SHOULD NOT include any other text in the response.
-
             Here is a list of functions in JSON format that you can invoke.
-
             [
                 {
                     "name": "get_weather",
@@ -267,9 +264,7 @@ Here is a list of functions in JSON format that you can invoke:
                     }
                 }
             ]
-
             Should you decide to return the function call(s), put them in the format of [func1(params_name=params_value, params_name2=params_value2...), func2(params)]
-
             You SHOULD NOT include any other text in the response.""",
                         ),
                     ]
@@ -298,7 +293,7 @@ Reminder:
             - Function calls MUST follow the specified format, start with <function= and end with </function>
             - Required parameters MUST be specified
             - Only call one function at a time
-- Put the entire function call reply on one line<|eot_id|>""",
+            - Put the entire function call reply on one line""",
                         ),
                         RawMessage(
                             role="user",
@@ -307,6 +302,44 @@ Reminder:
                     ]
                 ],
             ),
+            Llama4UseCase(
+                title="Parsing tool outputs",
+                description=textwrap.dedent(
+                    """This example shows how the model interprets the output of a tool call and synthesizes it into a response."""
+                ),
+                dialogs=[
+                    [
+                        RawMessage(
+                            role="system",
+                            content="""You are an expert assistant who can answer general questions or invoke tools when necessary. You have access to the following functions:\nUse the function 'trending_songs' to 'Returns the trending songs on a Music site':\n{"name": "trending_songs", "description": "Returns the trending songs on a Music site", "parameters": {"genre": {"description": "The genre of the songs to return", "param_type": "str", "required": false}, "n": {"description": "The number of songs to return", "param_type": "int", "required": true}}}\n\nThink very carefully before calling functions.\nIf you choose to call a function ONLY reply in the following format with no prefix or suffix:\n\n<function=example_function_name>{"example_name": "example_value"}</function>
+Reminder:
+- Function calls MUST follow the specified format, start with <function= and end with </function>
+- Required parameters MUST be specified
+- Only call one function at a time
+- Put the entire function call reply on one line
+- In addition to tool calls, you should also augment your responses by using the tool outputs.""",
+                        ),
+                        RawMessage(
+                            role="user",
+                            content="Get the top 2 latest trending songs",
+                        ),
+                        RawMessage(
+                            role="assistant",
+                            content='<function=trending_songs>{"n": "2"}</function>',
+                        ),
+                        RawMessage(
+                            role="tool",
+                            content='[{"name": "Song 1", "artist": "Artist 1", "genre": "Genre 1"}, {"name": "Song 2", "artist": "Artist 2", "genre": "Genre 2"}]',
+                        ),
+                    ]
+                ],
+                notes=textwrap.dedent(
+                    """
+                    - Tool outputs should be passed back to the model in the `tool` role, which uses the `<|ipython|>` tag.
+                    - The model parses the tool output contents until it encounters the `<|eom|>` tag. It uses this to synthesize an appropriate response to the query.
+                    """
+                ),
+            ),
         ]
     )
 

scripts/generate_prompt_format.py

@@ -13,7 +13,7 @@
 # the top-level of this source tree.
 
 # Run this script:
-# torchrun --nproc_per_node=8 scripts/generate_prompt_format.py meta-llama/Llama-4-17B-Omni-Instruct-BF16-16E ~/.llama/checkpoints/Llama-4-17B-Omni-Instruct-BF16-16E/ llama_stack.models.llama.llama4.prompts llama_stack/models/llama/llama4/prompt_format.md
+# torchrun --nproc_per_node=8 scripts/generate_prompt_format.py meta-llama/Llama-4-Scout-17B-16E ~/.llama/checkpoints/Llama-4-Scout-17B-16E/ llama_stack.models.llama.llama4.prompts llama_stack/models/llama/llama4/prompt_format.md
 
 
 import importlib