phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-31 22:24:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1571 commits 57 branches 151 tags 128 MiB
Commit graph

205 commits

Author SHA1 Message Date
Xi Yan
ce0784be0c Merge branch 'pr1573' into api_2 2025-03-12 23:44:34 -07:00
Xi Yan
8a6fa41a93 more purposes 2025-03-12 23:44:18 -07:00
Xi Yan
f90dcd2a69 Merge branch 'pr1573' into api_2 2025-03-12 23:36:03 -07:00
Xi Yan
0df33049e3 update doc 2025-03-12 23:32:54 -07:00
Xi Yan
b4d118fc5c update doc 2025-03-12 23:30:47 -07:00
Xi Yan
772339bebf update doc 2025-03-12 23:27:45 -07:00
Xi Yan
4f6f0f6a91 update doc 2025-03-12 23:27:01 -07:00
Xi Yan
25710c3b8a scoring updates 2025-03-12 21:58:49 -07:00
Xi Yan
3a87562e8d scoring updates 2025-03-12 21:54:12 -07:00
Xi Yan
7b50fdb2b1 Merge branch 'pr1573' into api_2 2025-03-12 21:42:00 -07:00
Xi Yan
4cc1958af9 huggingface obey consistency 2025-03-12 21:37:13 -07:00
Xi Yan
783f8890b1 better doc 2025-03-12 21:34:49 -07:00
Xi Yan
18796c215a better params fields 2025-03-12 21:33:27 -07:00
Xi Yan
a7abe6df74 better params fields 2025-03-12 21:31:22 -07:00
Xi Yan
20cdcd87a3 purpose 2025-03-12 21:23:20 -07:00
Xi Yan
6408bdbc9d scoring fn id 2025-03-12 21:09:17 -07:00
Xi Yan
d7dbc8cf64 Merge branch 'pr1573' into api_2 2025-03-12 21:02:30 -07:00
Xi Yan
790b2d5cc0 source 2025-03-12 18:51:46 -07:00
Xi Yan
a3173e8284 update 2025-03-12 18:46:40 -07:00
Xi Yan
18de4cd08a comments 2025-03-12 18:38:07 -07:00
Xi Yan
8942071b3b Merge branch 'main' into pr1573 2025-03-12 18:23:39 -07:00
Dinesh Yeduguru
99bbe0e70b
feat: Add new compact MetricInResponse type (#1593) 
# What does this PR do?
This change adds a compact type to include metrics in response as
opposed to the full MetricEvent which is relevant for internal logging
purposes.

## Test Plan
```
LLAMA_STACK_CONFIG=~/.llama/distributions/fireworks/fireworks-run.yaml pytest -s -v agents/test_agents.py --safety-shield meta-llama/Llama-Guard-3-8B --text-model meta-llama/Llama-3.1-8B-Instruct

 llama stack run ~/.llama/distributions/fireworks/fireworks-run.yaml

curl --request POST \
  --url http://localhost:8321/v1/inference/chat-completion \
  --header 'content-type: application/json' \
  --data '{
  "model_id": "meta-llama/Llama-3.1-70B-Instruct",
  "messages": [
    {
      "role": "user",
      "content": {
        "type": "text",
        "text": "where do humans live"
      }
    }
  ],
  "stream": false
}'

{
  "metrics": [
    {
      "metric": "prompt_tokens",
      "value": 10,
      "unit": null
    },
    {
      "metric": "completion_tokens",
      "value": 522,
      "unit": null
    },
    {
      "metric": "total_tokens",
      "value": 532,
      "unit": null
    }
  ],
  "completion_message": {
    "role": "assistant",
    "content": "Humans live in various parts of the world...............",
    "stop_reason": "out_of_tokens",
    "tool_calls": []
  },
  "logprobs": null
}
```
 2025-03-12 15:45:44 -07:00
Xi Yan
f840018088 Merge branch 'main' into pr1573 2025-03-12 12:31:49 -07:00
ehhuang
b7a9c45477
chore: deprecate ToolResponseMessage in agent.resume API (#1566) 
# Summary:
closes #1431 

# Test Plan:
LLAMA_STACK_CONFIG=fireworks pytest -s -v
tests/integration/agents/test_agents.py --safety-shield
meta-llama/Llama-Guard-3-8B --text-model
meta-llama/Llama-3.1-8B-Instruct
 2025-03-12 12:10:21 -07:00
Dinesh Yeduguru
58d08d100e
feat: Add back inference metrics and preserve context variables across asyncio boundary (#1552) 
# What does this PR do?
This PR adds back the changes in #1300  which were reverted in  #1476 .

It also adds logic to preserve context variables across asyncio
boundary. this is needed with the library client since the async
generator logic yields control to code outside the event loop, and on
resuming, does not have the same context as before and this requires
preserving the context vars.

address #1477 
## Test Plan


```
 curl --request POST \
  --url http://localhost:8321/v1/inference/chat-completion \
  --header 'content-type: application/json' \
  --data '{
  "model_id": "meta-llama/Llama-3.1-70B-Instruct",
  "messages": [
    {
      "role": "user",
      "content": {
        "type": "text",
        "text": "where do humans live"
      }
    }
  ],
  "stream": false
}' | jq .

{
  "metrics": [
    {
      "trace_id": "kCZwO3tyQC-FuAGb",
      "span_id": "bsP_5a5O",
      "timestamp": "2025-03-11T16:47:38.549084Z",
      "attributes": {
        "model_id": "meta-llama/Llama-3.1-70B-Instruct",
        "provider_id": "fireworks"
      },
      "type": "metric",
      "metric": "prompt_tokens",
      "value": 10,
      "unit": "tokens"
    },
    {
      "trace_id": "kCZwO3tyQC-FuAGb",
      "span_id": "bsP_5a5O",
      "timestamp": "2025-03-11T16:47:38.549449Z",
      "attributes": {
        "model_id": "meta-llama/Llama-3.1-70B-Instruct",
        "provider_id": "fireworks"
      },
      "type": "metric",
      "metric": "completion_tokens",
      "value": 369,
      "unit": "tokens"
    },
    {
      "trace_id": "kCZwO3tyQC-FuAGb",
      "span_id": "bsP_5a5O",
      "timestamp": "2025-03-11T16:47:38.549457Z",
      "attributes": {
        "model_id": "meta-llama/Llama-3.1-70B-Instruct",
        "provider_id": "fireworks"
      },
      "type": "metric",
      "metric": "total_tokens",
      "value": 379,
      "unit": "tokens"
    }
  ],
  "completion_message": {
    "role": "assistant",
    "content": "Humans live on the planet Earth, specifically on its landmasses and in its oceans. Here's a breakdown of where humans live:\n\n1. **Continents:** Humans inhabit all seven continents:\n\t* Africa\n\t* Antarctica ( temporary residents, mostly scientists and researchers)\n\t* Asia\n\t* Australia\n\t* Europe\n\t* North America\n\t* South America\n2. **Countries:** There are 196 countries recognized by the United Nations, and humans live in almost all of them.\n3. **Cities and towns:** Many humans live in urban areas, such as cities and towns, which are often located near coastlines, rivers, or other bodies of water.\n4. **Rural areas:** Some humans live in rural areas, such as villages, farms, and countryside.\n5. **Islands:** Humans inhabit many islands around the world, including those in the Pacific, Indian, and Atlantic Oceans.\n6. **Mountains and highlands:** Humans live in mountainous regions, such as the Himalayas, the Andes, and the Rocky Mountains.\n7. **Deserts:** Some humans live in desert regions, such as the Sahara, the Mojave, and the Atacama.\n8. **Coastal areas:** Many humans live in coastal areas, such as beaches, ports, and coastal cities.\n9. **Underwater habitats:** A few humans live in underwater habitats, such as research stations and submarines.\n10. **Space:** A small number of humans have lived in space, including astronauts on the International Space Station and those who have visited the Moon.\n\nOverall, humans can be found living in almost every environment on Earth, from the frozen tundra to the hottest deserts, and from the highest mountains to the deepest oceans.",
    "stop_reason": "end_of_turn",
    "tool_calls": []
  },
  "logprobs": null
}

```

Orignal repro no longer showing any error:
```
LLAMA_STACK_DISABLE_VERSION_CHECK=true llama stack run ~/.llama/distributions/fireworks/fireworks-run.yaml
python -m examples.agents.e2e_loop_with_client_tools localhost 8321
```

client logs:
https://gist.github.com/dineshyv/047c7e87b18a5792aa660e311ea53166
server logs:
https://gist.github.com/dineshyv/97a2174099619e9916c7c490be26e559
 2025-03-12 12:01:03 -07:00
Xi Yan
1d62ae595d include benchmarks 2025-03-12 00:45:32 -07:00
Xi Yan
b4d868a1e5 include benchmarks 2025-03-12 00:43:24 -07:00
Xi Yan
e68e8c96ae fix 2025-03-12 00:21:34 -07:00
Xi Yan
124040af77 params -> fn 2025-03-12 00:20:41 -07:00
Xi Yan
af4216f34f Merge branch 'pr1573' into api_2 2025-03-12 00:19:25 -07:00
Xi Yan
1d80ec7f81 upgrade doc 2025-03-12 00:17:58 -07:00
Xi Yan
0abedd070c comment 2025-03-12 00:13:27 -07:00
Xi Yan
5c954dd033 single type 2025-03-11 23:25:19 -07:00
Xi Yan
bec5a46915 single type 2025-03-11 23:20:16 -07:00
Xi Yan
bc71980769 alternative 2025-03-11 23:14:35 -07:00
Xi Yan
cd3a3a5e26 add alternative 2025-03-11 23:10:17 -07:00
Xi Yan
4236769b65 precommit 2025-03-11 22:49:44 -07:00
Xi Yan
58d9cb1276 docs 2025-03-11 22:46:52 -07:00
Xi Yan
f9ea90c4f7 docs 2025-03-11 22:45:48 -07:00
Xi Yan
11e57e17e6 custom 2025-03-11 22:39:50 -07:00
Xi Yan
504eeef413 custom 2025-03-11 22:39:22 -07:00
Xi Yan
8952e40201 custom 2025-03-11 22:14:06 -07:00
Xi Yan
5162889709 precommit 2025-03-11 22:13:05 -07:00
Xi Yan
685e863bb5 remove json_schema_type decorator 2025-03-11 22:08:15 -07:00
Xi Yan
98dfc99584 docs 2025-03-11 22:06:55 -07:00
Xi Yan
de382e7b45 merge description with metadata 2025-03-11 22:06:22 -07:00
Xi Yan
2bb6ca818a scoring api update 2025-03-11 21:53:47 -07:00
Xi Yan
bbb1947fb4 scoring api update 2025-03-11 21:52:01 -07:00
Xi Yan
b3ee4c00ce scoring function type 2025-03-11 21:50:25 -07:00
Xi Yan
70fdf6c04b precommit 2025-03-11 21:43:43 -07:00