phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 04:04:14 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/tests/integration/recordings/responses/6730dcde0b73.json
Matthew Farrellee 49d4a5cc84
feat: add embedding and dynamic model support to Together inference adapter (#3458) 
# What does this PR do?

adds embedding and dynamic model support to Together inference adapter

 - updated to use OpenAIMixin
 - workarounds for Together api quirks
 - recordings for together suite when subdirs=inference,pattern=openai

## Test Plan

```
$ TOGETHER_API_KEY=_NONE_ ./scripts/integration-tests.sh --stack-config server:ci-tests --setup together --subdirs inference --pattern openai
...

tests/integration/inference/test_openai_completion.py::test_openai_completion_non_streaming[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:completion:sanity] 
instantiating llama_stack_client
Port 8321 is already in use, assuming server is already running...
llama_stack_client instantiated in 0.121s
PASSED [  2%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_non_streaming_suffix[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:completion:suffix] SKIPPED [  4%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_streaming[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:completion:sanity] PASSED [  6%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_prompt_logprobs[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-1] SKIPPED [  8%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_guided_choice[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free] SKIPPED [ 10%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_01] PASSED [ 12%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] PASSED [ 14%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] SKIPPED [ 17%]
tests/integration/inference/test_openai_completion.py::test_inference_store[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 19%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 21%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming_with_file[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free] SKIPPED [ 23%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_single_string[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 25%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_multiple_strings[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 27%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_float[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 29%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_dimensions[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 31%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_user_parameter[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 34%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_empty_list_error[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 36%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_invalid_model_error[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 38%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_different_inputs_different_outputs[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 40%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_base64[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 42%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_base64_batch_processing[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 44%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_prompt_logprobs[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-0] SKIPPED [ 46%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_02] PASSED [ 48%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] PASSED [ 51%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] SKIPPED [ 53%]
tests/integration/inference/test_openai_completion.py::test_inference_store[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [ 55%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [ 57%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_single_string[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 59%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_multiple_strings[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 61%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_float[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 63%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_dimensions[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 65%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_user_parameter[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 68%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_empty_list_error[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 70%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_invalid_model_error[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 72%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_different_inputs_different_outputs[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 74%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_base64[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 76%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_base64_batch_processing[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 78%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_01] PASSED [ 80%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] PASSED [ 82%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] SKIPPED [ 85%]
tests/integration/inference/test_openai_completion.py::test_inference_store[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 87%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 89%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_02] PASSED [ 91%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] PASSED [ 93%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] SKIPPED [ 95%]
tests/integration/inference/test_openai_completion.py::test_inference_store[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [ 97%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [100%]

============================================ 30 passed, 17 skipped, 50 deselected, 4 warnings in 21.96s =============================================
```
2025-09-16 11:53:41 -07:00

756 lines
22 KiB
JSON
Raw Blame History

{
"request": {
"method": "POST",
"url": "https://api.together.xyz/v1/v1/chat/completions",
"headers": {},
"body": {
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"messages": [
{
"role": "user",
"content": "Hello, world!"
}
],
"stream": true
},
"endpoint": "/v1/chat/completions",
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free"
},
"response": {
"body": [
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": "Hello",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 9906
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "Hello",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": "!",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": null
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "!",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " It",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 1102
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " It",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": "'s",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 596
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "'s",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " nice",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 6555
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " nice",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " to",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 311
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " to",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " meet",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 3449
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " meet",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " you",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 499
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " you",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": ".",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 13
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": ".",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " Is",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 2209
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " Is",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " there",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 1070
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " there",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " something",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 2555
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " something",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " I",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 358
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " I",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " can",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 649
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " can",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " help",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 1520
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " help",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " you",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 499
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " you",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " with",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 449
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " with",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " or",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 477
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " or",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " would",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 1053
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " would",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " you",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 499
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " you",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " like",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 1093
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " like",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " to",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 311
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " to",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": " chat",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 6369
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " chat",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": "?",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 30
},
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "?",
"seed": null
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.chat.chat_completion_chunk.ChatCompletionChunk",
"__data__": {
"id": "oBUtTzC-62bZhn-9801a1ee1bea25d8",
"choices": [
{
"delta": {
"content": "",
"function_call": null,
"refusal": null,
"role": "assistant",
"tool_calls": null,
"token_id": 128009
},
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"text": "",
"seed": 16158686754257986000
}
],
"created": 1758039011,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "chat.completion.chunk",
"service_tier": null,
"system_fingerprint": null,
"usage": {
"completion_tokens": 25,
"prompt_tokens": 39,
"total_tokens": 64,
"completion_tokens_details": null,
"prompt_tokens_details": null,
"cached_tokens": 0
}
}
}
],
"is_streaming": true
}
}
Reference in a new issue View git blame Copy permalink