phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-10-04 04:04:14 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
llama-stack-mirror/tests/integration/recordings/responses/511eb1b92e34.json
Matthew Farrellee 49d4a5cc84
feat: add embedding and dynamic model support to Together inference adapter (#3458) 
# What does this PR do?

adds embedding and dynamic model support to Together inference adapter

 - updated to use OpenAIMixin
 - workarounds for Together api quirks
 - recordings for together suite when subdirs=inference,pattern=openai

## Test Plan

```
$ TOGETHER_API_KEY=_NONE_ ./scripts/integration-tests.sh --stack-config server:ci-tests --setup together --subdirs inference --pattern openai
...

tests/integration/inference/test_openai_completion.py::test_openai_completion_non_streaming[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:completion:sanity] 
instantiating llama_stack_client
Port 8321 is already in use, assuming server is already running...
llama_stack_client instantiated in 0.121s
PASSED [  2%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_non_streaming_suffix[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:completion:suffix] SKIPPED [  4%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_streaming[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:completion:sanity] PASSED [  6%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_prompt_logprobs[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-1] SKIPPED [  8%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_guided_choice[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free] SKIPPED [ 10%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_01] PASSED [ 12%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] PASSED [ 14%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] SKIPPED [ 17%]
tests/integration/inference/test_openai_completion.py::test_inference_store[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 19%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 21%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming_with_file[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free] SKIPPED [ 23%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_single_string[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 25%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_multiple_strings[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 27%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_float[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 29%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_dimensions[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 31%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_user_parameter[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 34%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_empty_list_error[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 36%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_invalid_model_error[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 38%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_different_inputs_different_outputs[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 40%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_base64[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 42%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_base64_batch_processing[openai_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 44%]
tests/integration/inference/test_openai_completion.py::test_openai_completion_prompt_logprobs[txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-0] SKIPPED [ 46%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_02] PASSED [ 48%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] PASSED [ 51%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] SKIPPED [ 53%]
tests/integration/inference/test_openai_completion.py::test_inference_store[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [ 55%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[openai_client-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [ 57%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_single_string[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 59%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_multiple_strings[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 61%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_float[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 63%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_dimensions[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 65%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_user_parameter[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 68%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_empty_list_error[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 70%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_invalid_model_error[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 72%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_different_inputs_different_outputs[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] PASSED [ 74%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_with_encoding_format_base64[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 76%]
tests/integration/inference/test_openai_embeddings.py::test_openai_embeddings_base64_batch_processing[llama_stack_client-emb=together/togethercomputer/m2-bert-80M-32k-retrieval] SKIPPED [ 78%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_01] PASSED [ 80%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] PASSED [ 82%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_01] SKIPPED [ 85%]
tests/integration/inference/test_openai_completion.py::test_inference_store[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 87%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-True] PASSED [ 89%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_non_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:non_streaming_02] PASSED [ 91%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] PASSED [ 93%]
tests/integration/inference/test_openai_completion.py::test_openai_chat_completion_streaming_with_n[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-inference:chat_completion:streaming_02] SKIPPED [ 95%]
tests/integration/inference/test_openai_completion.py::test_inference_store[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [ 97%]
tests/integration/inference/test_openai_completion.py::test_inference_store_tool_calls[client_with_models-txt=together/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free-False] PASSED [100%]

============================================ 30 passed, 17 skipped, 50 deselected, 4 warnings in 21.96s =============================================
```
2025-09-16 11:53:41 -07:00

1278 lines
36 KiB
JSON
Raw Blame History

{
"request": {
"method": "POST",
"url": "https://api.together.xyz/v1/v1/completions",
"headers": {},
"body": {
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"prompt": "Respond to this question and explain your answer. Complete the sentence using one word: Roses are red, violets are ",
"max_tokens": 50,
"stream": true,
"extra_body": {}
},
"endpoint": "/v1/completions",
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free"
},
"response": {
"body": [
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " __________________",
"seed": null,
"delta": {
"token_id": 44941,
"role": "assistant",
"content": " __________________"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "____",
"seed": null,
"delta": {
"token_id": 2179,
"role": "assistant",
"content": "____"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "_.",
"seed": null,
"delta": {
"token_id": 5056,
"role": "assistant",
"content": "_."
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " \n\n",
"seed": null,
"delta": {
"token_id": 4815,
"role": "assistant",
"content": " \n\n"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "##",
"seed": null,
"delta": {
"token_id": 567,
"role": "assistant",
"content": "##"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " Step",
"seed": null,
"delta": {
"token_id": 15166,
"role": "assistant",
"content": " Step"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " ",
"seed": null,
"delta": {
"token_id": 220,
"role": "assistant",
"content": " "
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "1",
"seed": null,
"delta": {
"token_id": 16,
"role": "assistant",
"content": "1"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": ":",
"seed": null,
"delta": {
"token_id": 25,
"role": "assistant",
"content": ":"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " Identify",
"seed": null,
"delta": {
"token_id": 65647,
"role": "assistant",
"content": " Identify"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " the",
"seed": null,
"delta": {
"token_id": 279,
"role": "assistant",
"content": " the"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " traditional",
"seed": null,
"delta": {
"token_id": 8776,
"role": "assistant",
"content": " traditional"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " completion",
"seed": null,
"delta": {
"token_id": 9954,
"role": "assistant",
"content": " completion"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " of",
"seed": null,
"delta": {
"token_id": 315,
"role": "assistant",
"content": " of"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " the",
"seed": null,
"delta": {
"token_id": 279,
"role": "assistant",
"content": " the"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " sentence",
"seed": null,
"delta": {
"token_id": 11914,
"role": "assistant",
"content": " sentence"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": ".\n",
"seed": null,
"delta": {
"token_id": 627,
"role": "assistant",
"content": ".\n"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "The",
"seed": null,
"delta": {
"token_id": 791,
"role": "assistant",
"content": "The"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " traditional",
"seed": null,
"delta": {
"token_id": 8776,
"role": "assistant",
"content": " traditional"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " completion",
"seed": null,
"delta": {
"token_id": 9954,
"role": "assistant",
"content": " completion"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " of",
"seed": null,
"delta": {
"token_id": 315,
"role": "assistant",
"content": " of"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " the",
"seed": null,
"delta": {
"token_id": 279,
"role": "assistant",
"content": " the"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " sentence",
"seed": null,
"delta": {
"token_id": 11914,
"role": "assistant",
"content": " sentence"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " \"",
"seed": null,
"delta": {
"token_id": 330,
"role": "assistant",
"content": " \""
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "R",
"seed": null,
"delta": {
"token_id": 49,
"role": "assistant",
"content": "R"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "oses",
"seed": null,
"delta": {
"token_id": 20274,
"role": "assistant",
"content": "oses"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " are",
"seed": null,
"delta": {
"token_id": 527,
"role": "assistant",
"content": " are"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " red",
"seed": null,
"delta": {
"token_id": 2579,
"role": "assistant",
"content": " red"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": ",",
"seed": null,
"delta": {
"token_id": 11,
"role": "assistant",
"content": ","
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " v",
"seed": null,
"delta": {
"token_id": 348,
"role": "assistant",
"content": " v"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "io",
"seed": null,
"delta": {
"token_id": 822,
"role": "assistant",
"content": "io"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "lets",
"seed": null,
"delta": {
"token_id": 10145,
"role": "assistant",
"content": "lets"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " are",
"seed": null,
"delta": {
"token_id": 527,
"role": "assistant",
"content": " are"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "...\"",
"seed": null,
"delta": {
"token_id": 21908,
"role": "assistant",
"content": "...\""
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " is",
"seed": null,
"delta": {
"token_id": 374,
"role": "assistant",
"content": " is"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " based",
"seed": null,
"delta": {
"token_id": 3196,
"role": "assistant",
"content": " based"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " on",
"seed": null,
"delta": {
"token_id": 389,
"role": "assistant",
"content": " on"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " a",
"seed": null,
"delta": {
"token_id": 264,
"role": "assistant",
"content": " a"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " well",
"seed": null,
"delta": {
"token_id": 1664,
"role": "assistant",
"content": " well"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "-known",
"seed": null,
"delta": {
"token_id": 22015,
"role": "assistant",
"content": "-known"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " poem",
"seed": null,
"delta": {
"token_id": 33894,
"role": "assistant",
"content": " poem"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": ".\n\n",
"seed": null,
"delta": {
"token_id": 382,
"role": "assistant",
"content": ".\n\n"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "##",
"seed": null,
"delta": {
"token_id": 567,
"role": "assistant",
"content": "##"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " Step",
"seed": null,
"delta": {
"token_id": 15166,
"role": "assistant",
"content": " Step"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " ",
"seed": null,
"delta": {
"token_id": 220,
"role": "assistant",
"content": " "
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": "2",
"seed": null,
"delta": {
"token_id": 17,
"role": "assistant",
"content": "2"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": ":",
"seed": null,
"delta": {
"token_id": 25,
"role": "assistant",
"content": ":"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " Recall",
"seed": null,
"delta": {
"token_id": 80640,
"role": "assistant",
"content": " Recall"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": null,
"index": 0,
"logprobs": null,
"text": " the",
"seed": null,
"delta": {
"token_id": 279,
"role": "assistant",
"content": " the"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": null
}
},
{
"__type__": "openai.types.completion.Completion",
"__data__": {
"id": "oBUszH9-4Yz4kd-98019fa76a947327",
"choices": [
{
"finish_reason": "length",
"index": 0,
"logprobs": null,
"text": " poem",
"seed": 12390303563326160000,
"delta": {
"token_id": 33894,
"role": "assistant",
"content": " poem"
}
}
],
"created": 1758038918,
"model": "meta-llama/Llama-3.3-70B-Instruct-Turbo-Free",
"object": "completion.chunk",
"system_fingerprint": null,
"usage": {
"completion_tokens": 50,
"prompt_tokens": 25,
"total_tokens": 75,
"completion_tokens_details": null,
"prompt_tokens_details": null,
"cached_tokens": 0
}
}
}
],
"is_streaming": true
}
}
Reference in a new issue View git blame Copy permalink