phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-17 09:32:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix(inference): AttributeError in streaming response cleanup (#4236)

Browse source 
This PR fixes issue #3185 
The code calls `await event_gen.aclose()` but OpenAI's `AsyncStream`
doesn't have an `aclose()` method - it has `close()` (which is async).
when clients cancel streaming requests, the server tries to clean up
with:

```python
await event_gen.aclose()  # ❌ AsyncStream doesn't have aclose()!
```

But `AsyncStream` has never had a public `aclose()` method. The error
message literally tells us:

```
AttributeError: 'AsyncStream' object has no attribute 'aclose'. Did you mean: 'close'?
                                                                            ^^^^^^^^
```

## Verification
* Reproduction script
[`reproduce_issue_3185.sh`](https://gist.github.com/r-bit-rry/dea4f8fbb81c446f5db50ea7abd6379b)
can be used to verify the fix.
*   Manual checks, validation against original OpenAI library code
This commit is contained in:
Roy Belio 2025-12-14 13:51:09 +01:00 • committed by GitHub
parent dfb9f6743a
commit c574db5f1d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
14 changed files with 213 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

7
docs/static/llama-stack-spec.yaml vendored
View file

@ -225,7 +225,7 @@ paths:
post:
responses:
'200':
description: An OpenAIChatCompletion.
description: An OpenAIChatCompletion. When streaming, returns Server-Sent Events (SSE) with OpenAIChatCompletionChunk objects.
content:
application/json:
schema:
@ -299,11 +299,14 @@ paths:
post:
responses:
'200':
description: An OpenAICompletion.
description: An OpenAICompletion. When streaming, returns Server-Sent Events (SSE) with OpenAICompletion chunks.
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAICompletion'
text/event-stream:
schema:
$ref: '#/components/schemas/OpenAICompletion'
'400':
description: Bad Request
$ref: '#/components/responses/BadRequest400'

7
docs/static/stainless-llama-stack-spec.yaml vendored
View file

@ -227,7 +227,7 @@ paths:
post:
responses:
'200':
description: An OpenAIChatCompletion.
description: An OpenAIChatCompletion. When streaming, returns Server-Sent Events (SSE) with OpenAIChatCompletionChunk objects.
content:
application/json:
schema:
@ -301,11 +301,14 @@ paths:
post:
responses:
'200':
description: An OpenAICompletion.
description: An OpenAICompletion. When streaming, returns Server-Sent Events (SSE) with OpenAICompletion chunks.
content:
application/json:
schema:
$ref: '#/components/schemas/OpenAICompletion'
text/event-stream:
schema:
$ref: '#/components/schemas/OpenAICompletion'
'400':
description: Bad Request
$ref: '#/components/responses/BadRequest400'