mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-12-03 09:53:45 +00:00
feat(cherry-pick): fixes for 0.3.1 release (#3998)
## Summary Cherry-picks 5 critical fixes from main to the release-0.3.x branch for the v0.3.1 release, plus CI workflow updates. **Note**: This recreates the cherry-picks from the closed PR #3991, now targeting the renamed `release-0.3.x` branch (previously `release-0.3.x-maint`). ## Commits 1. **2c56a8560** - fix(context): prevent provider data leak between streaming requests (#3924) - **CRITICAL SECURITY FIX**: Prevents provider credentials from leaking between requests - Fixed import path for 0.3.0 compatibility 2. **ddd32b187** - fix(inference): enable routing of models with provider_data alone (#3928) - Enables routing for fully qualified model IDs with provider_data - Resolved merge conflicts, adapted for 0.3.0 structure 3. **f7c2973aa** - fix: Avoid BadRequestError due to invalid max_tokens (#3667) - Fixes failures with Gemini and other providers that reject max_tokens=0 - Non-breaking API change 4. **d7f9da616** - fix(responses): sync conversation before yielding terminal events in streaming (#3888) - Ensures conversation sync executes even when streaming consumers break early 5. **0ffa8658b** - fix(logging): ensure logs go to stderr, loggers obey levels (#3885) - Fixes logging infrastructure 6. **75b49cb3c** - ci: support release branches and match client branch (#3990) - Updates CI workflows to support release-X.Y.x branches - Matches client branch from llama-stack-client-python for release testing - Fixes artifact name collisions ## Adaptations for 0.3.0 - Fixed import paths: `llama_stack.core.telemetry.tracing` → `llama_stack.providers.utils.telemetry.tracing` - Fixed import paths: `llama_stack.core.telemetry.telemetry` → `llama_stack.apis.telemetry` - Changed `self.telemetry_enabled` → `self.telemetry` (0.3.0 attribute name) - Removed `rerank()` method that doesn't exist in 0.3.0 ## Testing All imports verified and tests should pass once CI is set up.
This commit is contained in:
parent
bf091306fe
commit
39f33f7f12
11 changed files with 182 additions and 65 deletions
59
tests/unit/core/test_provider_data_context.py
Normal file
59
tests/unit/core/test_provider_data_context.py
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
# Copyright (c) Meta Platforms, Inc. and affiliates.
|
||||
# All rights reserved.
|
||||
#
|
||||
# This source code is licensed under the terms described in the LICENSE file in
|
||||
# the root directory of this source tree.
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from contextlib import contextmanager
|
||||
from contextvars import ContextVar
|
||||
|
||||
from llama_stack.core.utils.context import preserve_contexts_async_generator
|
||||
|
||||
# Define provider data context variable and context manager locally
|
||||
PROVIDER_DATA_VAR = ContextVar("provider_data", default=None)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def request_provider_data_context(headers):
|
||||
val = headers.get("X-LlamaStack-Provider-Data")
|
||||
provider_data = json.loads(val) if val else {}
|
||||
token = PROVIDER_DATA_VAR.set(provider_data)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
PROVIDER_DATA_VAR.reset(token)
|
||||
|
||||
|
||||
def create_sse_event(data):
|
||||
return f"data: {json.dumps(data)}\n\n"
|
||||
|
||||
|
||||
async def sse_generator(event_gen_coroutine):
|
||||
event_gen = await event_gen_coroutine
|
||||
async for item in event_gen:
|
||||
yield create_sse_event(item)
|
||||
await asyncio.sleep(0)
|
||||
|
||||
|
||||
async def async_event_gen():
|
||||
async def event_gen():
|
||||
yield PROVIDER_DATA_VAR.get()
|
||||
|
||||
return event_gen()
|
||||
|
||||
|
||||
async def test_provider_data_context_cleared_between_sse_requests():
|
||||
headers = {"X-LlamaStack-Provider-Data": json.dumps({"api_key": "abc"})}
|
||||
with request_provider_data_context(headers):
|
||||
gen1 = preserve_contexts_async_generator(sse_generator(async_event_gen()), [PROVIDER_DATA_VAR])
|
||||
|
||||
events1 = [event async for event in gen1]
|
||||
assert events1 == [create_sse_event({"api_key": "abc"})]
|
||||
assert PROVIDER_DATA_VAR.get() is None
|
||||
|
||||
gen2 = preserve_contexts_async_generator(sse_generator(async_event_gen()), [PROVIDER_DATA_VAR])
|
||||
events2 = [event async for event in gen2]
|
||||
assert events2 == [create_sse_event(None)]
|
||||
assert PROVIDER_DATA_VAR.get() is None
|
||||
Loading…
Add table
Add a link
Reference in a new issue