phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions

New quantized models (#301)

Browse source
This commit is contained in:
Ashwin Bharambe 2024-10-24 08:38:56 -07:00 • committed by GitHub
parent 05a8d47b98
commit 7afe51c84d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 292 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

2
llama_stack/apis/inference/client.py
View file

@ -172,7 +172,7 @@ async def run_mm_main(
],
)
cprint(f"User>{message.content}", "green")
iterator = client.chat_completion(
iterator = await client.chat_completion(
model=model,
messages=[message],
stream=stream,