phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions
llama-stack/llama_stack/distribution
History
Ashwin Bharambe 14f973a64f
Make LlamaStackLibraryClient work correctly (#581) 
This PR does a few things:

- it moves "direct client" to llama-stack repo instead of being in the
llama-stack-client-python repo
- renames it to `LlamaStackLibraryClient`
- actually makes synchronous generators work 
- makes streaming and non-streaming work properly

In many ways, this PR makes things finally "work"

## Test Plan

See a `library_client_test.py` I added. This isn't really quite a test
yet but it demonstrates that this mode now works. Here's the invocation
and the response:

```
INFERENCE_MODEL=meta-llama/Llama-3.2-3B-Instruct python llama_stack/distribution/tests/library_client_test.py ollama
```


![image](https://github.com/user-attachments/assets/17d4e116-4457-4755-a14e-d9a668801fe0)
2024-12-07 14:59:36 -08:00
..
routers Telemetry API redesign (#525) 2024-12-04 11:22:45 -08:00
server Small cleanup of console logs 2024-12-06 10:29:24 -08:00
store init registry once (#450) 2024-11-13 22:20:57 -08:00
tests Make LlamaStackLibraryClient work correctly (#581) 2024-12-07 14:59:36 -08:00
ui Add eval/scoring/datasetio API providers to distribution templates & UI developer guide (#564) 2024-12-05 16:29:32 -08:00
utils Ensure model_local_dir does not mangle "C:\" on Windows 2024-11-24 14:18:59 -08:00
__init__.py API Updates (#73) 2024-09-17 19:51:35 -07:00
build.py Make LlamaStackLibraryClient work correctly (#581) 2024-12-07 14:59:36 -08:00
build_conda_env.sh fix prompt guard (#177) 2024-10-03 11:07:53 -07:00
build_container.sh Make run yaml optional so dockers can start with just --env (#492) 2024-11-20 13:11:40 -08:00
client.py use API version in "remote" stack client 2024-11-19 15:59:47 -08:00
common.sh API Updates (#73) 2024-09-17 19:51:35 -07:00
configure.py use logging instead of prints (#499) 2024-11-21 11:32:53 -08:00
configure_container.sh docker: Check for selinux before using --security-opt (#167) 2024-10-02 10:37:41 -07:00
datatypes.py Auto-generate distro yamls + docs (#468) 2024-11-18 14:57:06 -08:00
distribution.py Kill "remote" providers and fix testing with a remote stack properly (#435) 2024-11-12 21:51:29 -08:00
inspect.py Remove "routing_table" and "routing_key" concepts for the user (#201) 2024-10-10 10:24:13 -07:00
library_client.py Make LlamaStackLibraryClient work correctly (#581) 2024-12-07 14:59:36 -08:00
request_headers.py fixes tests & move braintrust api_keys to request headers (#535) 2024-11-26 13:11:21 -08:00
resolver.py another print -> log fix 2024-11-22 13:35:34 -08:00
stack.py use logging instead of prints (#499) 2024-11-21 11:32:53 -08:00
start_conda_env.sh Move to use argparse, fix issues with multiple --env cmdline options 2024-11-18 16:31:59 -08:00
start_container.sh Move to use argparse, fix issues with multiple --env cmdline options 2024-11-18 16:31:59 -08:00