phoenix-oss/llama-stack
1
0
Fork 0
forked from phoenix-oss/llama-stack-mirror
Code Pull requests Releases Packages 2 Activity Actions
llama-stack/docs/openapi_generator
History
Ben Browning 8dfce2f596
feat: OpenAI Responses API (#1989) 
# What does this PR do?

This provides an initial [OpenAI Responses
API](https://platform.openai.com/docs/api-reference/responses)
implementation. The API is not yet complete, and this is more a
proof-of-concept to show how we can store responses in our key-value
stores and use them to support the Responses API concepts like
`previous_response_id`.

## Test Plan

I've added a new
`tests/integration/openai_responses/test_openai_responses.py` as part of
a test-driven development for this new API. I'm only testing this
locally with the remote-vllm provider for now, but it should work with
any of our inference providers since the only API it requires out of the
inference provider is the `openai_chat_completion` endpoint.

```
VLLM_URL="http://localhost:8000/v1" \
INFERENCE_MODEL="meta-llama/Llama-3.2-3B-Instruct" \
llama stack build --template remote-vllm --image-type venv --run
```

```
LLAMA_STACK_CONFIG="http://localhost:8321" \
python -m pytest -v \
  tests/integration/openai_responses/test_openai_responses.py \
  --text-model "meta-llama/Llama-3.2-3B-Instruct"
 ```

---------

Signed-off-by: Ben Browning <bbrownin@redhat.com>
Co-authored-by: Ashwin Bharambe <ashwin.bharambe@gmail.com>
2025-04-28 14:06:00 -07:00
..
pyopenapi feat: OpenAI Responses API (#1989) 2025-04-28 14:06:00 -07:00
generate.py feat: introduce llama4 support (#1877) 2025-04-05 11:53:35 -07:00
README.md docs: Remove duplicate docs on api docs generator (#1534) 2025-03-11 10:01:46 -07:00
run_openapi_generator.sh chore: remove straggler references to llama-models (#1345) 2025-03-01 14:26:03 -08:00

README.md

The RFC Specification (OpenAPI format) is generated from the set of API endpoints located in llama_stack/distribution/server/endpoints.py using the generate.py utility.