llama-stack/tests/client-sdk/inference
Vladimir Ivić b0c12d280a
Consolidating Inference tests under client-sdk tests (#751)
Summary:
Part of https://github.com/meta-llama/llama-stack/issues/651

We are adding more tests to the clients sdk for some basic coverage.

Those tests are inspired by the inference provider tests.

Test Plan:
Run tests via the command
```
LLAMA_STACK_CONFIG=llama_stack/templates/fireworks/run.yaml pytest tests/client-sdk/inference -v
```

Example output
```
tests/client-sdk/inference/test_inference.py::test_completion_non_streaming PASSED                                                                                                                                        [  7%]
tests/client-sdk/inference/test_inference.py::test_completion_streaming PASSED                                                                                                                                            [ 14%]
tests/client-sdk/inference/test_inference.py::test_completion_log_probs_non_streaming SKIPPED (Needs to be fixed)                                                                                                         [ 21%]
tests/client-sdk/inference/test_inference.py::test_completion_log_probs_streaming SKIPPED (Needs to be fixed)                                                                                                             [ 28%]
tests/client-sdk/inference/test_inference.py::test_completion_structured_output PASSED                                                                                                                                    [ 35%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_non_streaming[What are the names of planets in our solar system?-Earth] PASSED                                                                    [ 42%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_non_streaming[What are the names of the planets that have rings around them?-Saturn] PASSED                                                       [ 50%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_streaming[What's the name of the Sun in latin?-Sol] PASSED                                                                                        [ 57%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_streaming[What is the name of the US captial?-Washington] PASSED                                                                                  [ 64%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_with_tool_calling_and_non_streaming PASSED                                                                                                        [ 71%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_with_tool_calling_and_streaming PASSED                                                                                                            [ 78%]
tests/client-sdk/inference/test_inference.py::test_text_chat_completion_structured_output PASSED                                                                                                                          [ 85%]
tests/client-sdk/inference/test_inference.py::test_image_chat_completion_non_streaming PASSED                                                                                                                             [ 92%]

```
2025-01-13 17:46:02 -08:00
..
__init__.py [tests] add client-sdk pytests & delete client.py (#638) 2024-12-16 12:04:56 -08:00
test_inference.py Consolidating Inference tests under client-sdk tests (#751) 2025-01-13 17:46:02 -08:00