phoenix-oss/llama-stack-mirror
0
0
Fork 1
mirror of https://github.com/meta-llama/llama-stack.git synced 2025-12-17 13:02:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add Ollama inference mocks

Browse source 
Summary:
This commit adds mock support for Ollama inference testing.

Use  `--mock-overrides` during your test run:
```
pytest llama_stack/providers/tests/inference/test_text_inference.py -m "ollama" --mock-overrides inference=ollama --inference-model Llama3.2-1B-Instruct
```

The test will run using Ollama provider using mock Adapter.

Test Plan:
Run tests

```
pytest llama_stack/providers/tests/inference/test_text_inference.py -m "ollama" --mock-overrides inference=ollama --inference-model Llama3.2-1B-Instruct -v -s --tb=short --disable-warnings

====================================================================================================== test session starts ======================================================================================================
platform darwin -- Python 3.11.10, pytest-8.3.3, pluggy-1.5.0 -- /opt/homebrew/Caskroom/miniconda/base/envs/llama-stack/bin/python
cachedir: .pytest_cache
rootdir: /Users/vivic/Code/llama-stack
configfile: pyproject.toml
plugins: asyncio-0.24.0, anyio-4.6.2.post1
asyncio: mode=Mode.STRICT, default_loop_scope=None
collected 56 items / 48 deselected / 8 selected

llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_model_list[-ollama] Overriding inference=ollama with mocks from inference_ollama_mocks
Resolved 4 providers
 inner-inference => ollama
 models => __routing_table__
 inference => __autorouted__
 inspect => __builtin__

Models: Llama3.2-1B-Instruct served by ollama

PASSED
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_completion[-ollama] PASSED
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_completions_structured_output[-ollama] SKIPPED (This test is not quite robust)
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_chat_completion_non_streaming[-ollama] PASSED
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_structured_output[-ollama] SKIPPED (Other inference providers don't support structured output yet)
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_chat_completion_streaming[-ollama] PASSED
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_chat_completion_with_tool_calling[-ollama] PASSED
llama_stack/providers/tests/inference/test_text_inference.py::TestInference::test_chat_completion_with_tool_calling_streaming[-ollama] PASSED

==================================================================================== 6 passed, 2 skipped, 48 deselected, 6 warnings in 0.11s ====================================================================================
```
This commit is contained in:
Vladimir Ivic 2024-11-21 15:35:55 -08:00
parent ac1791f8b1
commit 560467e6fe
2 changed files with 223 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

17
llama_stack/providers/tests/inference/fixtures.py
View file

@ -23,8 +23,9 @@ from llama_stack.providers.remote.inference.together import TogetherImplConfig
from llama_stack.providers.remote.inference.vllm import VLLMInferenceAdapterConfig
from llama_stack.providers.tests.resolver import construct_stack_for_test
from ..conftest import ProviderFixture, remote_stack_fixture
from ..conftest import ProviderFixture, remote_stack_fixture, should_use_mock_overrides
from ..env import get_env_or_fail
from .mocks import * # noqa
@pytest.fixture(scope="session")
@ -182,6 +183,18 @@ INFERENCE_FIXTURES = [
async def inference_stack(request, inference_model):
fixture_name = request.param
inference_fixture = request.getfixturevalue(f"inference_{fixture_name}")
# Setup mocks if they are specified via the command line and they are defined
if should_use_mock_overrides(
request, f"inference={fixture_name}", f"inference_{fixture_name}_mocks"
):
try:
request.getfixturevalue(f"inference_{fixture_name}_mocks")
except pytest.FixtureLookupError:
print(
f"Fixture inference_{fixture_name}_mocks not implemented, skipping mocks."
)
test_stack = await construct_stack_for_test(
[Api.inference],
{"inference": inference_fixture.providers},
@ -189,4 +202,4 @@ async def inference_stack(request, inference_model):
models=[ModelInput(model_id=inference_model)],
)
return test_stack.impls[Api.inference], test_stack.impls[Api.models]
yield test_stack.impls[Api.inference], test_stack.impls[Api.models]