mirror of
https://github.com/meta-llama/llama-stack.git
synced 2025-07-24 21:29:53 +00:00
Add version command with comprehensive unit tests and build integration
- Implement llama version command with table and JSON output formats - Add build-time git information capture via scripts/generate_build_info.py - Include comprehensive unit test suite (test_version.py, test_version_simple.py, test_version_integration.py) - Add build system integration with custom setup.py command - Update .gitignore to exclude generated build info file Features: - Display Llama Stack and client versions - Show Python version and build information (git commit, date, branch, tag) - Optional component/provider listing organized by API - Support for both table and JSON output formats - Build-time capture of git metadata for version tracking Testing: - 8 unit tests covering all functionality - Integration tests for CLI execution - Simple dependency-free validation tests - Documentation for test suite and build process Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
This commit is contained in:
parent
dbdc811d16
commit
1da7714bab
11 changed files with 1431 additions and 1 deletions
95
tests/unit/cli/README.md
Normal file
95
tests/unit/cli/README.md
Normal file
|
@ -0,0 +1,95 @@
|
|||
# CLI Unit Tests
|
||||
|
||||
This directory contains unit tests for the Llama Stack CLI commands.
|
||||
|
||||
## Test Files
|
||||
|
||||
### `test_version.py`
|
||||
Comprehensive unit tests for the `llama version` command using pytest and mocking. These tests cover:
|
||||
|
||||
- Package version retrieval
|
||||
- Build information handling
|
||||
- Component information retrieval
|
||||
- JSON and table output formats
|
||||
- Error handling scenarios
|
||||
- Component type detection
|
||||
|
||||
**Requirements:** pytest, unittest.mock
|
||||
|
||||
**Run with:** `pytest tests/unit/cli/test_version.py -v`
|
||||
|
||||
### `test_version_simple.py`
|
||||
Simple unit tests that can run without external dependencies. These tests verify:
|
||||
|
||||
- File structure and existence
|
||||
- Code structure and imports
|
||||
- Logic validation
|
||||
- Configuration correctness
|
||||
|
||||
**Requirements:** None (uses only standard library)
|
||||
|
||||
**Run with:** `python tests/unit/cli/test_version_simple.py`
|
||||
|
||||
### `test_version_integration.py`
|
||||
Integration tests that test the actual CLI command execution. These tests:
|
||||
|
||||
- Execute the actual CLI commands
|
||||
- Verify command-line interface
|
||||
- Test real output formats
|
||||
- Validate build script execution
|
||||
|
||||
**Requirements:** Full Llama Stack environment
|
||||
|
||||
**Run with:** `python tests/unit/cli/test_version_integration.py`
|
||||
|
||||
### `test_stack_config.py`
|
||||
Tests for stack configuration parsing and upgrading.
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Run All CLI Tests
|
||||
```bash
|
||||
# With pytest (if available)
|
||||
pytest tests/unit/cli/ -v
|
||||
|
||||
# Simple tests only (no dependencies)
|
||||
python tests/unit/cli/test_version_simple.py
|
||||
```
|
||||
|
||||
### Run Specific Tests
|
||||
```bash
|
||||
# Run version command tests
|
||||
python tests/unit/cli/test_version_simple.py
|
||||
|
||||
# Run integration tests (requires full environment)
|
||||
python tests/unit/cli/test_version_integration.py
|
||||
```
|
||||
|
||||
## Test Coverage
|
||||
|
||||
The version command tests cover:
|
||||
|
||||
- ✅ Package version detection
|
||||
- ✅ Build information retrieval
|
||||
- ✅ Component discovery and listing
|
||||
- ✅ JSON output format
|
||||
- ✅ Table output format
|
||||
- ✅ Error handling
|
||||
- ✅ CLI integration
|
||||
- ✅ Build script functionality
|
||||
- ✅ File structure validation
|
||||
|
||||
## Adding New Tests
|
||||
|
||||
When adding new CLI commands or functionality:
|
||||
|
||||
1. Create unit tests following the pattern in `test_version.py`
|
||||
2. Add simple validation tests in a `test_<command>_simple.py` file
|
||||
3. Consider integration tests for end-to-end validation
|
||||
4. Update this README with the new test information
|
||||
|
||||
## Notes
|
||||
|
||||
- Simple tests are preferred for CI/CD as they have no external dependencies
|
||||
- Integration tests are useful for local development and full validation
|
||||
- Mock-based tests provide comprehensive coverage of edge cases
|
414
tests/unit/cli/test_version.py
Normal file
414
tests/unit/cli/test_version.py
Normal file
|
@ -0,0 +1,414 @@
|
|||
# Copyright (c) Meta Platforms, Inc. and affiliates.
|
||||
# All rights reserved.
|
||||
#
|
||||
# This source code is licensed under the terms described in the LICENSE file in
|
||||
# the root directory of this source tree.
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
from io import StringIO
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from llama_stack.cli.version import VersionCommand
|
||||
|
||||
|
||||
class TestVersionCommand:
|
||||
"""Test suite for the VersionCommand class"""
|
||||
|
||||
@pytest.fixture
|
||||
def version_command(self):
|
||||
"""Create a VersionCommand instance for testing"""
|
||||
# Create a mock subparsers object
|
||||
parser = argparse.ArgumentParser()
|
||||
subparsers = parser.add_subparsers()
|
||||
return VersionCommand(subparsers)
|
||||
|
||||
@pytest.fixture
|
||||
def mock_build_info(self):
|
||||
"""Mock build info data"""
|
||||
return {
|
||||
"git_commit": "abc123def456",
|
||||
"git_commit_date": "2025-01-15 10:30:00 -0800",
|
||||
"git_branch": "main",
|
||||
"git_tag": "v0.2.12",
|
||||
"build_timestamp": "2025-01-15T18:30:00.123456+00:00",
|
||||
}
|
||||
|
||||
@pytest.fixture
|
||||
def mock_components_info(self):
|
||||
"""Mock components info data"""
|
||||
return [
|
||||
{
|
||||
"api": "inference",
|
||||
"component": "meta-reference",
|
||||
"type": "inline",
|
||||
"provider_type": "inline::meta-reference",
|
||||
},
|
||||
{
|
||||
"api": "inference",
|
||||
"component": "vllm",
|
||||
"type": "inline",
|
||||
"provider_type": "inline::vllm",
|
||||
},
|
||||
{
|
||||
"api": "inference",
|
||||
"component": "ollama",
|
||||
"type": "remote",
|
||||
"provider_type": "remote::ollama",
|
||||
},
|
||||
{
|
||||
"api": "safety",
|
||||
"component": "llama-guard",
|
||||
"type": "inline",
|
||||
"provider_type": "inline::llama-guard",
|
||||
},
|
||||
]
|
||||
|
||||
def test_get_package_version_existing_package(self, version_command):
|
||||
"""Test getting version of an existing package"""
|
||||
with patch("llama_stack.cli.version.version") as mock_version:
|
||||
mock_version.return_value = "1.2.3"
|
||||
result = version_command._get_package_version("test-package")
|
||||
assert result == "1.2.3"
|
||||
mock_version.assert_called_once_with("test-package")
|
||||
|
||||
def test_get_package_version_missing_package(self, version_command):
|
||||
"""Test getting version of a non-existent package"""
|
||||
with patch("llama_stack.cli.version.version") as mock_version:
|
||||
from importlib.metadata import PackageNotFoundError
|
||||
|
||||
mock_version.side_effect = PackageNotFoundError()
|
||||
result = version_command._get_package_version("non-existent-package")
|
||||
assert result == "unknown"
|
||||
|
||||
def test_get_build_info_with_build_info_module(self, version_command, mock_build_info):
|
||||
"""Test getting build info when build_info module is available"""
|
||||
mock_build_info_dict = {
|
||||
"git_commit": mock_build_info["git_commit"],
|
||||
"git_commit_date": mock_build_info["git_commit_date"],
|
||||
"git_branch": mock_build_info["git_branch"],
|
||||
"git_tag": mock_build_info["git_tag"],
|
||||
"build_timestamp": mock_build_info["build_timestamp"],
|
||||
}
|
||||
|
||||
with patch("llama_stack.cli.version.BUILD_INFO", mock_build_info_dict):
|
||||
result = version_command._get_build_info()
|
||||
assert result["commit_hash"] == mock_build_info["git_commit"]
|
||||
assert result["commit_date"] == mock_build_info["git_commit_date"]
|
||||
assert result["branch"] == mock_build_info["git_branch"]
|
||||
assert result["tag"] == mock_build_info["git_tag"]
|
||||
assert result["build_timestamp"] == mock_build_info["build_timestamp"]
|
||||
|
||||
def test_get_build_info_without_build_info_module(self, version_command):
|
||||
"""Test getting build info when build_info module is not available"""
|
||||
with patch("llama_stack.cli.version.BUILD_INFO", side_effect=ImportError()):
|
||||
result = version_command._get_build_info()
|
||||
assert result["commit_hash"] == "unknown"
|
||||
assert result["commit_date"] == "unknown"
|
||||
assert result["branch"] == "unknown"
|
||||
assert result["tag"] == "unknown"
|
||||
assert result["build_timestamp"] == "unknown"
|
||||
|
||||
def test_get_components_info_success(self, version_command, mock_components_info):
|
||||
"""Test getting components info successfully"""
|
||||
# Mock the provider registry
|
||||
mock_registry = {
|
||||
"inference": {
|
||||
"inline::meta-reference": Mock(
|
||||
api=Mock(value="inference"),
|
||||
provider_type="inline::meta-reference",
|
||||
adapter_type=None,
|
||||
),
|
||||
"inline::vllm": Mock(
|
||||
api=Mock(value="inference"),
|
||||
provider_type="inline::vllm",
|
||||
adapter_type=None,
|
||||
),
|
||||
"remote::ollama": Mock(
|
||||
api=Mock(value="inference"),
|
||||
provider_type="remote::ollama",
|
||||
adapter_type="ollama",
|
||||
),
|
||||
},
|
||||
"safety": {
|
||||
"inline::llama-guard": Mock(
|
||||
api=Mock(value="safety"),
|
||||
provider_type="inline::llama-guard",
|
||||
adapter_type=None,
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
with patch("llama_stack.cli.version.get_provider_registry") as mock_get_registry:
|
||||
mock_get_registry.return_value = mock_registry
|
||||
result = version_command._get_components_info()
|
||||
|
||||
# Should have 4 components
|
||||
assert len(result) == 4
|
||||
|
||||
# Check that all expected components are present
|
||||
component_names = [comp["component"] for comp in result]
|
||||
assert "meta-reference" in component_names
|
||||
assert "vllm" in component_names
|
||||
assert "ollama" in component_names
|
||||
assert "llama-guard" in component_names
|
||||
|
||||
def test_get_components_info_exception(self, version_command):
|
||||
"""Test getting components info when an exception occurs"""
|
||||
with patch("llama_stack.cli.version.get_provider_registry") as mock_get_registry:
|
||||
mock_get_registry.side_effect = Exception("Test error")
|
||||
|
||||
# Capture stderr to check warning message
|
||||
with patch("sys.stderr", new_callable=StringIO) as mock_stderr:
|
||||
result = version_command._get_components_info()
|
||||
assert result == []
|
||||
assert "Warning: Could not load components information" in mock_stderr.getvalue()
|
||||
|
||||
def test_run_version_command_table_format(self, version_command, mock_build_info):
|
||||
"""Test running version command with table format"""
|
||||
args = argparse.Namespace(format="table", components=False)
|
||||
|
||||
with (
|
||||
patch.object(version_command, "_get_package_version") as mock_get_version,
|
||||
patch.object(version_command, "_get_build_info") as mock_get_build_info,
|
||||
patch("llama_stack.cli.version.print_table") as mock_print_table,
|
||||
patch("builtins.print") as mock_print,
|
||||
):
|
||||
mock_get_version.side_effect = lambda pkg: {
|
||||
"llama-stack": "0.2.12",
|
||||
"llama-stack-client": "0.2.12",
|
||||
}.get(pkg, "unknown")
|
||||
|
||||
mock_get_build_info.return_value = {
|
||||
"commit_hash": mock_build_info["git_commit"],
|
||||
"commit_date": mock_build_info["git_commit_date"],
|
||||
"branch": mock_build_info["git_branch"],
|
||||
"tag": mock_build_info["git_tag"],
|
||||
"build_timestamp": mock_build_info["build_timestamp"],
|
||||
}
|
||||
|
||||
version_command._run_version_command(args)
|
||||
|
||||
# Check that print was called with headers
|
||||
mock_print.assert_any_call("Llama Stack Version Information")
|
||||
mock_print.assert_any_call("=" * 50)
|
||||
mock_print.assert_any_call("\nBuild Information")
|
||||
mock_print.assert_any_call("-" * 30)
|
||||
|
||||
# Check that print_table was called twice (version and build info)
|
||||
assert mock_print_table.call_count == 2
|
||||
|
||||
def test_run_version_command_json_format(self, version_command, mock_build_info):
|
||||
"""Test running version command with JSON format"""
|
||||
args = argparse.Namespace(format="json", components=False)
|
||||
|
||||
with (
|
||||
patch.object(version_command, "_get_package_version") as mock_get_version,
|
||||
patch.object(version_command, "_get_build_info") as mock_get_build_info,
|
||||
patch("builtins.print") as mock_print,
|
||||
):
|
||||
mock_get_version.side_effect = lambda pkg: {
|
||||
"llama-stack": "0.2.12",
|
||||
"llama-stack-client": "0.2.12",
|
||||
}.get(pkg, "unknown")
|
||||
|
||||
mock_get_build_info.return_value = {
|
||||
"commit_hash": mock_build_info["git_commit"],
|
||||
"commit_date": mock_build_info["git_commit_date"],
|
||||
"branch": mock_build_info["git_branch"],
|
||||
"tag": mock_build_info["git_tag"],
|
||||
"build_timestamp": mock_build_info["build_timestamp"],
|
||||
}
|
||||
|
||||
version_command._run_version_command(args)
|
||||
|
||||
# Check that JSON was printed
|
||||
mock_print.assert_called_once()
|
||||
printed_output = mock_print.call_args[0][0]
|
||||
|
||||
# Parse the JSON to verify it's valid and contains expected fields
|
||||
json_output = json.loads(printed_output)
|
||||
assert json_output["llama_stack_version"] == "0.2.12"
|
||||
assert json_output["llama_stack_client_version"] == "0.2.12"
|
||||
assert json_output["git_commit"] == mock_build_info["git_commit"]
|
||||
assert json_output["git_branch"] == mock_build_info["git_branch"]
|
||||
assert json_output["build_timestamp"] == mock_build_info["build_timestamp"]
|
||||
|
||||
def test_run_version_command_with_components_table(self, version_command, mock_build_info, mock_components_info):
|
||||
"""Test running version command with components in table format"""
|
||||
args = argparse.Namespace(format="table", components=True)
|
||||
|
||||
with (
|
||||
patch.object(version_command, "_get_package_version") as mock_get_version,
|
||||
patch.object(version_command, "_get_build_info") as mock_get_build_info,
|
||||
patch.object(version_command, "_get_components_info") as mock_get_components_info,
|
||||
patch("builtins.print") as mock_print,
|
||||
):
|
||||
mock_get_version.side_effect = lambda pkg: {
|
||||
"llama-stack": "0.2.12",
|
||||
"llama-stack-client": "0.2.12",
|
||||
}.get(pkg, "unknown")
|
||||
|
||||
mock_get_build_info.return_value = {
|
||||
"commit_hash": mock_build_info["git_commit"],
|
||||
"commit_date": mock_build_info["git_commit_date"],
|
||||
"branch": mock_build_info["git_branch"],
|
||||
"tag": mock_build_info["git_tag"],
|
||||
"build_timestamp": mock_build_info["build_timestamp"],
|
||||
}
|
||||
|
||||
mock_get_components_info.return_value = mock_components_info
|
||||
|
||||
version_command._run_version_command(args)
|
||||
|
||||
# Check that components section was printed
|
||||
mock_print.assert_any_call("\nAvailable Components")
|
||||
mock_print.assert_any_call("-" * 30)
|
||||
|
||||
# Check that API sections were printed
|
||||
mock_print.assert_any_call("\nINFERENCE API:")
|
||||
mock_print.assert_any_call("\nSAFETY API:")
|
||||
|
||||
def test_run_version_command_with_components_json(self, version_command, mock_build_info, mock_components_info):
|
||||
"""Test running version command with components in JSON format"""
|
||||
args = argparse.Namespace(format="json", components=True)
|
||||
|
||||
with (
|
||||
patch.object(version_command, "_get_package_version") as mock_get_version,
|
||||
patch.object(version_command, "_get_build_info") as mock_get_build_info,
|
||||
patch.object(version_command, "_get_components_info") as mock_get_components_info,
|
||||
patch("builtins.print") as mock_print,
|
||||
):
|
||||
mock_get_version.side_effect = lambda pkg: {
|
||||
"llama-stack": "0.2.12",
|
||||
"llama-stack-client": "0.2.12",
|
||||
}.get(pkg, "unknown")
|
||||
|
||||
mock_get_build_info.return_value = {
|
||||
"commit_hash": mock_build_info["git_commit"],
|
||||
"commit_date": mock_build_info["git_commit_date"],
|
||||
"branch": mock_build_info["git_branch"],
|
||||
"tag": mock_build_info["git_tag"],
|
||||
"build_timestamp": mock_build_info["build_timestamp"],
|
||||
}
|
||||
|
||||
mock_get_components_info.return_value = mock_components_info
|
||||
|
||||
version_command._run_version_command(args)
|
||||
|
||||
# Check that JSON was printed
|
||||
mock_print.assert_called_once()
|
||||
printed_output = mock_print.call_args[0][0]
|
||||
|
||||
# Parse the JSON to verify it contains components
|
||||
json_output = json.loads(printed_output)
|
||||
assert "components" in json_output
|
||||
assert len(json_output["components"]) == 4
|
||||
|
||||
# Check that components have expected structure
|
||||
component = json_output["components"][0]
|
||||
assert "api" in component
|
||||
assert "component" in component
|
||||
assert "type" in component
|
||||
assert "provider_type" in component
|
||||
|
||||
def test_run_version_command_no_components_available(self, version_command, mock_build_info):
|
||||
"""Test running version command when no components are available"""
|
||||
args = argparse.Namespace(format="table", components=True)
|
||||
|
||||
with (
|
||||
patch.object(version_command, "_get_package_version") as mock_get_version,
|
||||
patch.object(version_command, "_get_build_info") as mock_get_build_info,
|
||||
patch.object(version_command, "_get_components_info") as mock_get_components_info,
|
||||
patch("builtins.print") as mock_print,
|
||||
):
|
||||
mock_get_version.side_effect = lambda pkg: {
|
||||
"llama-stack": "0.2.12",
|
||||
"llama-stack-client": "0.2.12",
|
||||
}.get(pkg, "unknown")
|
||||
|
||||
mock_get_build_info.return_value = {
|
||||
"commit_hash": mock_build_info["git_commit"],
|
||||
"commit_date": mock_build_info["git_commit_date"],
|
||||
"branch": mock_build_info["git_branch"],
|
||||
"tag": mock_build_info["git_tag"],
|
||||
"build_timestamp": mock_build_info["build_timestamp"],
|
||||
}
|
||||
|
||||
mock_get_components_info.return_value = []
|
||||
|
||||
version_command._run_version_command(args)
|
||||
|
||||
# Check that "no components" message was printed
|
||||
mock_print.assert_any_call("No components information available")
|
||||
|
||||
def test_python_version_format(self, version_command):
|
||||
"""Test that Python version is formatted correctly"""
|
||||
args = argparse.Namespace(format="json", components=False)
|
||||
|
||||
with (
|
||||
patch.object(version_command, "_get_package_version") as mock_get_version,
|
||||
patch.object(version_command, "_get_build_info") as mock_get_build_info,
|
||||
patch("builtins.print") as mock_print,
|
||||
):
|
||||
mock_get_version.return_value = "0.2.12"
|
||||
mock_get_build_info.return_value = {
|
||||
"commit_hash": "abc123",
|
||||
"commit_date": "2025-01-15",
|
||||
"branch": "main",
|
||||
"tag": "v0.2.12",
|
||||
"build_timestamp": "2025-01-15T18:30:00+00:00",
|
||||
}
|
||||
|
||||
version_command._run_version_command(args)
|
||||
|
||||
printed_output = mock_print.call_args[0][0]
|
||||
json_output = json.loads(printed_output)
|
||||
|
||||
# Check that Python version matches current Python version
|
||||
expected_python_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
assert json_output["python_version"] == expected_python_version
|
||||
|
||||
def test_component_type_detection(self, version_command):
|
||||
"""Test that component types are detected correctly"""
|
||||
# Test inline provider
|
||||
mock_registry = {
|
||||
"inference": {
|
||||
"inline::test": Mock(
|
||||
api=Mock(value="inference"),
|
||||
provider_type="inline::test",
|
||||
adapter_type=None,
|
||||
),
|
||||
"remote::test": Mock(
|
||||
api=Mock(value="inference"),
|
||||
provider_type="remote::test",
|
||||
adapter_type=None,
|
||||
),
|
||||
"adapter-test": Mock(
|
||||
api=Mock(value="inference"),
|
||||
provider_type="adapter-test",
|
||||
adapter_type="test-adapter",
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
with patch("llama_stack.cli.version.get_provider_registry") as mock_get_registry:
|
||||
mock_get_registry.return_value = mock_registry
|
||||
result = version_command._get_components_info()
|
||||
|
||||
# Find components by provider type to avoid conflicts
|
||||
inline_components = [comp for comp in result if comp["provider_type"] == "inline::test"]
|
||||
remote_components = [comp for comp in result if comp["provider_type"] == "remote::test"]
|
||||
adapter_components = [comp for comp in result if comp["provider_type"] == "adapter-test"]
|
||||
|
||||
assert len(inline_components) == 1
|
||||
assert inline_components[0]["type"] == "inline"
|
||||
|
||||
assert len(remote_components) == 1
|
||||
assert remote_components[0]["type"] == "remote"
|
||||
|
||||
assert len(adapter_components) == 1
|
||||
assert adapter_components[0]["type"] == "remote"
|
220
tests/unit/cli/test_version_integration.py
Normal file
220
tests/unit/cli/test_version_integration.py
Normal file
|
@ -0,0 +1,220 @@
|
|||
# Copyright (c) Meta Platforms, Inc. and affiliates.
|
||||
# All rights reserved.
|
||||
#
|
||||
# This source code is licensed under the terms described in the LICENSE file in
|
||||
# the root directory of this source tree.
|
||||
|
||||
"""
|
||||
Integration tests for the version command that can be run independently.
|
||||
These tests verify the basic functionality without requiring complex mocking.
|
||||
"""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def test_version_command_help():
|
||||
"""Test that the version command help works"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "llama_stack.cli.llama", "version", "--help"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=Path(__file__).parent.parent.parent.parent,
|
||||
)
|
||||
|
||||
assert result.returncode == 0
|
||||
assert "Display version information" in result.stdout
|
||||
assert "--format" in result.stdout
|
||||
assert "--components" in result.stdout
|
||||
print("✓ Version command help test passed")
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
print("✗ Version command help test timed out")
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"✗ Version command help test failed: {e}")
|
||||
raise
|
||||
|
||||
|
||||
def test_version_command_basic():
|
||||
"""Test basic version command execution"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "llama_stack.cli.llama", "version", "--format", "json"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=Path(__file__).parent.parent.parent.parent,
|
||||
)
|
||||
|
||||
assert result.returncode == 0
|
||||
|
||||
# Parse JSON output
|
||||
json_output = json.loads(result.stdout.split("\n")[-2]) # Get last non-empty line
|
||||
|
||||
# Check required fields
|
||||
required_fields = [
|
||||
"llama_stack_version",
|
||||
"llama_stack_client_version",
|
||||
"python_version",
|
||||
"git_commit",
|
||||
"git_commit_date",
|
||||
"git_branch",
|
||||
"git_tag",
|
||||
"build_timestamp",
|
||||
]
|
||||
|
||||
for field in required_fields:
|
||||
assert field in json_output, f"Missing field: {field}"
|
||||
|
||||
# Check that values are not empty (except for potentially unknown values)
|
||||
assert json_output["python_version"] != ""
|
||||
assert "." in json_output["python_version"] # Should be in format x.y.z
|
||||
|
||||
print("✓ Version command basic test passed")
|
||||
print(f" Llama Stack version: {json_output['llama_stack_version']}")
|
||||
print(f" Python version: {json_output['python_version']}")
|
||||
print(f" Git commit: {json_output['git_commit']}")
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
print("✗ Version command basic test timed out")
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"✗ Version command basic test failed: {e}")
|
||||
raise
|
||||
|
||||
|
||||
def test_version_command_with_components():
|
||||
"""Test version command with components flag"""
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "llama_stack.cli.llama", "version", "--format", "json", "--components"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=60, # Longer timeout for components
|
||||
cwd=Path(__file__).parent.parent.parent.parent,
|
||||
)
|
||||
|
||||
assert result.returncode == 0
|
||||
|
||||
# Parse JSON output
|
||||
json_output = json.loads(result.stdout.split("\n")[-2]) # Get last non-empty line
|
||||
|
||||
# Check that components field exists
|
||||
assert "components" in json_output
|
||||
assert isinstance(json_output["components"], list)
|
||||
|
||||
# If components exist, check their structure
|
||||
if json_output["components"]:
|
||||
component = json_output["components"][0]
|
||||
required_component_fields = ["api", "component", "type", "provider_type"]
|
||||
for field in required_component_fields:
|
||||
assert field in component, f"Missing component field: {field}"
|
||||
|
||||
print("✓ Version command with components test passed")
|
||||
print(f" Found {len(json_output['components'])} components")
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
print("✗ Version command with components test timed out")
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"✗ Version command with components test failed: {e}")
|
||||
raise
|
||||
|
||||
|
||||
def test_build_info_structure():
|
||||
"""Test that build_info.py has the correct structure"""
|
||||
try:
|
||||
# Import build info directly
|
||||
build_info_path = Path(__file__).parent.parent.parent.parent / "llama_stack" / "cli" / "build_info.py"
|
||||
|
||||
if build_info_path.exists():
|
||||
# Read the file content
|
||||
content = build_info_path.read_text()
|
||||
|
||||
# Check that it contains BUILD_INFO
|
||||
assert "BUILD_INFO" in content
|
||||
|
||||
# Check that it has the expected fields
|
||||
expected_fields = [
|
||||
"git_commit",
|
||||
"git_commit_date",
|
||||
"git_branch",
|
||||
"git_tag",
|
||||
"build_timestamp",
|
||||
]
|
||||
|
||||
for field in expected_fields:
|
||||
assert field in content, f"Missing field in build_info.py: {field}"
|
||||
|
||||
print("✓ Build info structure test passed")
|
||||
else:
|
||||
print("! Build info file not found - this is expected in development")
|
||||
|
||||
except Exception as e:
|
||||
print(f"✗ Build info structure test failed: {e}")
|
||||
raise
|
||||
|
||||
|
||||
def test_build_script_execution():
|
||||
"""Test that the build script can be executed"""
|
||||
try:
|
||||
script_path = Path(__file__).parent.parent.parent.parent / "scripts" / "generate_build_info.py"
|
||||
|
||||
if script_path.exists():
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script_path)],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
cwd=script_path.parent.parent,
|
||||
)
|
||||
|
||||
assert result.returncode == 0
|
||||
assert "Generated build info file" in result.stdout
|
||||
|
||||
print("✓ Build script execution test passed")
|
||||
else:
|
||||
print("! Build script not found")
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
print("✗ Build script execution test timed out")
|
||||
raise
|
||||
except Exception as e:
|
||||
print(f"✗ Build script execution test failed: {e}")
|
||||
raise
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
"""Run integration tests when executed directly"""
|
||||
print("Running version command integration tests...")
|
||||
|
||||
tests = [
|
||||
test_version_command_help,
|
||||
test_version_command_basic,
|
||||
test_version_command_with_components,
|
||||
test_build_info_structure,
|
||||
test_build_script_execution,
|
||||
]
|
||||
|
||||
passed = 0
|
||||
failed = 0
|
||||
|
||||
for test in tests:
|
||||
try:
|
||||
test()
|
||||
passed += 1
|
||||
except Exception as e:
|
||||
print(f"Test {test.__name__} failed: {e}")
|
||||
failed += 1
|
||||
|
||||
print(f"\nResults: {passed} passed, {failed} failed")
|
||||
|
||||
if failed > 0:
|
||||
sys.exit(1)
|
||||
else:
|
||||
print("All integration tests passed!")
|
231
tests/unit/cli/test_version_simple.py
Normal file
231
tests/unit/cli/test_version_simple.py
Normal file
|
@ -0,0 +1,231 @@
|
|||
# Copyright (c) Meta Platforms, Inc. and affiliates.
|
||||
# All rights reserved.
|
||||
#
|
||||
# This source code is licensed under the terms described in the LICENSE file in
|
||||
# the root directory of this source tree.
|
||||
|
||||
"""
|
||||
Simple unit tests for the version command that can be run independently.
|
||||
These tests focus on testing individual methods and functionality.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def test_build_info_file_structure():
|
||||
"""Test that build_info.py has the correct structure when it exists"""
|
||||
build_info_path = Path(__file__).parent.parent.parent.parent / "llama_stack" / "cli" / "build_info.py"
|
||||
|
||||
if build_info_path.exists():
|
||||
content = build_info_path.read_text()
|
||||
|
||||
# Check that it contains BUILD_INFO
|
||||
assert "BUILD_INFO" in content, "build_info.py should contain BUILD_INFO"
|
||||
|
||||
# Check that it has the expected fields
|
||||
expected_fields = [
|
||||
"git_commit",
|
||||
"git_commit_date",
|
||||
"git_branch",
|
||||
"git_tag",
|
||||
"build_timestamp",
|
||||
]
|
||||
|
||||
for field in expected_fields:
|
||||
assert field in content, f"Missing field in build_info.py: {field}"
|
||||
|
||||
print("✓ Build info file structure test passed")
|
||||
return True
|
||||
else:
|
||||
print("! Build info file not found - this is expected in development")
|
||||
return True
|
||||
|
||||
|
||||
def test_build_script_exists():
|
||||
"""Test that the build script exists and has the correct structure"""
|
||||
script_path = Path(__file__).parent.parent.parent.parent / "scripts" / "generate_build_info.py"
|
||||
|
||||
assert script_path.exists(), "Build script should exist"
|
||||
|
||||
content = script_path.read_text()
|
||||
|
||||
# Check for key functions
|
||||
assert "def get_git_info" in content, "Build script should have get_git_info function"
|
||||
assert "def generate_build_info_file" in content, "Build script should have generate_build_info_file function"
|
||||
assert "BUILD_INFO" in content, "Build script should reference BUILD_INFO"
|
||||
|
||||
print("✓ Build script exists and has correct structure")
|
||||
return True
|
||||
|
||||
|
||||
def test_version_module_structure():
|
||||
"""Test that the version module has the correct structure"""
|
||||
version_path = Path(__file__).parent.parent.parent.parent / "llama_stack" / "cli" / "version.py"
|
||||
|
||||
assert version_path.exists(), "Version module should exist"
|
||||
|
||||
content = version_path.read_text()
|
||||
|
||||
# Check for key classes and methods
|
||||
assert "class VersionCommand" in content, "Should have VersionCommand class"
|
||||
assert "def _get_package_version" in content, "Should have _get_package_version method"
|
||||
assert "def _get_build_info" in content, "Should have _get_build_info method"
|
||||
assert "def _get_components_info" in content, "Should have _get_components_info method"
|
||||
assert "def _run_version_command" in content, "Should have _run_version_command method"
|
||||
|
||||
# Check for proper imports
|
||||
assert "from llama_stack.cli.subcommand import Subcommand" in content, "Should import Subcommand"
|
||||
assert "from llama_stack.distribution.distribution import get_provider_registry" in content, (
|
||||
"Should import get_provider_registry"
|
||||
)
|
||||
|
||||
print("✓ Version module structure test passed")
|
||||
return True
|
||||
|
||||
|
||||
def test_cli_integration():
|
||||
"""Test that the version command is properly integrated into the CLI"""
|
||||
llama_cli_path = Path(__file__).parent.parent.parent.parent / "llama_stack" / "cli" / "llama.py"
|
||||
|
||||
assert llama_cli_path.exists(), "Main CLI module should exist"
|
||||
|
||||
content = llama_cli_path.read_text()
|
||||
|
||||
# Check that version command is imported and added
|
||||
assert "from .version import VersionCommand" in content, "Should import VersionCommand"
|
||||
assert "VersionCommand.create(subparsers)" in content, "Should add VersionCommand to subparsers"
|
||||
|
||||
print("✓ CLI integration test passed")
|
||||
return True
|
||||
|
||||
|
||||
def test_gitignore_entry():
|
||||
"""Test that build_info.py is properly ignored in git"""
|
||||
gitignore_path = Path(__file__).parent.parent.parent.parent / ".gitignore"
|
||||
|
||||
if gitignore_path.exists():
|
||||
content = gitignore_path.read_text()
|
||||
assert "llama_stack/cli/build_info.py" in content, "build_info.py should be in .gitignore"
|
||||
print("✓ Gitignore entry test passed")
|
||||
return True
|
||||
else:
|
||||
print("! .gitignore not found")
|
||||
return True
|
||||
|
||||
|
||||
def test_component_type_detection_logic():
|
||||
"""Test the component type detection logic"""
|
||||
|
||||
# Simulate the component type detection logic from the version command
|
||||
def detect_component_type(provider_type, adapter_type=None):
|
||||
if provider_type.startswith("inline::"):
|
||||
return "inline", provider_type.replace("inline::", "")
|
||||
elif provider_type.startswith("remote::"):
|
||||
return "remote", provider_type.replace("remote::", "")
|
||||
elif adapter_type:
|
||||
return "remote", adapter_type
|
||||
else:
|
||||
return "unknown", provider_type
|
||||
|
||||
# Test cases
|
||||
test_cases = [
|
||||
("inline::meta-reference", None, "inline", "meta-reference"),
|
||||
("remote::ollama", None, "remote", "ollama"),
|
||||
("some-provider", "adapter-name", "remote", "adapter-name"),
|
||||
("unknown-provider", None, "unknown", "unknown-provider"),
|
||||
]
|
||||
|
||||
for provider_type, adapter_type, expected_type, expected_name in test_cases:
|
||||
comp_type, comp_name = detect_component_type(provider_type, adapter_type)
|
||||
assert comp_type == expected_type, f"Expected type {expected_type}, got {comp_type}"
|
||||
assert comp_name == expected_name, f"Expected name {expected_name}, got {comp_name}"
|
||||
|
||||
print("✓ Component type detection logic test passed")
|
||||
return True
|
||||
|
||||
|
||||
def test_python_version_format():
|
||||
"""Test that Python version formatting works correctly"""
|
||||
|
||||
# Simulate the Python version formatting from the version command
|
||||
python_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
|
||||
# Check format
|
||||
parts = python_version.split(".")
|
||||
assert len(parts) == 3, "Python version should have 3 parts"
|
||||
assert all(part.isdigit() for part in parts), "All parts should be numeric"
|
||||
|
||||
print(f"✓ Python version format test passed: {python_version}")
|
||||
return True
|
||||
|
||||
|
||||
def test_json_output_structure():
|
||||
"""Test the expected JSON output structure"""
|
||||
|
||||
# Expected structure for JSON output
|
||||
expected_fields = {
|
||||
"llama_stack_version": str,
|
||||
"llama_stack_client_version": str,
|
||||
"python_version": str,
|
||||
"git_commit": str,
|
||||
"git_commit_date": str,
|
||||
"git_branch": str,
|
||||
"git_tag": str,
|
||||
"build_timestamp": str,
|
||||
}
|
||||
|
||||
# Test that we can create a valid JSON structure
|
||||
test_data = {field: "test_value" for field in expected_fields.keys()}
|
||||
|
||||
# Should be valid JSON
|
||||
json_str = json.dumps(test_data, indent=2)
|
||||
parsed = json.loads(json_str)
|
||||
|
||||
# Should have all expected fields
|
||||
for field in expected_fields.keys():
|
||||
assert field in parsed, f"Missing field: {field}"
|
||||
|
||||
print("✓ JSON output structure test passed")
|
||||
return True
|
||||
|
||||
|
||||
def run_all_tests():
|
||||
"""Run all simple unit tests"""
|
||||
tests = [
|
||||
test_build_info_file_structure,
|
||||
test_build_script_exists,
|
||||
test_version_module_structure,
|
||||
test_cli_integration,
|
||||
test_gitignore_entry,
|
||||
test_component_type_detection_logic,
|
||||
test_python_version_format,
|
||||
test_json_output_structure,
|
||||
]
|
||||
|
||||
passed = 0
|
||||
failed = 0
|
||||
|
||||
print("Running simple version command unit tests...")
|
||||
|
||||
for test in tests:
|
||||
try:
|
||||
if test():
|
||||
passed += 1
|
||||
except Exception as e:
|
||||
print(f"✗ Test {test.__name__} failed: {e}")
|
||||
failed += 1
|
||||
|
||||
print(f"\nResults: {passed} passed, {failed} failed")
|
||||
|
||||
return failed == 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
"""Run tests when executed directly"""
|
||||
success = run_all_tests()
|
||||
if not success:
|
||||
sys.exit(1)
|
||||
else:
|
||||
print("All simple unit tests passed!")
|
Loading…
Add table
Add a link
Reference in a new issue