Add graphical unit tests

CLAUDE.md

@@ -186,12 +186,36 @@ The `unittest.sh` script:
 - Sets up the proper paths and heapsize
 - Can run tests on device using `mpremote` with the `ondevice` argument
 - Runs all `test_*.py` files when no argument is provided
+- On device, assumes the OS is already running (boot.py and main.py already executed), so tests run against the live system
+- Test infrastructure (graphical_test_helper.py) is automatically installed by `scripts/install.sh`
 
 **Available unit test modules**:
 - `test_shared_preferences.py`: Tests for `mpos.config.SharedPreferences` (configuration storage)
 - `test_intent.py`: Tests for `mpos.content.intent.Intent` (intent creation, extras, flags)
 - `test_package_manager.py`: Tests for `PackageManager` (version comparison, app discovery)
 - `test_start_app.py`: Tests for app launching (requires SDL display initialization)
+- `test_graphical_about_app.py`: Graphical test that verifies About app UI and captures screenshots
+
+**Graphical tests** (UI verification with screenshots):
+```bash
+# Run graphical tests on desktop
+./tests/unittest.sh tests/test_graphical_about_app.py
+
+# Run graphical tests on device
+./tests/unittest.sh tests/test_graphical_about_app.py ondevice
+
+# Convert screenshots from raw RGB565 to PNG
+cd tests/screenshots
+./convert_to_png.sh  # Converts all .raw files in the directory
+```
+
+Graphical tests use `tests/graphical_test_helper.py` which provides utilities like:
+- `wait_for_render()`: Wait for LVGL to process UI events
+- `capture_screenshot()`: Take screenshot as RGB565 raw data
+- `find_label_with_text()`: Find labels containing specific text
+- `verify_text_present()`: Verify expected text is on screen
+
+Screenshots are saved as `.raw` files (RGB565 format) and can be converted to PNG using `tests/screenshots/convert_to_png.sh`
 
 **Manual tests** (interactive, for hardware-specific features):
 - `manual_test_camera.py`: Camera and QR scanning

scripts/install.sh

@@ -76,6 +76,13 @@ $mpremote fs cp -r resources :/
 
 popd
 
+# Install test infrastructure (for running ondevice tests)
+echo "Installing test infrastructure..."
+$mpremote fs mkdir :/tests
+$mpremote fs mkdir :/tests/screenshots
+testdir=$(readlink -f "$mydir/../tests")
+$mpremote fs cp "$testdir/graphical_test_helper.py" :/tests/graphical_test_helper.py
+
 if [ -z "$appname" ]; then
 	echo "Not resetting so the installed app can be used immediately."
 	$mpremote reset

tests/graphical_test_helper.py

@@ -0,0 +1,201 @@
+"""
+Graphical testing helper module for MicroPythonOS.
+
+This module provides utilities for graphical/visual testing that work on both
+desktop (unix/macOS) and device (ESP32).
+
+Important: Tests using this module should be run with boot and main files
+already executed (so display, theme, and UI infrastructure are initialized).
+
+Usage:
+    from graphical_test_helper import wait_for_render, capture_screenshot
+
+    # Start your app
+    mpos.apps.start_app("com.example.myapp")
+
+    # Wait for UI to render
+    wait_for_render()
+
+    # Verify content
+    assert verify_text_present(lv.screen_active(), "Expected Text")
+
+    # Capture screenshot
+    capture_screenshot("tests/screenshots/mytest.raw")
+"""
+
+import lvgl as lv
+
+
+def wait_for_render(iterations=10):
+    """
+    Wait for LVGL to process UI events and render.
+
+    This processes the LVGL task handler multiple times to ensure
+    all UI updates, animations, and layout changes are complete.
+
+    Args:
+        iterations: Number of task handler iterations to run (default: 10)
+    """
+    import time
+    for _ in range(iterations):
+        lv.task_handler()
+        time.sleep(0.01)  # Small delay between iterations
+
+
+def capture_screenshot(filepath, width=320, height=240, color_format=lv.COLOR_FORMAT.RGB565):
+    """
+    Capture screenshot of current screen using LVGL snapshot.
+
+    The screenshot is saved as raw binary data in the specified color format.
+    To convert RGB565 to PNG, use:
+        ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 320x240 -i file.raw file.png
+
+    Args:
+        filepath: Path where to save the raw screenshot data
+        width: Screen width in pixels (default: 320)
+        height: Screen height in pixels (default: 240)
+        color_format: LVGL color format (default: RGB565 for memory efficiency)
+
+    Returns:
+        bytearray: The screenshot buffer
+
+    Raises:
+        Exception: If screenshot capture fails
+    """
+    # Calculate buffer size based on color format
+    if color_format == lv.COLOR_FORMAT.RGB565:
+        bytes_per_pixel = 2
+    elif color_format == lv.COLOR_FORMAT.RGB888:
+        bytes_per_pixel = 3
+    else:
+        bytes_per_pixel = 4  # ARGB8888
+
+    size = width * height * bytes_per_pixel
+    buffer = bytearray(size)
+    image_dsc = lv.image_dsc_t()
+
+    # Take snapshot of active screen
+    lv.snapshot_take_to_buf(lv.screen_active(), color_format, image_dsc, buffer, size)
+
+    # Save to file
+    with open(filepath, "wb") as f:
+        f.write(buffer)
+
+    return buffer
+
+
+def get_all_labels(obj, labels=None):
+    """
+    Recursively find all label widgets in the object hierarchy.
+
+    This traverses the entire widget tree starting from obj and
+    collects all LVGL label objects.
+
+    Args:
+        obj: LVGL object to search (typically lv.screen_active())
+        labels: Internal accumulator list (leave as None)
+
+    Returns:
+        list: List of all label objects found in the hierarchy
+    """
+    if labels is None:
+        labels = []
+
+    # Check if this object is a label
+    try:
+        if obj.get_class() == lv.label_class:
+            labels.append(obj)
+    except:
+        pass  # Not a label or no get_class method
+
+    # Recursively check children
+    try:
+        child_count = obj.get_child_count()
+        for i in range(child_count):
+            child = obj.get_child(i)
+            get_all_labels(child, labels)
+    except:
+        pass  # No children or error accessing them
+
+    return labels
+
+
+def find_label_with_text(obj, search_text):
+    """
+    Find a label widget containing specific text.
+
+    Searches the entire widget hierarchy for a label whose text
+    contains the search string (substring match).
+
+    Args:
+        obj: LVGL object to search (typically lv.screen_active())
+        search_text: Text to search for (can be substring)
+
+    Returns:
+        LVGL label object if found, None otherwise
+    """
+    labels = get_all_labels(obj)
+    for label in labels:
+        try:
+            text = label.get_text()
+            if search_text in text:
+                return label
+        except:
+            pass  # Error getting text from this label
+    return None
+
+
+def get_screen_text_content(obj):
+    """
+    Extract all text content from all labels on screen.
+
+    Useful for debugging or comprehensive text verification.
+
+    Args:
+        obj: LVGL object to search (typically lv.screen_active())
+
+    Returns:
+        list: List of all text strings found in labels
+    """
+    labels = get_all_labels(obj)
+    texts = []
+    for label in labels:
+        try:
+            text = label.get_text()
+            if text:
+                texts.append(text)
+        except:
+            pass  # Error getting text
+    return texts
+
+
+def verify_text_present(obj, expected_text):
+    """
+    Verify that expected text is present somewhere on screen.
+
+    This is the primary verification method for graphical tests.
+    It searches all labels for the expected text.
+
+    Args:
+        obj: LVGL object to search (typically lv.screen_active())
+        expected_text: Text that should be present (can be substring)
+
+    Returns:
+        bool: True if text found, False otherwise
+    """
+    return find_label_with_text(obj, expected_text) is not None
+
+
+def print_screen_labels(obj):
+    """
+    Debug helper: Print all label text found on screen.
+
+    Useful for debugging tests to see what text is actually present.
+
+    Args:
+        obj: LVGL object to search (typically lv.screen_active())
+    """
+    texts = get_screen_text_content(obj)
+    print(f"Found {len(texts)} labels on screen:")
+    for i, text in enumerate(texts):
+        print(f"  {i}: {text}")

tests/screenshots/.gitignore

@@ -0,0 +1,9 @@
+# Ignore all screenshot files
+*.raw
+
+# Ignore converted PNG files (can be regenerated from .raw)
+*.png
+
+# Allow this .gitignore and README.md
+!.gitignore
+!README.md

tests/screenshots/README.md

@@ -0,0 +1,61 @@
+# Test Screenshots
+
+This directory contains screenshots captured during graphical tests.
+
+## File Format
+
+Screenshots are saved as raw binary data in RGB565 format:
+- 2 bytes per pixel
+- For 320x240 screen: 153,600 bytes per file
+- Filename format: `{test_name}_{hardware_id}.raw`
+
+## Converting to PNG
+
+### Quick Method (Recommended)
+
+Use the provided convenience script to convert all screenshots:
+
+```bash
+cd tests/screenshots
+./convert_to_png.sh
+```
+
+For custom dimensions:
+```bash
+./convert_to_png.sh 296 240
+```
+
+### Manual Conversion
+
+To view individual screenshots, convert them to PNG using ffmpeg:
+
+```bash
+# For 320x240 screenshots (default)
+ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 320x240 -i screenshot.raw screenshot.png
+
+# For other sizes (e.g., 296x240 for some hardware)
+ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 296x240 -i screenshot.raw screenshot.png
+```
+
+## Visual Regression Testing
+
+Screenshots can be used for visual regression testing by:
+1. Capturing a "golden" reference screenshot
+2. Comparing new screenshots against the reference
+3. Detecting visual changes
+
+For pixel-by-pixel comparison, you can use ImageMagick:
+
+```bash
+# Convert both to PNG first
+ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 320x240 -i reference.raw reference.png
+ffmpeg -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s 320x240 -i current.raw current.png
+
+# Compare
+compare -metric AE reference.png current.png diff.png
+```
+
+## .gitignore
+
+Screenshot files (.raw and .png) are ignored by git to avoid bloating the repository.
+Reference/golden screenshots should be stored separately or documented clearly.

tests/screenshots/convert_to_png.sh

@@ -0,0 +1,81 @@
+#!/bin/bash
+
+# Convert raw RGB565 screenshots to PNG format
+# This script converts all .raw files in the current directory to PNG using ffmpeg
+
+# Default dimensions (can be overridden with arguments)
+WIDTH=320
+HEIGHT=240
+
+# Parse command line arguments
+if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then
+    echo "Usage: $0 [width] [height]"
+    echo
+    echo "Convert all .raw screenshot files to PNG format."
+    echo
+    echo "Arguments:"
+    echo "  width   Screen width in pixels (default: 320)"
+    echo "  height  Screen height in pixels (default: 240)"
+    echo
+    echo "Examples:"
+    echo "  $0              # Convert with default 320x240"
+    echo "  $0 296 240      # Convert with custom dimensions"
+    echo
+    exit 0
+fi
+
+if [ -n "$1" ]; then
+    WIDTH="$1"
+fi
+
+if [ -n "$2" ]; then
+    HEIGHT="$2"
+fi
+
+# Check if ffmpeg is available
+if ! command -v ffmpeg &> /dev/null; then
+    echo "ERROR: ffmpeg is not installed or not in PATH"
+    echo "Please install ffmpeg to convert screenshots"
+    exit 1
+fi
+
+# Count .raw files
+raw_count=$(find . -maxdepth 1 -name "*.raw" | wc -l)
+
+if [ $raw_count -eq 0 ]; then
+    echo "No .raw files found in current directory"
+    exit 0
+fi
+
+echo "Converting $raw_count screenshot(s) from RGB565 to PNG..."
+echo "Dimensions: ${WIDTH}x${HEIGHT}"
+echo
+
+converted=0
+failed=0
+
+# Convert each .raw file to .png
+for raw_file in *.raw; do
+    [ -e "$raw_file" ] || continue  # Skip if no .raw files exist
+
+    png_file="${raw_file%.raw}.png"
+
+    echo -n "Converting $raw_file -> $png_file ... "
+
+    if ffmpeg -y -v quiet -vcodec rawvideo -f rawvideo -pix_fmt rgb565 -s ${WIDTH}x${HEIGHT} -i "$raw_file" "$png_file" 2>/dev/null; then
+        echo "✓"
+        converted=$((converted + 1))
+    else
+        echo "✗ FAILED"
+        failed=$((failed + 1))
+    fi
+done
+
+echo
+echo "Conversion complete: $converted succeeded, $failed failed"
+
+if [ $converted -gt 0 ]; then
+    echo
+    echo "PNG files created:"
+    ls -lh *.png 2>/dev/null | awk '{print "  " $9 " (" $5 ")"}'
+fi

tests/test_graphical_about_app.py

@@ -0,0 +1,177 @@
+"""
+Graphical test for the About app.
+
+This test verifies that the About app displays correct information,
+specifically that the Hardware ID shown matches the actual hardware ID.
+
+This is a proof of concept for graphical testing that:
+1. Starts an app programmatically
+2. Verifies UI content via direct widget inspection
+3. Captures screenshots for visual regression testing
+4. Works on both desktop and device
+
+Usage:
+    Desktop: ./tests/unittest.sh tests/test_graphical_about_app.py
+    Device:  ./tests/unittest.sh tests/test_graphical_about_app.py ondevice
+"""
+
+import unittest
+import lvgl as lv
+import mpos.apps
+import mpos.info
+import mpos.ui
+import os
+from graphical_test_helper import (
+    wait_for_render,
+    capture_screenshot,
+    find_label_with_text,
+    verify_text_present,
+    print_screen_labels
+)
+
+
+class TestGraphicalAboutApp(unittest.TestCase):
+    """Test suite for About app graphical verification."""
+
+    def setUp(self):
+        """Set up test fixtures before each test method."""
+        # Get absolute path to screenshots directory
+        # When running tests, we're in internal_filesystem/, so go up one level
+        import sys
+        if sys.platform == "esp32":
+            self.screenshot_dir = "tests/screenshots"
+        else:
+            # On desktop, tests directory is in parent
+            self.screenshot_dir = "../tests/screenshots"
+
+        # Ensure screenshots directory exists
+        try:
+            os.mkdir(self.screenshot_dir)
+        except OSError:
+            pass  # Directory already exists
+
+        # Store hardware ID for verification
+        self.hardware_id = mpos.info.get_hardware_id()
+        print(f"Testing with hardware ID: {self.hardware_id}")
+
+    def tearDown(self):
+        """Clean up after each test method."""
+        # Navigate back to launcher (closes the About app)
+        try:
+            mpos.ui.back_screen()
+            wait_for_render(5)  # Allow navigation to complete
+        except:
+            pass  # Already on launcher or error
+
+    def test_about_app_shows_correct_hardware_id(self):
+        """
+        Test that About app displays the correct Hardware ID.
+
+        Verification approach:
+        1. Start the About app
+        2. Wait for UI to render
+        3. Find the "Hardware ID:" label
+        4. Verify it contains the actual hardware ID
+        5. Capture screenshot for visual verification
+        """
+        print("\n=== Starting About app test ===")
+
+        # Start the About app
+        result = mpos.apps.start_app("com.micropythonos.about")
+        self.assertTrue(result, "Failed to start About app")
+
+        # Wait for UI to fully render
+        wait_for_render(iterations=15)
+
+        # Get current screen
+        screen = lv.screen_active()
+
+        # Debug: Print all labels found (helpful for development)
+        print("\nLabels found on screen:")
+        print_screen_labels(screen)
+
+        # Verify that Hardware ID text is present
+        hardware_id_label = find_label_with_text(screen, "Hardware ID:")
+        self.assertIsNotNone(
+            hardware_id_label,
+            "Could not find 'Hardware ID:' label on screen"
+        )
+
+        # Get the full text from the Hardware ID label
+        hardware_id_text = hardware_id_label.get_text()
+        print(f"\nHardware ID label text: {hardware_id_text}")
+
+        # Verify the hardware ID matches
+        expected_text = f"Hardware ID: {self.hardware_id}"
+        self.assertEqual(
+            hardware_id_text,
+            expected_text,
+            f"Hardware ID mismatch. Expected '{expected_text}', got '{hardware_id_text}'"
+        )
+
+        # Also verify using the helper function
+        self.assertTrue(
+            verify_text_present(screen, self.hardware_id),
+            f"Hardware ID '{self.hardware_id}' not found on screen"
+        )
+
+        # Capture screenshot for visual regression testing
+        screenshot_path = f"{self.screenshot_dir}/about_app_{self.hardware_id}.raw"
+        print(f"\nCapturing screenshot to: {screenshot_path}")
+
+        try:
+            buffer = capture_screenshot(screenshot_path, width=320, height=240)
+            print(f"Screenshot captured: {len(buffer)} bytes")
+
+            # Verify screenshot file was created
+            stat = os.stat(screenshot_path)
+            self.assertTrue(
+                stat[6] > 0,  # stat[6] is file size
+                "Screenshot file is empty"
+            )
+            print(f"Screenshot file size: {stat[6]} bytes")
+
+        except Exception as e:
+            self.fail(f"Failed to capture screenshot: {e}")
+
+        print("\n=== About app test completed successfully ===")
+
+    def test_about_app_shows_os_version(self):
+        """
+        Test that About app displays the OS version.
+
+        This is a simpler test that just verifies version info is present.
+        """
+        print("\n=== Starting About app OS version test ===")
+
+        # Start the About app
+        result = mpos.apps.start_app("com.micropythonos.about")
+        self.assertTrue(result, "Failed to start About app")
+
+        # Wait for UI to render
+        wait_for_render(iterations=15)
+
+        # Get current screen
+        screen = lv.screen_active()
+
+        # Verify that MicroPythonOS version text is present
+        self.assertTrue(
+            verify_text_present(screen, "MicroPythonOS version:"),
+            "Could not find 'MicroPythonOS version:' on screen"
+        )
+
+        # Verify the actual version string is present
+        os_version = mpos.info.CURRENT_OS_VERSION
+        self.assertTrue(
+            verify_text_present(screen, os_version),
+            f"OS version '{os_version}' not found on screen"
+        )
+
+        print(f"Found OS version: {os_version}")
+        print("=== OS version test completed successfully ===")
+
+
+if __name__ == "__main__":
+    # Note: This file is executed by unittest.sh which handles unittest.main()
+    # But we include it here for completeness
+    unittest.main()

tests/unittest.sh

@@ -31,16 +31,42 @@ one_test() {
 	fi
 	pushd "$fs"
 	echo "Testing $file"
+
+	# Detect if this is a graphical test (filename contains "graphical")
+	if echo "$file" | grep -q "graphical"; then
+		echo "Detected graphical test - including boot and main files"
+		is_graphical=1
+		# Get absolute path to tests directory for imports
+		tests_abs_path=$(readlink -f "$testdir")
+	else
+		is_graphical=0
+	fi
+
 	if [ -z "$ondevice" ]; then
-		"$binary" -X heapsize=8M -c "import sys ; sys.path.append('lib')
+		# Desktop execution
+		if [ $is_graphical -eq 1 ]; then
+			# Graphical test: include boot_unix.py and main.py
+			"$binary" -X heapsize=8M -c "$(cat boot_unix.py main.py)
+import sys ; sys.path.append('lib') ; sys.path.append(\"$tests_abs_path\")
 $(cat $file)
 result = unittest.main() ; sys.exit(0 if result.wasSuccessful() else 1) "
+		else
+			# Regular test: no boot files
+			"$binary" -X heapsize=8M -c "import sys ; sys.path.append('lib')
+$(cat $file)
+result = unittest.main() ; sys.exit(0 if result.wasSuccessful() else 1) "
+		fi
 		result=$?
 	else
+		# Device execution
+		# NOTE: On device, the OS is already running with boot.py and main.py executed,
+		# so we don't need to (and shouldn't) re-run them. The system is already initialized.
 		cleanname=$(echo "$file" | sed "s#/#_#g")
 		testlog=/tmp/"$cleanname".log
 		echo "$test logging to $testlog"
-		mpremote.py exec "import sys ; sys.path.append('lib')
+		if [ $is_graphical -eq 1 ]; then
+			# Graphical test: system already initialized, just add test paths
+			mpremote.py exec "import sys ; sys.path.append('lib') ; sys.path.append('tests')
 $(cat $file)
 result = unittest.main()
 if result.wasSuccessful():
@@ -48,6 +74,17 @@ if result.wasSuccessful():
 else:
     print('TEST WAS A FAILURE')
 " | tee "$testlog"
+		else
+			# Regular test: no boot files
+			mpremote.py exec "import sys ; sys.path.append('lib')
+$(cat $file)
+result = unittest.main()
+if result.wasSuccessful():
+    print('TEST WAS A SUCCESS')
+else:
+    print('TEST WAS A FAILURE')
+" | tee "$testlog"
+		fi
 		grep "TEST WAS A SUCCESS" "$testlog"
 		result=$?
 	fi