You've already forked open-source-firmware-validation
mirror of
https://github.com/Dasharo/open-source-firmware-validation.git
synced 2026-03-06 14:51:55 -08:00
4f10afbdbc
* dasharo-compatibility/network-speed.robot: Move keywords to lib The keywords are generic for network performance testing. They can be reused by other tests, e.g. incoming ODROID netcard test. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/odroid-netcard.robot: Add netcard test Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/memory-ibecc.robot: Add IBECC tests Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs: Add ODROID H4 Ultra Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-performance/fast-boot.robot: Make the test work with telnet The test assumed that the device is already booted to an OS, which probably is true for laptops. Add necessary Power On's and initial booting to OS on the beginning of the test to handle a case where the serial connection can be used to navigate and select OSes to boot. Also do not log in to OS again after "Measure FW Boot Time On Linux", because this keyword keeps the connection to the OS. TEST=Run the test and PASS it on ODROID H4 Ultra Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/sleep-lib.robot: Make sleep tests work with telnet Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * test_cases.json: Add new test cases Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Do not verify FW version after flashing Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/custom-network-boot-entries.robot: Ensure netboot is enabled Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Add CPU temperature threshold Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Update release date Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * variables.robot: Fix the configs for ODROID platforms Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/miniPCIe-slot-verification.robot: Fix skip Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/nvme-support.robot: Use configs to get disk name Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs: Drop unused CLEVO_DISK Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Synchronize configs Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Enable Windows tests Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * os-config/ansible/os-specific-tasks/201.yaml: Add docker installation Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * scripts/lib/robot.sh: Fix typo in directory name Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Disable fan speed measurements Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/usb-hid-msc-lib.robot: Fix condition for mounting USB Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/device-detection.robot: Close connections at the end of teardown Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/device-detection.robot: Add proper checks for suite skip Judging by the content of test cases, the suite depends on the SSH connection method and the DCU backed for option manipulation. It is not suitable for runnign in other configuration, like telnet connection method and option management via firmware setup. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Update release version Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/efibootmgr.robot: Make test suitable for telnet platforms Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * keywords.robot: Wait for the power to stabilize before pressing power button Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * util/basic-platform-setup.robot: Mark suite as automated Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * test_cases.json: Sort new tests by ID Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/audio-subsystem.robot: Use generic keyword for command Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/cpu-fan-speed-measure.robot: Skip suite earlier to save time Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/network.robot: Make intervals larger for Get Hostname Ip Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Disable hybrid core support Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/network.robot: Add variable for Get Ip interval Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * scripts/regression-rerun-failed.sh: Add script to rerun failed tests Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/efibootmgr.robot: Ensure network boot is disabled initially Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Fix CPU temperature threshold Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/power-after-fail-lib.robot: Fix import failure Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/reset-to-defaults.robot: Skip fan profile test if not supported Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/platform-suspend-and-resume.robot: Check for skip in suite setup Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * compatibility/reset-to-defaults.robot: Skip sleep type test if not supported Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * Deploy UEFI Shell where necessary Basic platform setup deploye UEFI SHell, however running a whole suite like comaptibility, e.g. during regression, causes the FW to eb flashed again and the UEFI Shell is lost right after basic platform setup finishes its job. Ensure the UEFI Shell is deployed wherever it is used. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/esp-scanning-lib.robot: Sync filesystem after cleaning ESP Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/DTS: Skip flashing if FW variable not set Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/dasharo-tools-suite.robot: Fix internal flashing "Flash Via Internal Programmer" sends a file to DUT from host. However, the test downloads th binary with wget prior to the keyword execution. Skip the wrapper and call "Flash Via Internal Programmer With Args" directly. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-security/tpm: Fix TPM tests Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-security/verified-boot.robot: Sync filesystem before power cycling Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/audio-subsystem.robot: Fix headphones Windows tests Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/audio-subsystem.robot: Make audo devices platform-specific Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/bios/menus.robot: Fix variable type Fixes the error of variable value not a list or list-like. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-security/verified-boot.robot: Drop VBO011.001 duplication Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/sleep-lib.robot: Fix suspend test on Telnet Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Remove obsolete override Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * scripts/regression.sh: Run stability suite too Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4-ultra.robot: Sync DTS E2E variables Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4: Sync DMI info Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-stability/tpm-detect.robot: Drop check for NVME in TPM detection Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/power-after-fail.robot: Fix powering on based on var state When power after fail test are executed, the state of power after fail variable is changed. The Power On keyword does not take it int oaccount and uses only the default state of the variable. This results in obsolete power button presses which often lead to powering off the platform right after PSU is powered on. Because of that the tests fail due to power LED not being lit up. Define another variable that is updated runtime and represents current state of the power after faile variable. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * keywords: Fix usage in teardown Fixes the problem: 'Set Tags' cannot be used in suite teardown. Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility: Fix list comparison in Ethernet tests Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs/odroid-h4-ultra.robot: Update Ethernet info Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * util/basic-platform-setup.robot: Add minimal-regression tag Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/secure-boot-lib.robot: Fix launching UEFI Shell Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * lib/linux.robot: Add MKTME to dmesg error allowlist Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * variables.robot: Add ODROID H4 Ultra Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility: Move netcard test to ETHPERF Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/network-speed.robot: merge netcard test with ETHPERF001 Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * dasharo-compatibility/device-detection.robot: Fix duplicated close connection Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> * platform-configs: Drop unused ODROID_NETCARD_SUPPORT Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com> --------- Signed-off-by: Michał Żygowski <michal.zygowski@3mdeb.com>
78 lines
2.1 KiB
Bash
Executable File
78 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
# SPDX-FileCopyrightText: 2025 3mdeb <contact@3mdeb.com>
|
|
#
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
source "${SCRIPT_DIR}/lib/robot.sh"
|
|
|
|
# FW_FILE and DEVICE_IP are required for full regression
|
|
check_env_variable "FW_FILE"
|
|
check_env_variable "DEVICE_IP"
|
|
|
|
if [ ! -f "$FW_FILE" ]; then
|
|
echo "Error: Environment variable FW_FILE doesn't point to a file."
|
|
exit 1
|
|
fi
|
|
|
|
_REGRESSION_RUN="True"
|
|
export _REGRESSION_RUN
|
|
|
|
check_test_station_variables
|
|
|
|
if [ $# -lt 1 ]; then
|
|
echo "Error: Argument with directory with logs from last regression missing"
|
|
exit 1
|
|
fi
|
|
|
|
REGRESSION_LOG_DIR=$1
|
|
shift
|
|
|
|
if [ ! -d $REGRESSION_LOG_DIR ]; then
|
|
echo "Error: Directory with logs from last regression does not exist"
|
|
exit 1
|
|
fi
|
|
|
|
RUN_DATE="${RUN_DATE:-$(date '+%Y_%m_%d_%H_%M_%S')}"
|
|
export RUN_DATE
|
|
|
|
if [ -n "${DIR_PREFIX}" ]; then
|
|
dir_prefix="${DIR_PREFIX}_"
|
|
else
|
|
dir_prefix=""
|
|
fi
|
|
|
|
if [[ -z $LOGS_DIR ]]; then
|
|
_logs_dir="logs/${CONFIG}/${dir_prefix}regression_${RUN_DATE}"
|
|
else
|
|
_logs_dir="$LOGS_DIR/${CONFIG}/${dir_prefix}regression_${RUN_DATE}"
|
|
fi
|
|
|
|
rerun_suite () {
|
|
local _suite=$1
|
|
|
|
shift
|
|
|
|
if [ ! -f $REGRESSION_LOG_DIR/${_suite}_out.xml ]; then
|
|
echo "Error: Output XML from last $_suite regression does not exist"
|
|
else
|
|
execute_robot "$_suite" -- \
|
|
--rerunfailed $REGRESSION_LOG_DIR/${_suite}_out.xml "${@}"
|
|
rebot --output $_logs_dir/${_suite}_out_merged.xml \
|
|
--log $_logs_dir/${_suite}_log_merged.html \
|
|
--report $_logs_dir/${_suite}_report_merged.html \
|
|
--merge $REGRESSION_LOG_DIR/${_suite}_out.xml $_logs_dir/${_suite}_out.xml
|
|
mv $_logs_dir/${_suite}_out_merged.xml $_logs_dir/${_suite}_out.xml
|
|
mv $_logs_dir/${_suite}_log_merged.html $_logs_dir/${_suite}_log.html
|
|
mv $_logs_dir/${_suite}_report_merged.html $_logs_dir/${_suite}_report.html
|
|
fi
|
|
}
|
|
|
|
rerun_suite dasharo-compatibility "${@}"
|
|
rerun_suite dasharo-security "${@}"
|
|
rerun_suite dasharo-stability "${@}"
|
|
if [ -z "$NO_PERFORMANCE" ]; then
|
|
rerun_suite dasharo-performance "${@}"
|
|
fi
|