Files
Filip Gołaś a1a542fd5f Osfv stable release improvements (#1254)
* osfv_stability_{run,reports}.py: Add

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_run: add release rules file

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports: add run count

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports: Fix: account for parallelism in total run time

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_run.py: accept env vars

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports.py: save logs to dirs depending on date and revision

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_run.py: Save results in paths per run date and rev

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports.py: Parse date, revision, branch and support json output

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_*.py: Allow configurable LOGS_DIR using env vars

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports.py: Handle broken robot xmls

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports.py: Unpack merged results

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

osfv_stability_reports.py: Add suites to skip config

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Deploy Uefi Shell: Ensure it doesnt break the bootorder

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* os-specific-tasks/201.yml: Remove uefi shell installation

it breaks bootorder and tests deploy it anyway

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To OS: Rename to Boot And Login To OS

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Windows: Rename to Boot And Login To Windows

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Windows: Add keyword

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Logint To Booted OS: Add keyword

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Serial User Login Linux: Remove dead code

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Linux: Remove retrials of Login To Linux Via SSH

Login To Linux Via SSH already retries all its code,
so this is redundant and multiplies the wait time

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* fix login ssh timeouts

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Windows Via SSH: Reduce login timeout from 1h@20 tries to 5m@5 tries

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Implement login to OS recovery

When the boot order unexpectedly changes and testing via SSH,
the whole regression will start failing.

With this fallback, after we fail to log in to an OS, there
will at least be the hope of recovering the regression
eventually.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Ensure Custom Bootentry: fix indexing empty list

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* BPS010: Deploy UEFI Shell: Add

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* linux-packages-playbook.yaml: Add systemwide update task

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Windows Via SSH: Fail if login fails

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* TMP TEST: reduce trials for linux login

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* basic-platform-setup.robot: Normalize Boot&Login keyword variants usage in suite

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Set Nextboot Bootentry: Move implementation to lib/custom_boot_entries

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Load OS Credentials: Create keyword for flaky repeated import

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Recover Broken Bootorder: resume login after recover

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* options-lib_dcu.robot: Boot System... Do not assume default_boot is the target

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/flash.robot: Fix missing 'x' in Wait Unil KW Succeeds

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To* kwds: Add configurable retries amount for ssh variants

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Flash rc0 before tests, update instructions

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Revert "TMP TEST: reduce trials for linux login"

This reverts commit d41693da2a3a600beb7a4a26d95f6e7f90a2430a.

* lib/dcu.robot: Remove redundant sleep after flashing

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/network.robot: Send File To DUT: check hashes, rm before send

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* platform-configs: Update INTEL_CBNT_BOOTGUARD_FUSING_SUPPORT state

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* options-lib_dcu.robot: Remove redundant sleep in Set UEFI Option

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Load OS Credentials: Remove redundant OS login

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/flash.robot: Flash ME in BTG supported devices

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/flash.robot: Increase timeouts a bit to compensate for large RAM training

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Fix Login To Linux Via SSH not doing retries

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Rework suite to work without additional manual setup steps

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* novacustom-nuc_box.robot: Remove capsule update support: No capsules published

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* flash.robot: Flash FD using internal programmer to avoid FD mismatch bricks

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* regression-scope/configs/devices: Add capsule update env vars

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Remove manual step from Flash Firmware If Not QEMU

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keyword & options-lib_dcu: Remove redundant Login To Windows keyword variant

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Remove redundant serial console test flow

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_suite_list*: Add Basic Platform Setup to release tests

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Restore custom logo checking

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* requirements.txt: Bump osfv-scripts to fix time import

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/dcu.robot: Prevent temp file collisions with uuids

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* msi-pro-690-a-wifi-ddr4_00039.json: change fw version to support custom logo

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/flash.robot: Increase response timeout after flash to 300s

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Fix Login To Linux With Root Priviledges doing nothing on serial

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Login To Booted OS: Only attempt recover when via pure SSH

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Restore FUM dialog handling when serial console supported

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Adjust for execution on both serial and ssh

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule_update_tests.sh: Remove only previous run files of the same name

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/network.robot: Rm target path before moving

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Revert "lib/network.robot: Rm target path before moving"

This reverts commit 3bca42177f528f17eba13095dff0189a932fce8d.

* lib/network.robot: Add sync on the end to prevent interrupting the file transfer

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Make FUM screen optional, it depends on FW version

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Fix mismatch in waiting for tianocore string

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* ci//pcengines-apu3_00252.json: Fix FW file apu variant

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* platform-configs/protectli: Reduce repeated DMIDECODE values

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* protectli-vp6650_00002.json: Update sonoff ip

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keys-and-keywords/ubuntu-keywords.robot: Fix applying log level immediately

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* ci//pcengines-apu3_00252.json: Update device IP address

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* menus.robot: Add warning if OS not found and will try to boot the default disk

Otherwise leads to cryptic fails failing because some disk name was not
found, while the real culprit is the OS not being installed.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* linux-packages-playbook.yaml: accept gsettings modifications to fail

will fail if Gnome is not installed (server OSes)

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* BPS006: Add workaround for sudo-rs which is incompatible with ansible

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_devices.txt: Fix protectli vp6650 filename

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* novacustom-nu_box-155H_00254.json: Update device IP

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* msi-z690-common.robot: Change keyboard model to fit hardware matrix

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* uefi-shell.robot: Use library keywords to perform test

Otherwise it was failing on MSI z690.
It's always better to use common and working
keywords instead of reinventing the wheel.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* osfv-test-data: Bump version

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* tpm-support.robot: Add missing skips for serial-only tests

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* verified-boot.robot: Run DCU locally to keep DUT from downloading containers

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* platform-configs/pcengines: Update dmidecode values

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* platform-configs/novacustom: Update dmidecode values

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/dcu.robot: Put temp tiles in /tmp to prevent littering if cleanup fails

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* BPS006: Put temp tiles in /tmp to prevent littering if cleanup fails

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/flash.robot: Read Firmware include ME and IFD

We might assume the read firmware contains some IFD
or ME later, not reading them would result in a brick
and doesn't increase the read time that much.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* SSH Boot System: Always reinit connection to fix 'No open connections' errors

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* cpu-performance.robot: Limit test runs to 3

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* manual_cpu_performance.robot: Limit runs to 3

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* novacustom-v540tnd_00266.json: Update device ip

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* novacustom-nuc_box-155H_00254.json: Update device IP

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Booted OS: Fix infinite recovery loop

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_devices.txt: Remove nucbox

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Fix ssh_retries being passed as timeout

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* SSH Login keywords: Limit retries to 5m instead of fixed try count

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_suite_list.txt: Remove duplicated __init__s

robot crashes otherwise

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/prepare-platform.resource: Unify redundant Prepare Platform definitions

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* util/prepare-platform.robot: Add util test for running Prepare Platform

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_suite_list.txt: Use util/prepare-platform instead of module inits

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* dts-lib.robot: Add semiauto test skip for flashing kwd

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* develop_pr_auto_regression.py: Only compare revs if files list not overridden

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* osfv_stability_run.py: Fix logs_base undefined on custom logs dir

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* scripts/ci/regression-scope: Preserve suites order

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* develop_pr_auto_regression.py: Fix orphan robot processes

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_rules.json: Add automated tag filter

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* osfv_stability_reports.py: Add per-suite cache

For suites which take a long long time to process,
to not lose the time spent processing it if the
processing is interrupted

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* osfv_stability_reports.py: Differentiate run dates in caches and reports

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* release_tests_devices.tst: Revert using temporary desk setup devices

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* scripts/regression.sh: Fix scope change

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* cpu-performance: Fix FORCE_TIMES_TO_RUN envvar set command

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Booted OS: Fix continuing on fail to login

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* test_cases.json: fix spelling and such (#1244)

* test_cases.json: fix spelling and such

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

test_cases.json: fix

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* test_cases.json: fixname

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* test_cases.json:remaining tests

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* test_cases.json: add WLE

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* check_tests_consistency.py: script lists diffs in test_cases.json and modules

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* dasharo-compatibility/*:fix

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* test_cases.json: fix

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

---------

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* test_cases.json: Add missing tests and fix changed_to's (#1246)

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* test_cases.json:fix net spd test (#1247)

Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>

* docs/novacustom.md: document v0.9.0 downgrade bug

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* docs/novacustom.md: add link to ec downgrade procedure

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

---------

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>
Signed-off-by: Wiktor Mowinski <wiktor.mowinski@3mdeb.com>
Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>
Co-authored-by: Wiktor Mówiński <wiktor.mowinski@3mdeb.com>
Co-authored-by: Michał Kopeć <michal.kopec@3mdeb.com>
2026-03-10 08:53:21 +01:00

131 lines
5.2 KiB
Plaintext

*** Settings ***
Documentation Common header for OSFV Power management keywords
Library Collections
Library OperatingSystem
Library Process
Library String
Library RequestsLibrary
Library SSHLibrary
Resource ../../variables.robot
Resource ../../keywords.robot
Resource ../../keys.robot
*** Keywords ***
Check Power Supply
${is_laptop}= Check The Platform Is A Laptop
VAR ${LAPTOP_PLATFORM}= ${is_laptop} scope=SUITE
IF ${LAPTOP_PLATFORM}
IF ${TESTS_IN_UBUNTU_SUPPORT}
${bat0_present} ${ac_online} ${usb_pd_online}= Check Power Supply On Linux
ELSE IF ${TESTS_IN_WINDOWS_SUPPORT}
${bat0_present} ${ac_online} ${usb_pd_online}= Check Power Supply On Windows
ELSE IF ${HEADS_PAYLOAD_SUPPORT}
Log Check Power Supply on Heads not implemented yet ERROR
ELSE
Fail Fail: Check Power Supply is not implemented enough
END
VAR ${BATTERY_PRESENT}= ${bat0_present} scope=SUITE
VAR ${AC_CONNECTED}= ${ac_online} scope=SUITE
VAR ${USB_PD_CONNECTED}= ${usb_pd_online} scope=SUITE
END
Check The Platform Is A Laptop
${laptop_producer}= Run Keyword And Return Status Should Contain Any ${PLATFORM} novacustom tuxedo
${nuc}= Run Keyword And Return Status Should Contain Any ${PLATFORM} nuc_box
${laptop_platform}= Evaluate ${laptop_producer} and not ${nuc}
RETURN ${laptop_platform}
Check Power Supply On Linux
Power On
Boot System Or From Connected Disk ${DEFAULT_BOOT_OS_ID}
Login To Linux
${bat0_present_raw}= Execute Command In Terminal cat /sys/class/power_supply/BAT0/present
${bat0_present}= Run Keyword And Return Status Should Be Equal ${bat0_present_raw} 1
${ac_online_raw}= Execute Command In Terminal cat /sys/class/power_supply/AC/online
Should Not Contain ${ac_online_raw} No such file or directory
${ac_online}= Run Keyword And Return Status Should Be Equal ${ac_online_raw} 1
# FIXME: USB-PD detection is not yet possible.
${usb_pd_online_raw}= Execute Command In Terminal cat /sys/class/power_supply/USB-PD/online
Log 'cat /sys/class/power_supply/USB-PD/online' not implemented yet, if implemented, remove # WARN
# Should Not Contain ${usb_pd_online_raw} No such file or directory
${usb_pd_online}= Run Keyword And Return Status Should Be Equal ${usb_pd_online_raw} 1
RETURN ${bat0_present} ${ac_online} ${usb_pd_online}
Check Power Supply On Windows
Power On
Boot And Login To Windows
${raw_output}= Execute Command In Terminal (Get-WmiObject Win32_Battery).BatteryStatus
${bat0_present}= Run Keyword And Return Status Should Not Be Empty ${raw_output}
# ${ac_online_raw}= Execute Command In Terminal (Get-WmiObject Win32_Battery).BatteryStatus
${ac_online_empty}= Run Keyword And Return Status Should Be Empty ${raw_output}
${ac_online_equal_2}= Run Keyword And Return Status Should Be Equal ${raw_output} 2
# IF ${ac_online_raw_empty} or ${ac_online_raw_equal_2}
# Set Local Variable ${AC_ONLINE}= ${TRUE}
# END
IF ${ac_online_empty}
VAR ${ac_online}= ${TRUE}
ELSE IF ${ac_online_equal_2}
VAR ${ac_online}= ${TRUE}
ELSE
VAR ${ac_online}= ${None}
END
# FIXME: USB-PD detection is not yet possible.
Log Check power supply USB-PD not implemented yet WARN
${usb_pd_online}= Run Keyword And Return Status
... Should Be Equal
... ${raw_output}
... insert the correct USB-PD detection method here
RETURN ${bat0_present} ${ac_online} ${usb_pd_online}
Check Battery Level On Linux
[Documentation] Returns a battery level as percentage.
${power_level}= Execute Command In Terminal
... cat /sys/class/power_supply/BAT0/capacity
Should Not Contain ${power_level} No such file or directory
RETURN ${power_level}
Check Battery Level On Windows
[Documentation] Returns a battery level as percentage.
${power_level}= Execute Command In Terminal
... (Get-WmiObject -Class Win32_Battery).EstimatedChargeRemaining
Should Not Be Empty ${power_level}
RETURN ${power_level}
Power Cycle Into Ubuntu
Power On
Boot System Or From Connected Disk 201
Login To Linux
Power Cycle Into Windows
Power On
Boot And Login To Windows
Power Cycle Into Firmware Setup
Power On
Enter Setup Menu Tianocore
Execute Cold Boot
[Documentation] Performs cold boot, either with RTE relay or Sonoff
Power On
Set UEFI Option PowerStateAfterPowerAcLoss Powered On
Sleep 2
IF '${POWER_CTRL}' == 'RteCtrl'
Rte Psu Off
ELSE IF '${POWER_CTRL}' == 'sonoff'
Sonoff Off
END
Sleep 12
IF '${POWER_CTRL}' == 'RteCtrl'
Rte Psu On
ELSE IF '${POWER_CTRL}' == 'sonoff'
Sonoff On
END