Files
Filip Gołaś 120f200c6d V560tnx v1.0.0 rc10 (#1202)
* .gitignore: Add _envvars, _roms directories

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Allow for the DUT ssh key to change when trying to connect via SSH in Login To Linux via SSH

* lib/tpm.robot: Check all PCRs before failing in Validate PCRs Against Event Log

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* miniPCIe-slot-verification: Skip LTE_CARD tests if card is not defined

* Stress Test Windows: use Execute Command In Terminal

Otherwise there is no timeout for the keyword and the
tests could hang forever

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* CBMEM001: Add semiauto implementation

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Set Nextboot Bootentry: compare strings as variables to fix utf decode errors

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* novacustom-mtl.robot: Add USB model t oconfig

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* novacustom-v560tne: v1.0.0-rc10 platform configs changes

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

novacustom-v560tne: Add windows support

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v560tne: temp RTE serial tests

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v560tne: dmidecode update version

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v560tne: Restore fedora support flag

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v560tne: Add USB drives model names for detection tests

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v560tne: Add DOCKING_STATION_USB_C_CHARGING_SUPPORT

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v560tne: add qubes support

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

v540tnd: Restore ubuntu default boot os

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

REMOVE platform-configs: restore SSH

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* acpi-driver.robot: Fix typo in deb package name

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* minipcie-slot-verification: Replace hardcoded wifi name with the configs

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Switch To Root User: Add workaround for passwordless root

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* os-specific-tasks/203.yml: Add qubesos ansible config

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* fwupd-support.robot: automate qubes

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* cpu-performance.robot: Add missing phoronix batch mode setup to automated suite

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Login To Windows Via SSH: Allow the key to change across retries

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Get Index Of Matching Option In Menu: do not use fuzzy match

It works a bit differently - the * is not a special character in
fuzzy search

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Execute Shutdown Command: Add linux systemd support

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

Execute Shutdown Command: Add some workaround for no power_led

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Measure Warmboot Time: add workaround for no RTC boot

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* platform-configs: Add RTC_BOOT_SUPPORT

https://github.com/Dasharo/open-source-firmware-validation/issues/1068

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* CBMEM suite: dynamic semiauto skips for no RTC boot support

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* network-interface-after-suspend: Add workaroud for no RTC boot

https://github.com/Dasharo/open-source-firmware-validation/issues/1068

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* linux-packages-playbook.yaml: Remove masking systemd suspend

It makes it impossible to suspend the device in conventional
ways like using the UI or via suspend key combination.
It has more cons that pros

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* cbnt.robot: Fix invalid ibg menu parsing

Due to confusing line wrapping in the menu the
state of the options following to the target
were being parsed

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* SBO: Fix test ids ENV_ID

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* fast-boot.robot: Increase wait time for systemd-analyze

50s was not always enough for bootup to finish.
150s should be better.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* gpu-performance.robot: stability fixes

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

gpu-performance.robot: Fix string comparison error and x11 invalid display

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

gpu-performance.robot: Detect not running x11

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

gpu-performance.robot: Note that X11 is required

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

gpu-performance.robot: fix check for installed unigine-super

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

gpu-performance.robot: Add comment about display numbering bug

https://github.com/Dasharo/open-source-firmware-validation/issues/1215

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

lib/performance/common: Setup Phoronix: Upload results to openbenchmarking

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

disk-io.robot: Verify correct disk model before checking performance

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

disk-io.robot: move sleep to after skips to reduce unnecessary idle time

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* phoronix-test-suite tests: Cap run count to 3

Using:  FORCE_TIMES_TO_RUN env variable

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/docs.robot: Increase time for docking station to init

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* util/fwupd-support-lvfs: stylistic changes

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

util/fwupd-support-lvfs: Remove unused variable

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

util/fwupd-support-lvfs.robot: Add missing newline in fwupd config

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* cbnt.robot: Remove newline characters from cbmem regex

The newline characters are not matched by this keyword.
Its matched line-by-line so newline characters
never appear

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/utc.robot: stability fixes

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

lib/utc.robot: reboot tests add delay to prevent immediate re-login

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

lib/utc: hotplug tests: reorder disconnect/connect to end with connected

Otherwise the docking station would be left disconnected after
the tests causing further automated tests to possibly fail

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

UTC: Add dynamic skips for laptop semiautos

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* fwupd-support: Add --assume-yes to get-devices

First time its executed after a fwupd update it will prompt to send the
results to the vendor. It was not handled before.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* Execute Reboot Command: Add delay for no-power-ctrl devices

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* EXTERNAL_HEADSET: Change to list of acceptable values

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* BOOTED_OS_ID: Prevent resetting on new suite

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* warm/coldboot tests: Add dynamic semiauto skips if not supported

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* lib/robot.sh: Run all suites at once and split results later

We've made it so that the test suites are run separately so that
the logs can be organized neatly before.

Turns out that makes the state between suites get lost.
It's especially important for SSH tests which need to
always remember which OS is booted to use the correct
SSH credentials.

This commit changes this so that everything is run at once,
but the results are later parsed and split per-suite using
rebot.

An important thing to remember is that testers will often
interrupt the run using ctrl+c to get the test results
early and skip further execution. The change handles that
by trapping the INT signal and running the split
anyway.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

lib/robot.sh: Move robot to fg instead of waiting

When running robot as a subprocess it wont receive the control
over stdin otherwise. Some semi-auto tests require user input
in terminal which will crash otherwise.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

lib/robot.sh: fix detecting tags

tests that are fullauto on some and semi on some platforms
need to have both tags so that they are executed no matter
the `-i` param and the skips can take care of whether
the test is automated on given platform or not

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* TEST_TAGS: rename to INCLUDE_TAGS

Theres a builtin ${TEST TAGS}, but for some reason ${TEST_TAGS}
seems to be like an alias to it.

Our variable that keeps track of the tags requested by tester
needs to be changed because of that

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* options-lib_dcu: Set nextboot before Set UEFI Option

Set UEFI Option of options-lib_dcu reads the current
SMM Store, modifies it, and flashes it back.

If we set the Nextboot variable in SMM before reading
the SMM store it will stay after flash and reboot reducing
an unnecessary DEFAULT_BOOT_OS_ID boot.

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* sleep-lib.robot: warn on failed fwts sleep

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

sleep-lib: Add description of s3 test also working for s2idle(soix)

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* keywords.robot: Identify Path To USB: Fix returning TYPE instead of PATH

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* capsule-update.robot: Skip unnecessary setup when on ssh

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* scripts/lib/robot.sh & related: Restore old logs directory structure

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* audio-subsystem.robot: Delete old version of suite

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>

* platform-configs/include/protectli-vp66xx.robot: enable redundant boot

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* coreboot-redundant-boot.robot: work with the new log convention

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* platform-configs/include/protectli-vp66xx.robot: enable CBnT w/ status menu

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* lib/bios/menus.robot: Parse Menu Snapshot Into Construction: drop fuzzy search

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* platform-configs/include/protectli-vp66xx.robot: remove unused variable

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* coreboot-redundant-boot.robot: RTC BUC Control Bit Should Be: don't convert case

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

* coreboot-redundant-boot.robot: parse log from current boot only

Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>

---------

Signed-off-by: Filip Gołaś <filip.golas@3mdeb.com>
Signed-off-by: Michał Kopeć <michal.kopec@3mdeb.com>
Co-authored-by: Michał Kopeć <michal.kopec@3mdeb.com>
2026-02-06 13:02:09 +01:00

186 lines
8.3 KiB
Plaintext

*** Settings ***
Documentation Collection of keywords related to System Sleep States
Resource ../keywords.robot
*** Keywords ***
Check If Platform Sleep Type Can Be Selected
[Documentation] Check if there is a Platform sleep type option
VAR ${PLATFORM_SLEEP_TYPE_SELECTABLE}= ${FALSE} scope=GLOBAL
IF not ${TESTS_IN_FIRMWARE_SUPPORT} RETURN
IF ${DASHARO_POWER_MGMT_MENU_SUPPORT} == ${FALSE} RETURN
Power On
${setup_menu}= Enter Setup Menu Tianocore And Return Construction
${dasharo_menu}= Enter Dasharo System Features ${setup_menu}
${power_menu}= Enter Dasharo Submenu ${dasharo_menu} Power Management Options
${selectable}= Run Keyword And Return Status
... Get Option State
... ${power_menu}
... Platform sleep type
VAR ${PLATFORM_SLEEP_TYPE_SELECTABLE}= ${selectable} scope=GLOBAL
Save Changes And Reset
Set Platform Sleep Type
[Documentation] Set Platform sleep type to the given value
[Arguments] ${platform_sleep_type}
Power On
IF '${platform_sleep_type}' == 'S0ix'
VAR ${platform_sleep_type_text}= Suspend to Idle (S0ix)
ELSE IF '${platform_sleep_type}' == 'S3'
VAR ${platform_sleep_type_text}= Suspend to RAM (S3)
ELSE
Fail Wrong Argument
END
${setup_menu}= Enter Setup Menu Tianocore And Return Construction
${dasharo_menu}= Enter Dasharo System Features ${setup_menu}
${power_menu}= Enter Dasharo Submenu ${dasharo_menu} Power Management Options
Set Option State ${power_menu} Platform sleep type ${platform_sleep_type_text}
Save Changes And Reset
Check Platform Sleep Type Is Correct On Linux
[Documentation] Check Platform sleep type in Linux
[Arguments] ${platform_sleep_type}=${EMPTY}
IF '${platform_sleep_type}' == 'S0ix'
${power_mem_sleep}= Execute Command In Terminal cat /sys/power/mem_sleep
Should Contain Any ${power_mem_sleep} [s2idle] shallow [s2idle] deep # S0ix
ELSE IF '${platform_sleep_type}' == 'S3'
${power_mem_sleep}= Execute Command In Terminal cat /sys/power/mem_sleep
Should Contain ${power_mem_sleep} s2idle [deep] # S3
END
Detect Or Install FWTS
[Documentation] Keyword allows to check if Firmware Test Suite (fwts)
... has been already installed on the device. Otherwise, triggers
... process of obtaining and installation.
[Arguments] ${package}=fwts
VAR ${is_package_installed}= ${FALSE}
Log To Console \nChecking if ${package} is installed...
${is_package_installed}= Check If Package Is Installed ${package}
IF ${is_package_installed}
Log To Console \nPackage ${package} is installed
RETURN
ELSE
Log To Console \nPackage ${package} is not installed
END
Log To Console \nInstalling required package (${package})...
Get And Install FWTS
Sleep 10s
${is_package_installed}= Check If Package Is Installed ${package}
IF not ${is_package_installed}
FAIL \nRequired package (${package}) cannot be installed
END
Log To Console \nRequired package (${package}) installed successfully
Get And Install FWTS
[Documentation] Keyword allows to obtain and install Firmware Test Suite
... (fwts) tool.
Set DUT Response Timeout 500s
Write Into Terminal add-apt-repository ppa:firmware-testing-team/ppa-fwts-stable
Read From Terminal Until Press [ENTER] to continue or Ctrl-c to cancel
Write Bare Into Terminal ${ENTER}
Read From Terminal Until Reading package lists... Done
Write Into Terminal apt-get install --assume-yes fwts
Read From Terminal Until Prompt
Perform Suspend Test Using FWTS
[Documentation] Keyword allows to perform suspend and resume procedure
... test by using Firmware Test Suite tool.
... Despite the name, the `s3` test performs either an s3 deep or an
... s0ix (s2idle) suspend depending on the system default at `/sys/power/mem_sleep`.
... Type can be forced using `--s3-sleep-type deep` or `--s3-sleep-type s2idle`.
... https://github.com/fwts/fwts/blob/f299f280f2b1ceb22a6cc558aea9703ac61523c7/src/acpi/s3/s3.c#L216-L220
[Arguments] ${test_duration}=25 ${sleep_type}=${None}
VAR ${is_suspend_performed_correctly}= ${FALSE}
VAR ${test_time_out}= ${${test_duration}-5}
IF '${DUT_CONNECTION_METHOD}' == 'Telnet'
Execute Command In Terminal
... fwts s3 --s3-sleep=5 --s3-max-delay=10 -f -r /tmp/suspend_test_log.log
... ${test_time_out}s
# Clean up console before reading file
Read From Terminal
Write Bare Into Terminal ${ENTER}
Read From Terminal Until Prompt
ELSE
Write Into Terminal fwts s3 -f -r /tmp/suspend_test_log.log
Sleep ${test_duration}s
Login To Linux
Switch To Root User
END
${test_result}= Execute Command In Terminal cat /tmp/suspend_test_log.log
TRY
Should Contain ${test_result} 0 failed
Should Contain ${test_result} 0 warning
Should Contain ${test_result} 0 aborted
Should Contain ${test_result} 0 skipped
VAR ${is_suspend_performed_correctly}= ${TRUE}
EXCEPT
VAR ${is_suspend_performed_correctly}= ${FALSE}
Log FWTS suspend failed WARN
END
RETURN ${is_suspend_performed_correctly}
Get Current Suspend Mode For FWTS Linux
[Documentation] Returns either `s3` or `s0idle` depending
... on which sleep mode is currently active on linux,
... The output should be
${out}= Execute Command In Terminal cat /sys/power/mem_sleep
IF '[s2idle]' in """${out}"""
RETURN s0idle
ELSE IF '[deep]' in """${out}"""
RETURN s3
END
RETURN unknown sleep mode
Perform Hibernation Test Using FWTS
[Documentation] Keyword allows to perform hibernation and resume procedure
... test by using Firmware Test Suite tool
[Arguments] ${test_duration}=40
VAR ${is_hibernation_performed_correctly}= ${FALSE}
IF '${POWER_CTRL}' == 'none' Set Nextboot ${BOOTED_OS_ID}
Execute Command In Terminal fwts s4 -f -r /tmp/hibernation_test_log.log
Sleep ${test_duration}s
Boot System Or From Connected Disk ${BOOTED_OS_ID}
Login To Linux
Switch To Root User
${test_result}= Execute Command In Terminal cat /tmp/hibernation_test_log.log
TRY
Should Contain ${test_result} 0 failed
Should Contain ${test_result} 0 warning
Should Contain ${test_result} 0 aborted
Should Contain ${test_result} 0 skipped
VAR ${is_hibernation_performed_correctly}= ${TRUE}
EXCEPT
VAR ${is_hibernation_performed_correctly}= ${FALSE}
END
RETURN ${is_hibernation_performed_correctly}
Perform Warmboot Using Rtcwake
[Documentation] Executes a command that will cause a warmboot
Skip If
... not ${RTC_BOOT_SUPPORT} and ${INCLUDE_TAGS} is not ${None} and 'semiauto' not in ${INCLUDE_TAGS}
# Using "Execute Command In Terimal" will cause the test to wait
# for command prompt to appear before continuing but the prompt
# will not appear again until we Login after reboot, so the test
# would hang here and fail.
# Sometimes it may take long to shutdown all systemd services,
# so the waiting times have to be excessive to avoid false negatives.
IF '${POWER_CTRL}' == 'none' Set Nextboot ${BOOTED_OS_ID}
Write Into Terminal rtcwake -m off -s 60
Set DUT Response Timeout 300s
Sleep 60s
Perform Suspend And Wake Using Rtcwake
[Documentation] Suspends and then wakes up the device after some time
# Using "Execute Command In Terimal" will cause the test to wait
# for command prompt to appear before continuing but the prompt
# will not appear again until we Login after reboot, so the test
# would hang here and fail.
# Sometimes it may take long to shutdown all systemd services,
# so the waiting times have to be excessive to avoid false negatives.
Write Into Terminal rtcwake -m ram -s 20
Set DUT Response Timeout 300s
Sleep 20s