dts-scripts/tests

DTS scripts unit tests

This directory contains stub scripts for testing DTS update and deploy logic in complex scenarios.

Running on host

Running test on the host may result in unpredictable results because of the missing programs and different version of certain tools. It is advised to run the DTS image in QEMU as a development environment. Running on host is generally not supported and should be avoided.

Running in QEMU

Credentials setup

We need credentials for each test variant. You can use provided template and fill it in accordingly.

cp des-credentials.sh.example des-credentials.sh

Running automatically

Some scenarios are have been already migrated into OSFV.

robot -L TRACE -v config:qemu -v rte_ip:127.0.0.1 -v snipeit:no dts/dts-tests.robot

Running manually

1. Boot the latest DTS image in QEMU. Recommended steps:

   • start QEMU according to OSFV documentation (use os switch, not firmware)
   • enable network boot and boot into DTS via iPXE
   • enable SSH server (option 8 in main menu)

2. Deploy updated scripts and tests into qemu

   PORT=5222 ./scripts/local-deploy.sh 127.0.0.1
   

3. Execute desired test as described in below section. E.g.:

   ssh -p 5222 root@127.0.0.1
   export BOARD_VENDOR="Notebook" SYSTEM_MODEL="NV4xPZ" BOARD_MODEL="NV4xPZ"
   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.7.2" TEST_DES=y DES_TYPE="heads" && dts-boot
   

Test cases

The general idea is that we override some variables, so DTS scripts consider they are running on the given board. Then we select Install or Update actions from DTS menu, and check if the flow is as expected in certain scenario.

After each dts-boot -> 5) Check and apply Dasharo firmware updates scenario execution, we can drop to DTS shell and continue with the next scenario.

NovaCustom

export BOARD_VENDOR="Notebook" SYSTEM_MODEL="NV4xPZ" BOARD_MODEL="NV4xPZ"

1. Dasharo v1.7.2 on NV4x_PZ eligible for updates to heads with heads DES and regular update:

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.7.2" TEST_DES=y DES_TYPE="heads" && dts-boot
   

   Expected output:

   • heads fw should be offered

2. Dasharo v1.7.2 on NV4x_PZ eligible for updates to heads without DES (regular update only):

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.7.2" TEST_DES=n && dts-boot
   

   Expected output:

   • no update should be offered

3. Dasharo v1.6.0 on NV4x_PZ not eligible for updates to heads with heads DES (regular update only):

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.6.0" TEST_DES=y DES_TYPE="heads" && dts-boot
   

   Expected output:

   • UEFI fw update should be offered (this is too old release to transition to heads directly, need to flash latest UEFI fw first)

4. Dasharo v1.6.0 on NV4x_PZ not eligible for updates to heads without heads DES (regular update only):

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.6.0" TEST_DES=n && dts-boot
   

   Expected output:

   • UEFI fw update should be offered

5. Dasharo heads v0.9.0 on NV4x_PZ eligible for updates to heads with heads DES and switch back (heads updates):

   export BIOS_VERSION="Dasharo (coreboot+heads) v0.9.0" TEST_DES=y DES_TYPE="heads" && dts-boot
   

   Expected output:

   • migration to UEFI should be offered first
   • if we say n to switch, heads update should be offered

6. Dasharo heads v0.9.0 on NV4x_PZ without DES switch back, no heads updates:

   export BIOS_VERSION="Dasharo (coreboot+heads) v0.9.0" TEST_DES=n && dts-boot
   

   Expected output:

   • migration to UEFI should be offered first
   • if we say n to switch, no heads update should be offered

Another case is to edit dts-functions.sh and set DASHARO_REL_VER to v1.7.3 to detect possible regular firmware updates and HEADS_REL_VER_DES to v0.9.1 to detect possible heads firmware updates and repeat all test cases. The URLs for non-existing versions may fail.

The NovaCustom test binaries for credentials in dts-boot are placed in /projects/projects/2022/novacustom/dts_test on 3mdeb cloud. These are just public coreboot+UEFI v1.7.2 binaries. Analogically with MSI, cloud directory is /projects/projects/2022/msi/dts_test and binaries are simply Z690-A public coreboot+UEFI v1.1.1 binaries with changed names for both Z690-A and Z790-P (resigned with appropriate keys).

MSI MS-7D25

export BOARD_VENDOR="Micro-Star International Co., Ltd." SYSTEM_MODEL="MS-7D25" BOARD_MODEL="PRO Z690-A WIFI DDR4(MS-7D25)"

1. Dasharo v1.1.1 on MS-7D25 eligible for updates to heads with heads DES and regular update:

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.1.1" TEST_DES=y DES_TYPE="heads" && dts-boot
   

2. Dasharo v1.1.1 on MS-7D25 eligible for updates to heads without DES (regular update only):

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.1.1" TEST_DES=n && dts-boot
   

3. Dasharo v1.1.2 on MS-7D25 eligible for updates to heads with heads DES (regular update only through regular DES):

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.1.2" TEST_DES=y DES_TYPE="heads" && dts-boot
   

4. Dasharo v1.1.2 on MS-7D25 not eligible for updates to heads without heads DES (regular update only through regular DES):

   export BIOS_VERSION="Dasharo (coreboot+UEFI) v1.1.2" TEST_DES=n && dts-boot
   

5. Dasharo heads v0.9.0 on MS-7D25 eligible for updates to heads with heads DES and switch back (regular update and switch-back):

   export BIOS_VERSION="Dasharo (coreboot+heads) v0.9.0" TEST_DES=y DES_TYPE="heads" && dts-boot
   

6. Dasharo heads v0.9.0 on MS-7D25 without DES switch back, no heads updates (regular update and switch-back):

   export BIOS_VERSION="Dasharo (coreboot+heads) v0.9.0" TEST_DES=n && dts-boot
   

MSI MS-7E06

export BOARD_VENDOR="Micro-Star International Co., Ltd." SYSTEM_MODEL="MS-7E06" BOARD_MODEL="PRO Z790-P WIFI (MS-7E06)"

1. Dasharo heads v0.9.0 on MS-7E06 eligible for updates to heads with heads DES and switch back (regular update and switch-back only through regular DES, no community release):

   export BIOS_VERSION="Dasharo (coreboot+heads) v0.9.0" TEST_DES=y DES_TYPE=heads && dts-boot
   

   Expected output:

   • migration to UEFI should be offered first
   • if we say n to switch, no heads (no more recent update available yet)

2. Dasharo heads v0.9.0 on MS-7E06 without DES switch back, no heads updates (regular update and switch-back only through regular DES, no community release):

   export BIOS_VERSION="Dasharo (coreboot+heads) v0.9.0" TEST_DES=n && dts-boot
   

   Expected output:

   • should print info on DES availability in the shop
   • migration to UEFI should be offered

PC Engines

export BOARD_VENDOR="PC Engines" SYSTEM_MODEL="APU2" BOARD_MODEL="APU2"

1. Initial deployment from legacy firmware (no DES credentials)

   export BIOS_VERSION="v4.19.0.1" TEST_DES=n && dts-boot
   

   Expected output:

   • no DES - no deployment should be offered
   • info on DES availailbity in the shop should be shown

2. Initial deployment from legacy firmware (UEFI DES credentials)

   export BIOS_VERSION="v4.19.0.1" TEST_DES=y DES_TYPE="UEFI" && dts-boot
   

   Expected output:

   • UEFI deployment should be offered
   • info on DES availailbity in the shop should not be shown

3. Initial deployment from legacy firmware (seabios DES credentials)

   export BIOS_VERSION="v4.19.0.1" TEST_DES=y DTS_TYPE="seabios" && dts-boot
   

   Expected output:

   • Seabios deployment should be offered
   • info on DES availailbity in the shop should not be shown

4. Initial deployment from legacy firmware (correct DES credentials)

   export BIOS_VERSION="v4.19.0.1" TEST_DES=n && dts-boot
   

   Expected output:

   • seabios deployment should be offered
   • info on DES availailbity in the shop should not be shown