4536 Commits

Author SHA1 Message Date
Sergio Cazzolato
52913814cc Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..cdf5cfd47b (#14157)
cdf5cfd47b Remove centos-7 support
f3996cc3fa change the spread label
1e309f41c6 change how legacy parameter is determine in remote.pull
c43c35f7e3 run remote refresh and wait-for for xenial (skip bionic)
5262d30da7 make sure the test jobs are executed in runners with the spread label
cb74259b7a add openstack systems
0b41fd40d3 fix tests.pkgs on arch-linux
558e109793 run fedora-40 spread tests in openstack
6f6187416d fix list implementation
b4a5439c9b added more type annotatios to log_helper
58da1e36c3 mypy cleaned
1ff651e680 update wording of remote.pull
18615b1667 just usc scp -O when ssh version is -ge 9
cc68c9868b Added type annotations for log-filter
66f90d10cd Adding -O to scp command to make it compatible in uc24 tests
496cb7b5b3 removing support for centos-8
f2eef30db4 Updated the log helper and log parser
5a375ebf73 Formatting for python utils
d3eed3faa5 fix codespell in CODE_OF_CONDUCT.md
18bcca6b14 new log helper
d60381fcd9 add run number to filtered filename
5dde2d67b8 consider the tests execution in main
6b9a3aabcc change filtered log name
b2756aa579 default file is .filtered.log
500b9dace4 Fix tests workflow
45db26a3d2 fix shellcheck error in log-filter
fe45c27b7d create a var to store filter params
5a9b66d7dc filter spread results
51f9b055af New tool used to filter the spread.log output
b8d20c1d5b fix snaps.name test with correct siffix spelling
f640ac72e3 Add missing test details
f0754df304 Filter the error y debug output in log-parser
fc10196efd Add suggestions to details
94ac5ffe58 Add details on tests
501578c719 add more checks in os.query to check is-core_xx
e8929207ff fix os-query for ubuntu comparing with core
226114641f os.query won't check SPREAD_SYSTEM anymore to compare core systems
b89ec98b23 use local variables in os.query tool
dacfd81de9 fix is_core functions
1db5214d5f Improve the remote docs (#36)
2e4a3153a2 1 more comment
3a0fc57e1e add explanation about why we check for ( Do | Doing )
4cf8e635bf fix os.query test after merge
b89b4f8647 fix artifacts name
d30cee6da0 Merge remote-tracking branch 'upstream/main'
5ef5dcbe8f Tests use artifacts in spread tests (#51)
555c43d2ab Support auto-refresh with Do instead of Doing
96c2b0c19c remove tests support for ubuntu 23.04 (EoL)
74082c0c34 Tests improve remote wait (#49)
5121bfb659 remove support for opensuse leap 15.4 (#48)
30df700d08 Add new systems support (#47)
1f08938925 Support check amazon linux version (#46)
43533bdd97 Change the exit value checking for test formats (#45)
3c88244c04 Update check-test-format to support a dir and a list of files (#44)
510d95f429 add extra check for error in auto-refresh detection function
3289d4031b Try open the log with latin-1 encoding when utf-8 is not working
9db785499f improved how the tools are waiting for system reboot
2a5c4414a3 fix shellcheck errors
5e7b63883d Fixes for osquery and tests pkgs (#43)
4c9145e2ac support reboot waiting for auto-refresh
45768f5188 show changes in unknown status after refresh
8013c30c2a Remove support for ubuntu 22.10
b32b80bf54 Fix remote.rait-for test in bionic
5675c625e9 Enable fedora 38
55f4471957 Support for new oss
f2e88b357c New tool used to query spread json reports
cacd35ede0 utils/spread-shellcheck: explain disabled warnings (#42)
c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs
b84eea92e2 spread-shellcheck: fix quotes in environment variables (#41)
ab1e51c29f New comparison in os-query for core systems (#40)
e5ae22a5d4 systemd units can be overwritten
63540b845a Fix error messages in remote pull and push
75e8a426a5 make sure the unit is removed in tests.systemd test
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
162e93bd35 update tests.systemd CLI options to be the same than retry command
14aa43a405 new feature to re-run failed spread tests (#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: cdf5cfd47b4b9d528ab4544c1d6900dec797fbf2
2024-07-04 17:42:20 -03:00
Sergio Cazzolato
f40b9595a1 tests: remove centos 7 support (#14147)
* tets: remove support for centos-7

Centos-7 is EOL since June-30

Signed-off-by: Sergio Cazzolato <sergio.cazzolato@canonical.com>

* update the release_tst.go with the real centos 9 info

---------

Signed-off-by: Sergio Cazzolato <sergio.cazzolato@canonical.com>
2024-07-04 11:28:21 -03:00
Valentin David
b1dc3c675d tests: properly build snapd snap (#14141)
* tests: properly build snapd snap

Now we build also the test version of snapd snap in `snap-builds`
workflow job. We copy this into the spread tests. And we use that
snap, which we only instrument instead of copying the snapd deb build.

If the snap is not available, then we build it in spread. On CI, this
happens on arm since the workflow does not build it. It will also
happen when triggering test manually.

* tests: couple of small improvements to test syntax, move WORK_DIR into script scope, use PWD instead of dot notation

---------

Co-authored-by: Philip Meulengracht <the_meulengracht@hotmail.com>
2024-07-03 13:36:31 +02:00
Sergio Cazzolato
a7c15c2ca1 tests: fix fedora-40 issues and remove centos-8 support in tests (#14145)
This changs fixes some errors that are reproduced in fedora-40 executed
in openstack backend.
Also it is removed the centos-8 support in tests.
2024-07-01 15:34:06 -03:00
Sergio Cazzolato
d6efbc4d14 tests:updated the backends list used for openstack (#14131)
ubuntu arm-64 is removed because it fails to boot in nightly
ubuntu jammy and noble are added to be executed in nighty run
2024-06-27 17:25:40 -03:00
Maciej Borzecki
ca6668a481 tests/core/netplan: override ld.so cache to avoid the host's cache files
The test breaks on UC24 in a peculiar way. The `netplan` CLI tool is a Python
script which attempts to load libnetplan.so.* through ctypes. However, in a
snap, the contents of /etc come from the host, so in case on UC24 environment,
the netplan client invoked from eg. core20 would observe actual ld.so.cache from
UC24. Thus Python's ctype library, would call `ldconfig-p`, which then consumes
the ld.so.cache from UC24, thus listing incorrect version of the libnetplan
library (specifically UC24 has libnetplan.so.1, while earlier versions had
libnetplan.so.0.0).

Attempt to fix this by providing a custom wrapper for ldconfig, which generates
a cache on the side under $SNAP_DATA, thus using the libraries which are
actually visible to the snap.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
2024-06-26 11:32:52 +02:00
Sergio Cazzolato
2f506b5955 tests: increase the number of workers we use in openstack
This is needed for the nightly suite where we run the other systems
2024-06-25 08:45:05 +02:00
Alfonso Sánchez-Beato
8f06a9927e tests/lib/nested.sh: fix check 2024-06-21 13:00:02 +01:00
Sergio Cazzolato
7f9afe80e4 tests: run spread tests in fedora-40 (#14099)
* Adding new backends

As we cannot update the spread.yaml with the new backends, because
spread from master does not supported them. I added the new file with
the definitions which can be merged when it needs to be used.
A merging tool is comming next

* Add env details to openstack spread.yaml

* update disk for openstack

* update testflinger yaml config files

* openstack systems updated

* new script to add backend to spread.yaml

* add backend also to get failed tests

* add more workers to run fedora-40 tests
2024-06-20 09:20:20 -03:00
Miguel Pires
8128ed20bb many: rename aspect/bundle to view/registry
This changes the naming of the aspects feature to be "registry" instead
of bundle (i.e., a configuration space backed with its own storage) and
"view" instead of aspect. Once this lands, anyone that has this enabled
needs to unset the experimental flag and rename the state entry before
refreshing snapd and then re-enable.

Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
2024-06-17 17:16:57 +02:00
Sergio Cazzolato
144717cb4d tests: fix uc24 nested tests part 2 (#14069)
* update gadget.yaml for nested remodel tests

* add new snaps with base: core24

* tests: fixes for uc24 nested tests

This change contains:
. fixes for model
. use of core24 snaps (jq, rsync and sh)
. test fixes
. using legacy scp in snapd-testing-tools remote-pull (to avoid errors)

* use -O when the scp version is -ge than 9

* re-enable hybrid-remodel test on noble

* update wording of remote.pull

* Adding more workers for nested tests

This is to avoid instances are cleaned before the execution finishes
2024-06-14 09:29:55 -03:00
Sergio Cazzolato
3f5225c2aa tests: new spread log utils (#14062)
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..496cb7b5b3

496cb7b5b3 removing support for centos-8
f2eef30db4 Updated the log helper and log parser
5a375ebf73 Formatting for python utils
d3eed3faa5 fix codespell in CODE_OF_CONDUCT.md
18bcca6b14 new log helper
d60381fcd9 add run number to filtered filename
5dde2d67b8 consider the tests execution in main
6b9a3aabcc change filtered log name
b2756aa579 default file is .filtered.log
500b9dace4 Fix tests workflow
45db26a3d2 fix shellcheck error in log-filter
fe45c27b7d create a var to store filter params
5a9b66d7dc filter spread results
51f9b055af New tool used to filter the spread.log output
b8d20c1d5b fix snaps.name test with correct siffix spelling
f640ac72e3 Add missing test details
f0754df304 Filter the error y debug output in log-parser
fc10196efd Add suggestions to details
94ac5ffe58 Add details on tests
501578c719 add more checks in os.query to check is-core_xx
e8929207ff fix os-query for ubuntu comparing with core
226114641f os.query won't check SPREAD_SYSTEM anymore to compare core systems
b89ec98b23 use local variables in os.query tool
dacfd81de9 fix is_core functions
1db5214d5f Improve the remote docs (#36)
2e4a3153a2 1 more comment
3a0fc57e1e add explanation about why we check for ( Do | Doing )
4cf8e635bf fix os.query test after merge
b89b4f8647 fix artifacts name
d30cee6da0 Merge remote-tracking branch 'upstream/main'
5ef5dcbe8f Tests use artifacts in spread tests (#51)
555c43d2ab Support auto-refresh with Do instead of Doing
96c2b0c19c remove tests support for ubuntu 23.04 (EoL)
74082c0c34 Tests improve remote wait (#49)
5121bfb659 remove support for opensuse leap 15.4 (#48)
30df700d08 Add new systems support (#47)
1f08938925 Support check amazon linux version (#46)
43533bdd97 Change the exit value checking for test formats (#45)
3c88244c04 Update check-test-format to support a dir and a list of files (#44)
510d95f429 add extra check for error in auto-refresh detection function
3289d4031b Try open the log with latin-1 encoding when utf-8 is not working
9db785499f improved how the tools are waiting for system reboot
2a5c4414a3 fix shellcheck errors
5e7b63883d Fixes for osquery and tests pkgs (#43)
4c9145e2ac support reboot waiting for auto-refresh
45768f5188 show changes in unknown status after refresh
8013c30c2a Remove support for ubuntu 22.10
b32b80bf54 Fix remote.rait-for test in bionic
5675c625e9 Enable fedora 38
55f4471957 Support for new oss
f2e88b357c New tool used to query spread json reports
cacd35ede0 utils/spread-shellcheck: explain disabled warnings (#42)
c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs
b84eea92e2 spread-shellcheck: fix quotes in environment variables (#41)
ab1e51c29f New comparison in os-query for core systems (#40)
e5ae22a5d4 systemd units can be overwritten
63540b845a Fix error messages in remote pull and push
75e8a426a5 make sure the unit is removed in tests.systemd test
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
162e93bd35 update tests.systemd CLI options to be the same than retry command
14aa43a405 new feature to re-run failed spread tests (#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 496cb7b5b3c160dc86e84fb7adca3bd5c862e88e

* log-parser does not support anymore --print-detail

* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 496cb7b5b3..cc68c9868b

cc68c9868b Added type annotations for log-filter
66f90d10cd Adding -O to scp command to make it compatible in uc24 tests

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: cc68c9868bcc504497e0a8e459810c8ca2aa0c8d

* Squashed 'tests/lib/external/snapd-testing-tools/' changes from cc68c9868b..18615b1667

18615b1667 just usc scp -O when ssh version is -ge 9

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 18615b1667b6f292d29333f34a973938a28c0e7b

* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 18615b1667..1ff651e680

1ff651e680 update wording of remote.pull

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 1ff651e6804b32184f7b62f9492c58d95a397cc1

* Update tests/lib/external/snapd-testing-tools/tests/remote.retry/task.yaml

Co-authored-by: Zygmunt Bazyli Krynicki <me@zygoon.pl>

* fix merge issue

* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1ff651e680..58da1e36c3

58da1e36c3 mypy cleaned

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 58da1e36c3c95398af5b2111ebcef87eebb7bd46

* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 58da1e36c3..b4a5439c9b

b4a5439c9b added more type annotatios to log_helper

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: b4a5439c9bd31f11f7fa08515a8142b399fcbcc8

* Squashed 'tests/lib/external/snapd-testing-tools/' changes from b4a5439c9b..6f6187416d

6f6187416d fix list implementation

git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 6f6187416da05d9b6799c10bfb6534085a13524b

---------

Co-authored-by: Zygmunt Bazyli Krynicki <me@zygoon.pl>
2024-06-14 08:55:22 -03:00
alfonsosanchezbeato
20cde6b251 many: build drivers tree when current mount is not the target mount (#14047)
* tests: fix muinstaller tests for 24

* many: build drivers tree when current mount is not the target mount

In some cases (when using the snapd install API or when installing
from initramfs), the place where the kernel snap / components used for the
installation are mounted is different to the final location in the
installed system. This change considers this so the drivers tree is
generated with symlinks pointing to the final expected location.

* overlord: use model to check if we need to set-up drivers tree

instead of using a device context, as for the installation using snapd
API case we have a model but not a context.

* tests/lib/tools/setup_nested_hybrid_system.sh: re-try kpartx -d

* tests/muinstaller-real: check that drivers tree is created

* tests/muinstaller-real: we need a bigger disk with latest kernel

* tests/lib/tools/setup_nested_hybrid_system.sh: clean up

after building muinstaller. On classic we have weird issues otherwise
due to a desktop agent installing lxd.

* tests/lib/prepare-restore.sh: purge lxd-installer

lxd-installer was causing failures in the restore step for 24.04.
2024-06-11 20:07:52 +01:00
Sergio Cazzolato
7d480d74e9 tests: first set of changes to make nested tests run (#13889)
* First list of changes to make nested tests run

* avoid using OVMF with 4M

* Check for cloud init status in uc24

* removed OVMF_SIZE var which is not used anymore

* fix merge issue

* update gadget.yaml for test-snapd-remove-pc-24 snap

* remove mbr and source: shim.efi.signed from gadget.yaml

* restoring source: shim.efi.signed in seed partition

* Fixed the incorrect uuid of the core24 snap

* fixed the snapcraft.yaml for jq-core24
2024-06-10 08:56:55 +02:00
Sergio Cazzolato
7b5b2233b4 tests: add rpm-build as dependency for opensuse systems
To make opensuse tests work in openstack, it is needed to install
rpm-build.
2024-06-07 09:32:12 +02:00
Sergio Cazzolato
3c2241c89f tests: increase the StartLimitBurst default value to 10 (#14051)
* tests: increase the StartLimitBurst default value to 10

As to prepare the snapd we restart at least 4 times (5 when we don't do
re-exec) in fast vms it is easy to reach the starts limit and make the
tests fail.

* Update prepare.sh
2024-06-06 12:52:39 -03:00
Zeyad Yasser
3fc2456b9c run-checks: skip high-level tests requirement for interfaces with a standalone test (#14033)
* run-checks: skip high-level tests requirement for interfaces with a standalone test

interfaces-many-{core,snap}-provided tests are doing minimal testing for
all interfaces. This became one of the slowest tests that it started to timeout.

A check is added to relax the requirement for high-level tests and
skip if a standalone test already exists for the interface which at
least connects and disconnects the interface.

The requirement can be relaxed further in the future to skip the
connect/disconnect checks allowing more tests to be skipped.

Signed-off-by: Zeyad Gouda <zeyad.gouda@canonical.com>

* run-checks: relax interface checks and error on redundant tests

Signed-off-by: Zeyad Gouda <zeyad.gouda@canonical.com>

* tests: s/interfaces-input/interfaces-raw-input

Signed-off-by: Zeyad Gouda <zeyad.gouda@canonical.com>

---------

Signed-off-by: Zeyad Gouda <zeyad.gouda@canonical.com>
2024-06-05 12:12:51 +02:00
Sergio Cazzolato
05c117cc18 tests: revert change to used --comment in adduser (#14038)
The nested tests are failing bucause of this problem

--comment is not supported by all the adduser versions (Unknown option:
comment)
2024-06-03 22:43:09 -03:00
Valentin David
75f813b41b tests: do not use /etc/timezone
/etc/timezone is an old file that was only supported on Debian and
Ubuntu but has been removed in 24.04 and upcoming Debian.

Whether /etc/timezone works or not is a problem for systemd. We do not
have to test it.
2024-06-03 14:59:03 +02:00
Valentin David
2034c7edb2 boot,bootloader: add support for shim fallback and setting EFI boot variables on install (#13511)
* boot: added function to set EFI variables

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: renamed trustedShimFallbackBinary to seedShimPath

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: refactored setting EFI boot variables at install

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: adjusted variable names and fixed variable initialization

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: improve setting Boot#### EFI variable

Notably, splits off the process of reading a Boot#### variable and
extracting its DevicePath into its own function `readBootVariable` which
can be mocked and otherwise simplifies the `setBootNumberVariable`
function.

Also, fixes behavior around the final BootFFFF variable.  Previously, it
was not possible to select the BootFFFF variable if it was unused, due
to overflow concerns on uint16.  Now, the behavior around BootFFFF is
identical to that of any other boot variable, by using an int internally
instead of uint16, which also allows a more robust check for whether
there were no matching variables.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: added unit tests for setting EFI Boot#### variable

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: refactored setting EFI boot variables

Rewrote EFI boot variable functions to more closely match the behavior
of shim fallback: https://github.com/rhboot/shim/blob/main/fallback.c

In particular, the following have changed:

1. Existing Boot#### variables must fully match the new load option to
   be considered a match.  In particular, the load option attributes,
   label, and device path must all be byte-for-byte identical.
   Previously, only the device paths were compared.
2. Matching Boot#### variables are no longer overwritten.  Since the
   variable data must now byte-for-byte match the new load option, there
   is no need to overwrite the existing variable.
3. Since existing Boot#### variables are no longer overwritten, the
   variable attributes are no longer checked for those variables.
   Instead, it is assumed that the Boot#### variable attributes are
   viable for it to be used as a boot option.  This matches the behavior
   of `rhboot/shim/fallback.c`, for better or for worse.
4. When modifying the BootOrder variable, boot option numbers are no
   longer pruned if there is no matching Boot#### variable.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot,bootloader: introduce UefiBootloader to build EFI load options

Previously, the path of the shim binary relative to the EFI partition
was passed into `SetEfiBootVariables`. However, different bootloaders
may wish to set up `OptionalData` in the load option.

Additionally, not all `TrustedAssetBootloaders` will attempt to set
EFI boot variables, and not all bootloaders which should set EFI boot
variables necessarily support secure boot. Thus, these should be
decoupled.

This commit adds a new `UefiBootloader` interface with the
`ConstructShimEfiLoadOption` method, which builds an EFI load option
from the shim path for the given bootloader.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot,bootloader: fixed linting errors and improved EFI boot variable test clarity

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

bootloader: improved unit test for grub EFI load option creation

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: set EFI boot variables in `MakeRunnableSystem`

Previously, attempted to set boot variables in
`MakeRecoverySystemBootable`, which is called by `MakeBootableImage`,
which is called when building the image file, rather than during install
mode.

`MakeRunnableSystem` is called on first boot during install mode, and
thus should be responsible for setting EFI boot variables.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: use seed bootloader when setting EFI variables

In install mode, the bootloader located in ubuntu-seed should be used
when setting the EFI boot variables. Previously, the bootloader in
ubuntu-boot was accidentally re-used.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: added simple test to execute setefibootvar.go code

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: fixed standalone set EFI vars code test to work with different layouts

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: moved simple setefibootvar.go check to nested test

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: added check for idempotence when setting EFI boot variables

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

bootloader: adjust comments, organization, and add TODO

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot,bootloader: fix setting EFI boot variables

Make function to search for EFI asset device path and construct load
option common so each UefiBootloader does not have to re-implement it.
Instead, the bootloader returns the description, asset file path, and
optional data, which can then be used to create the EFI load option.

Also, in `makeRunnableSystem`, the bootloader in ubuntu-seed must have
`NoSlashBoot` in order to correctly find the grub.cfg file and thus the
grub bootloader. This commit fixes this bug, and refactors a bit to
account for the changes in responsibilities between the bootloader and
the setefibootvars.go code.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

bootloader: fixed grub EFI load option test with tmp rootdir

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

go.mod: move golang.org/x/text import next to other golang.org/x/ imports

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: adjust opts to look for recovery bootloader when setting EFI variables

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: do not overwrite BootOrder if unchanged, and unexport EFI variable helper functions

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: unexport `setEfiBootOrderVariable`

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: move code to detect bootloader and set EFI variables accordingly into dedicated function

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: unexport `setUbuntuSeedEfiBootVariables` and accompanying error

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot,bootloader: ensure nil optionalData for EFI variable is equivalent to 0-length slice

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: handle empty boot order and other boot var improvements

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

boot: make setefibootvars functions linux-only

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

* tests: add nested spread test for setting EFI vars

The test checks that EFI boot variables exist for the following:
1. A Boot#### variable pointing to the shim file path.
2. A BootOrder variable with the #### from the above Boot#### as first.

Since the layout of EFI assets is dependent on the gadget snap, the test
downloads and unpacks the gadget, then modifies the contents so that one
variant has the shim and grub binaries in `EFI/boot/` and another
variant has the shim and grub binaries in `EFI/ubuntu/` and the fallback
binary in `EFI/boot/`.

After building a core image around that modified gadget, the VM is
booted and the test checks that the EFI variables are set correctly.
Then, the test modifies the gadget to match the other variant's initial
layout, and then installs the newly modified gadget. This should trigger
re-setting EFI boot variables as well.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: fix problems in spread test for setting EFI boot variables

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: disabled TPM on EFI boot vars test and separated gadget script

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: fixed EFI vars test to use correct toolbox and include all EFI assets

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: modify-gadget.sh re-use existing gadget so edition is incremented

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: fix mangled EFI var search string and other improvements

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

tests: polish tests for setting EFI boot variables

Notably, allow tests/nested/core/core20-set-efi-boot-variables to run on
arm64 as well as amd64, simplify setefivars.go to search for multiple
assets on multiple architectures, and allow
tests/nested/manual/core20-set-efi-boot-vars to run on any ubuntu-2*.

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>

* bootloader/grub.go: only consider new shim asset in boot entry for now

* tests/nested/core/core20-set-efi-boot-variables: fix details

* boot: update uefi variables on gadget update

* tests/nested/manual/core20-set-efi-boot-vars: work-around file not deleted

* tests/nested/manual/core20-set-efi-boot-vars: use fb.efi like other tests

* tests/nested/manual/core20-set-efi-boot-vars: drop use of toolbox snap

* tests/nested/manual/core20-set-efi-boot-vars: drop work-around for not deleted files

* tests/nested/manual/core20-set-efi-boot-vars: verify install does add a boot entry

* tests/nested/manual/core20-set-efi-boot-vars: run only on version that have UC

* tests/nested/manual/core20-set-efi-boot-vars: obey GADGET_CHANNEL

* tests/nested/manual/core20-set-efi-boot-vars: move get_boot_entry.py to libs

* tests/nested/manual/core20-set-efi-boot-vars: factorize copy of variables

... so we can reuse the script in other tests

* tests/nested/core/core20-set-efi-boot-variables: stop using toolbox snap

* tests/nested/core/core20-set-efi-boot-variables: only run on versions with UC available

* overlord/devicestate: test using EfiLoadOptionParameters

* boot: test that variables are set

* boot: test observers' UpdateBootEntry

* tests/nested/manual/core20-set-efi-boot-vars: also test without secure boot

* many: use trusted install observer when UEFI variables are supported

* boot/makebootable.go: rename sealer to observer

* boot/grub.go: fix function name in doc

* cmd/snap-bootstrap: verify that ObserveExistingTrustedRecoveryAssets is called

* boot: add tests for SetEfiBootVariables

* many: comment on calls to ObserveExistingTrustedRecoveryAssets

---------

Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
Co-authored-by: Oliver Calder <oliver.calder@canonical.com>
2024-06-03 10:03:50 +02:00
Sergio Cazzolato
1f1ffe3557 tests: create test user with useradd in uc24 (#14028)
* tests: create test user with useradd in uc24

This change is needed to create the test user in uc24 with home
directory.

This is needed to fix tests in uc24 (edge validation)

* use --create-home and --comment

--gecos is deprecated
2024-05-31 16:04:01 -03:00
Miguel Pires
8edf6a844a tests: deal with pre-release suffix in systemd versions (#14013)
* tests: deal with pre-release suffix in systemd versions

Debian sid is using an rc version of systemd which tests.session and
various other tests weren't dealing with. This change cuts the
pre-release suffix from all of them although some were only used by
distros in which this isn't a problem. This is more consistent and
robust IMO.

Signed-off-by: Miguel Pires <miguel.pires@canonical.com>

* tests: fix var ref

Signed-off-by: Miguel Pires <miguel.pires@canonical.com>

---------

Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
2024-05-30 17:21:43 -03:00
alfonsosanchezbeato
b052830f37 many: build drivers tree on install (#13923)
Build drivers tree on installation so kernel modules can be found on first boot after installation (otherwise the modules would be available too late, after initial installation of the kernel has happened while seeding).

Also, make sure that the tree is not re-created when seeding, and add a spread test for kernel-modules components.

* dirs: add function to get root of drivers trees

* kernel: do not recreate drivers tree if it already exists

On installation, a drivers tree has already been copied to the data
partition. Do not re-create if found when seeding happens (note though
that on preseeding we will not find the tree and we will still create
it).

* gadget/install: create device tree on installations

Create a device tree on installation so we have it on first boot.

* o/snapstate: fix task message

* tests/nested/manual: add kernel-modules-components test

This test installs a kernel-modules component and verifies that the
shipped kernel module is installed and can be loaded.

* tests/lib: do not error out if cloud-init finished with

'recoverable error' status.

* kernel: have destination as argument when building drivers tree

We need to tackle the generic installation case as opposed to creating
the drivers tree when seeding the system. In that case the destination
tree can vary depending on installation type.

* overlord: pass kernel information to install package

so the drivers tree can be created.

* cmd/snap-bootstrap: build drivers tree on installation from initramfs

* tests: adapt to additional parameters in install.Run

* snap/naming: add ancillary method to split a full component name

into snap and component names.

* gadget/install: fix nosecboot tests
2024-05-29 15:58:22 +01:00
Maciej Borzecki
bff650d8d2 HACKING, packaging: update dependencies and information on installing development dependencies (#13968)
* HACKING: update information on installing development dependencies

Update instructions on installing development dependencies. Include an example
for Fedora.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>

* tests/lib/prepare-restore: find out what the build deps are and install them

Use rpmspec for fimding out the build dependencies and then proceed with
installing

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>

* tests/lib/prepare-restore: do not pass --with testkeys, comment why

Old rpmspec did not support --with <arg>.

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>

---------

Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
2024-05-29 11:24:25 +02:00
Valentin David
814a8fb903 tests/lib/muinstaller: update snapd dependency 2024-05-28 21:20:19 +02:00