* tests: properly build snapd snap
Now we build also the test version of snapd snap in `snap-builds`
workflow job. We copy this into the spread tests. And we use that
snap, which we only instrument instead of copying the snapd deb build.
If the snap is not available, then we build it in spread. On CI, this
happens on arm since the workflow does not build it. It will also
happen when triggering test manually.
* tests: couple of small improvements to test syntax, move WORK_DIR into script scope, use PWD instead of dot notation
---------
Co-authored-by: Philip Meulengracht <the_meulengracht@hotmail.com>
* update gadget.yaml for nested remodel tests
* add new snaps with base: core24
* tests: fixes for uc24 nested tests
This change contains:
. fixes for model
. use of core24 snaps (jq, rsync and sh)
. test fixes
. using legacy scp in snapd-testing-tools remote-pull (to avoid errors)
* use -O when the scp version is -ge than 9
* re-enable hybrid-remodel test on noble
* update wording of remote.pull
* Adding more workers for nested tests
This is to avoid instances are cleaned before the execution finishes
* tests: fix muinstaller tests for 24
* many: build drivers tree when current mount is not the target mount
In some cases (when using the snapd install API or when installing
from initramfs), the place where the kernel snap / components used for the
installation are mounted is different to the final location in the
installed system. This change considers this so the drivers tree is
generated with symlinks pointing to the final expected location.
* overlord: use model to check if we need to set-up drivers tree
instead of using a device context, as for the installation using snapd
API case we have a model but not a context.
* tests/lib/tools/setup_nested_hybrid_system.sh: re-try kpartx -d
* tests/muinstaller-real: check that drivers tree is created
* tests/muinstaller-real: we need a bigger disk with latest kernel
* tests/lib/tools/setup_nested_hybrid_system.sh: clean up
after building muinstaller. On classic we have weird issues otherwise
due to a desktop agent installing lxd.
* tests/lib/prepare-restore.sh: purge lxd-installer
lxd-installer was causing failures in the restore step for 24.04.
* First list of changes to make nested tests run
* avoid using OVMF with 4M
* Check for cloud init status in uc24
* removed OVMF_SIZE var which is not used anymore
* fix merge issue
* update gadget.yaml for test-snapd-remove-pc-24 snap
* remove mbr and source: shim.efi.signed from gadget.yaml
* restoring source: shim.efi.signed in seed partition
* Fixed the incorrect uuid of the core24 snap
* fixed the snapcraft.yaml for jq-core24
/etc/timezone is an old file that was only supported on Debian and
Ubuntu but has been removed in 24.04 and upcoming Debian.
Whether /etc/timezone works or not is a problem for systemd. We do not
have to test it.
* boot: added function to set EFI variables
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: renamed trustedShimFallbackBinary to seedShimPath
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: refactored setting EFI boot variables at install
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: adjusted variable names and fixed variable initialization
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: improve setting Boot#### EFI variable
Notably, splits off the process of reading a Boot#### variable and
extracting its DevicePath into its own function `readBootVariable` which
can be mocked and otherwise simplifies the `setBootNumberVariable`
function.
Also, fixes behavior around the final BootFFFF variable. Previously, it
was not possible to select the BootFFFF variable if it was unused, due
to overflow concerns on uint16. Now, the behavior around BootFFFF is
identical to that of any other boot variable, by using an int internally
instead of uint16, which also allows a more robust check for whether
there were no matching variables.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: added unit tests for setting EFI Boot#### variable
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: refactored setting EFI boot variables
Rewrote EFI boot variable functions to more closely match the behavior
of shim fallback: https://github.com/rhboot/shim/blob/main/fallback.c
In particular, the following have changed:
1. Existing Boot#### variables must fully match the new load option to
be considered a match. In particular, the load option attributes,
label, and device path must all be byte-for-byte identical.
Previously, only the device paths were compared.
2. Matching Boot#### variables are no longer overwritten. Since the
variable data must now byte-for-byte match the new load option, there
is no need to overwrite the existing variable.
3. Since existing Boot#### variables are no longer overwritten, the
variable attributes are no longer checked for those variables.
Instead, it is assumed that the Boot#### variable attributes are
viable for it to be used as a boot option. This matches the behavior
of `rhboot/shim/fallback.c`, for better or for worse.
4. When modifying the BootOrder variable, boot option numbers are no
longer pruned if there is no matching Boot#### variable.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot,bootloader: introduce UefiBootloader to build EFI load options
Previously, the path of the shim binary relative to the EFI partition
was passed into `SetEfiBootVariables`. However, different bootloaders
may wish to set up `OptionalData` in the load option.
Additionally, not all `TrustedAssetBootloaders` will attempt to set
EFI boot variables, and not all bootloaders which should set EFI boot
variables necessarily support secure boot. Thus, these should be
decoupled.
This commit adds a new `UefiBootloader` interface with the
`ConstructShimEfiLoadOption` method, which builds an EFI load option
from the shim path for the given bootloader.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot,bootloader: fixed linting errors and improved EFI boot variable test clarity
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
bootloader: improved unit test for grub EFI load option creation
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: set EFI boot variables in `MakeRunnableSystem`
Previously, attempted to set boot variables in
`MakeRecoverySystemBootable`, which is called by `MakeBootableImage`,
which is called when building the image file, rather than during install
mode.
`MakeRunnableSystem` is called on first boot during install mode, and
thus should be responsible for setting EFI boot variables.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: use seed bootloader when setting EFI variables
In install mode, the bootloader located in ubuntu-seed should be used
when setting the EFI boot variables. Previously, the bootloader in
ubuntu-boot was accidentally re-used.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: added simple test to execute setefibootvar.go code
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: fixed standalone set EFI vars code test to work with different layouts
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: moved simple setefibootvar.go check to nested test
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: added check for idempotence when setting EFI boot variables
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
bootloader: adjust comments, organization, and add TODO
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot,bootloader: fix setting EFI boot variables
Make function to search for EFI asset device path and construct load
option common so each UefiBootloader does not have to re-implement it.
Instead, the bootloader returns the description, asset file path, and
optional data, which can then be used to create the EFI load option.
Also, in `makeRunnableSystem`, the bootloader in ubuntu-seed must have
`NoSlashBoot` in order to correctly find the grub.cfg file and thus the
grub bootloader. This commit fixes this bug, and refactors a bit to
account for the changes in responsibilities between the bootloader and
the setefibootvars.go code.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
bootloader: fixed grub EFI load option test with tmp rootdir
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
go.mod: move golang.org/x/text import next to other golang.org/x/ imports
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: adjust opts to look for recovery bootloader when setting EFI variables
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: do not overwrite BootOrder if unchanged, and unexport EFI variable helper functions
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: unexport `setEfiBootOrderVariable`
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: move code to detect bootloader and set EFI variables accordingly into dedicated function
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: unexport `setUbuntuSeedEfiBootVariables` and accompanying error
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot,bootloader: ensure nil optionalData for EFI variable is equivalent to 0-length slice
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: handle empty boot order and other boot var improvements
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
boot: make setefibootvars functions linux-only
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
* tests: add nested spread test for setting EFI vars
The test checks that EFI boot variables exist for the following:
1. A Boot#### variable pointing to the shim file path.
2. A BootOrder variable with the #### from the above Boot#### as first.
Since the layout of EFI assets is dependent on the gadget snap, the test
downloads and unpacks the gadget, then modifies the contents so that one
variant has the shim and grub binaries in `EFI/boot/` and another
variant has the shim and grub binaries in `EFI/ubuntu/` and the fallback
binary in `EFI/boot/`.
After building a core image around that modified gadget, the VM is
booted and the test checks that the EFI variables are set correctly.
Then, the test modifies the gadget to match the other variant's initial
layout, and then installs the newly modified gadget. This should trigger
re-setting EFI boot variables as well.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: fix problems in spread test for setting EFI boot variables
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: disabled TPM on EFI boot vars test and separated gadget script
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: fixed EFI vars test to use correct toolbox and include all EFI assets
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: modify-gadget.sh re-use existing gadget so edition is incremented
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: fix mangled EFI var search string and other improvements
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
tests: polish tests for setting EFI boot variables
Notably, allow tests/nested/core/core20-set-efi-boot-variables to run on
arm64 as well as amd64, simplify setefivars.go to search for multiple
assets on multiple architectures, and allow
tests/nested/manual/core20-set-efi-boot-vars to run on any ubuntu-2*.
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
* bootloader/grub.go: only consider new shim asset in boot entry for now
* tests/nested/core/core20-set-efi-boot-variables: fix details
* boot: update uefi variables on gadget update
* tests/nested/manual/core20-set-efi-boot-vars: work-around file not deleted
* tests/nested/manual/core20-set-efi-boot-vars: use fb.efi like other tests
* tests/nested/manual/core20-set-efi-boot-vars: drop use of toolbox snap
* tests/nested/manual/core20-set-efi-boot-vars: drop work-around for not deleted files
* tests/nested/manual/core20-set-efi-boot-vars: verify install does add a boot entry
* tests/nested/manual/core20-set-efi-boot-vars: run only on version that have UC
* tests/nested/manual/core20-set-efi-boot-vars: obey GADGET_CHANNEL
* tests/nested/manual/core20-set-efi-boot-vars: move get_boot_entry.py to libs
* tests/nested/manual/core20-set-efi-boot-vars: factorize copy of variables
... so we can reuse the script in other tests
* tests/nested/core/core20-set-efi-boot-variables: stop using toolbox snap
* tests/nested/core/core20-set-efi-boot-variables: only run on versions with UC available
* overlord/devicestate: test using EfiLoadOptionParameters
* boot: test that variables are set
* boot: test observers' UpdateBootEntry
* tests/nested/manual/core20-set-efi-boot-vars: also test without secure boot
* many: use trusted install observer when UEFI variables are supported
* boot/makebootable.go: rename sealer to observer
* boot/grub.go: fix function name in doc
* cmd/snap-bootstrap: verify that ObserveExistingTrustedRecoveryAssets is called
* boot: add tests for SetEfiBootVariables
* many: comment on calls to ObserveExistingTrustedRecoveryAssets
---------
Signed-off-by: Oliver Calder <oliver.calder@canonical.com>
Co-authored-by: Oliver Calder <oliver.calder@canonical.com>
Because we need /snap/snapd/current symlink for the snapd snap to work
when seeded, we need to update the test to use an up to date base.
We cannot use an old core22 snap with a new snapd snap.
* tests: test refresh mixing essential and apps in hybrid systems
Check that on a hybrid system, refreshes of both essential snaps and
apps and non-model bases don't make the apps wait for a reboot. In
particular check that the refresh of the kernel, gadget and model base
wait for a reboot. Check that apps that depend on the model base also
wait for it to be refreshed and that apps that don't can completely
refresh before the reboot.
Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
* tests: use correct model for system in muinstaller-real
Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
---------
Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
Build drivers tree on installation so kernel modules can be found on first boot after installation (otherwise the modules would be available too late, after initial installation of the kernel has happened while seeding).
Also, make sure that the tree is not re-created when seeding, and add a spread test for kernel-modules components.
* dirs: add function to get root of drivers trees
* kernel: do not recreate drivers tree if it already exists
On installation, a drivers tree has already been copied to the data
partition. Do not re-create if found when seeding happens (note though
that on preseeding we will not find the tree and we will still create
it).
* gadget/install: create device tree on installations
Create a device tree on installation so we have it on first boot.
* o/snapstate: fix task message
* tests/nested/manual: add kernel-modules-components test
This test installs a kernel-modules component and verifies that the
shipped kernel module is installed and can be loaded.
* tests/lib: do not error out if cloud-init finished with
'recoverable error' status.
* kernel: have destination as argument when building drivers tree
We need to tackle the generic installation case as opposed to creating
the drivers tree when seeding the system. In that case the destination
tree can vary depending on installation type.
* overlord: pass kernel information to install package
so the drivers tree can be created.
* cmd/snap-bootstrap: build drivers tree on installation from initramfs
* tests: adapt to additional parameters in install.Run
* snap/naming: add ancillary method to split a full component name
into snap and component names.
* gadget/install: fix nosecboot tests
In order to build from 22.04, we provide all runtime libraries in the
resulting snap. The glibc dynamic linker (ld.so) is rebuilt to only
load libraries from the snap at /snap/snapd/current, and not from the
host system. All ELF binaries have their interpreter path patched to
use that dynamic linker.
This stops using the build from dpkg. Binaries that are not used from
the snapd snap have also been removed.
Snapcraft 8.x is expected now to build the snap.
* enable nested tests for uc24
* fix issue related to cloud init
* some test fixes
* Add missing details for tests
* Adding missing details for nested tests
* addin support for ubuntu-24.04 in upload-snapd-to-gce
* updated the password for external users in spread.yaml
When a gadget is installed, we compare the previous gadget to detect
removed files and remove them. Preserved files that are removed from
content list are kept.
This is important to be able to remove old boot firmware since some,
like shim, change behavior depending of files that are present.
This does not support directories. If the gadget install directories
with content, the files in the directories will not be removed.
* fix run-tests test to allow to run multiple tests on custom variant
Ths is needed to re-exec failed tests on beta validation
* add acho to show tests to be executed
* fix shellcheck disable=SC2086
* add stderr to spread.log
* tests:new perf test to install many snaps with --no-wait and check changes
Install different snaps many times based on arch availability. In this
scenario all the snaps are installed in parallel and the test checks
the system can handel all the installs and remains active.
* performance tests reorganization
To avoid executing perf tests when the nested tests are executed, the
tests have been moved under tests/perf
tests/perf/main -> suite with the tests executed in the target device
tests/perf/nested -> suite with the tests executed in a nested
environment
* Update tests/perf/nested/install-many-snaps-no-wait/task.yaml
Co-authored-by: Miguel Pires <miguelpires94@gmail.com>
* Update tests/perf/nested/install-many-snaps-no-wait/task.yaml
Co-authored-by: Miguel Pires <miguelpires94@gmail.com>
* Update tests/perf/main/install-many-snaps-no-wait/task.yaml
Co-authored-by: Miguel Pires <miguelpires94@gmail.com>
* improve logs check and fixes based on review comments
* addressing review comments
---------
Co-authored-by: Miguel Pires <miguelpires94@gmail.com>
