* tests: properly build snapd snap
Now we build also the test version of snapd snap in `snap-builds`
workflow job. We copy this into the spread tests. And we use that
snap, which we only instrument instead of copying the snapd deb build.
If the snap is not available, then we build it in spread. On CI, this
happens on arm since the workflow does not build it. It will also
happen when triggering test manually.
* tests: couple of small improvements to test syntax, move WORK_DIR into script scope, use PWD instead of dot notation
---------
Co-authored-by: Philip Meulengracht <the_meulengracht@hotmail.com>
* tests: fix muinstaller tests for 24
* many: build drivers tree when current mount is not the target mount
In some cases (when using the snapd install API or when installing
from initramfs), the place where the kernel snap / components used for the
installation are mounted is different to the final location in the
installed system. This change considers this so the drivers tree is
generated with symlinks pointing to the final expected location.
* overlord: use model to check if we need to set-up drivers tree
instead of using a device context, as for the installation using snapd
API case we have a model but not a context.
* tests/lib/tools/setup_nested_hybrid_system.sh: re-try kpartx -d
* tests/muinstaller-real: check that drivers tree is created
* tests/muinstaller-real: we need a bigger disk with latest kernel
* tests/lib/tools/setup_nested_hybrid_system.sh: clean up
after building muinstaller. On classic we have weird issues otherwise
due to a desktop agent installing lxd.
* tests/lib/prepare-restore.sh: purge lxd-installer
lxd-installer was causing failures in the restore step for 24.04.
* First list of changes to make nested tests run
* avoid using OVMF with 4M
* Check for cloud init status in uc24
* removed OVMF_SIZE var which is not used anymore
* fix merge issue
* update gadget.yaml for test-snapd-remove-pc-24 snap
* remove mbr and source: shim.efi.signed from gadget.yaml
* restoring source: shim.efi.signed in seed partition
* Fixed the incorrect uuid of the core24 snap
* fixed the snapcraft.yaml for jq-core24
* tests: deal with pre-release suffix in systemd versions
Debian sid is using an rc version of systemd which tests.session and
various other tests weren't dealing with. This change cuts the
pre-release suffix from all of them although some were only used by
distros in which this isn't a problem. This is more consistent and
robust IMO.
Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
* tests: fix var ref
Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
---------
Signed-off-by: Miguel Pires <miguel.pires@canonical.com>
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..f640ac72e3
f640ac72e3 Add missing test details
f0754df304 Filter the error y debug output in log-parser
fc10196efd Add suggestions to details
94ac5ffe58 Add details on tests
501578c719 add more checks in os.query to check is-core_xx
e8929207ff fix os-query for ubuntu comparing with core
226114641f os.query won't check SPREAD_SYSTEM anymore to compare core systems
b89ec98b23 use local variables in os.query tool
dacfd81de9 fix is_core functions
1db5214d5f Improve the remote docs (#36)
2e4a3153a2 1 more comment
3a0fc57e1e add explanation about why we check for ( Do | Doing )
4cf8e635bf fix os.query test after merge
b89b4f8647 fix artifacts name
d30cee6da0 Merge remote-tracking branch 'upstream/main'
5ef5dcbe8f Tests use artifacts in spread tests (#51)
555c43d2ab Support auto-refresh with Do instead of Doing
96c2b0c19c remove tests support for ubuntu 23.04 (EoL)
74082c0c34 Tests improve remote wait (#49)
5121bfb659 remove support for opensuse leap 15.4 (#48)
30df700d08 Add new systems support (#47)
1f08938925 Support check amazon linux version (#46)
43533bdd97 Change the exit value checking for test formats (#45)
3c88244c04 Update check-test-format to support a dir and a list of files (#44)
510d95f429 add extra check for error in auto-refresh detection function
3289d4031b Try open the log with latin-1 encoding when utf-8 is not working
9db785499f improved how the tools are waiting for system reboot
2a5c4414a3 fix shellcheck errors
5e7b63883d Fixes for osquery and tests pkgs (#43)
4c9145e2ac support reboot waiting for auto-refresh
45768f5188 show changes in unknown status after refresh
8013c30c2a Remove support for ubuntu 22.10
b32b80bf54 Fix remote.rait-for test in bionic
5675c625e9 Enable fedora 38
55f4471957 Support for new oss
f2e88b357c New tool used to query spread json reports
cacd35ede0 utils/spread-shellcheck: explain disabled warnings (#42)
c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs
b84eea92e2 spread-shellcheck: fix quotes in environment variables (#41)
ab1e51c29f New comparison in os-query for core systems (#40)
e5ae22a5d4 systemd units can be overwritten
63540b845a Fix error messages in remote pull and push
75e8a426a5 make sure the unit is removed in tests.systemd test
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
162e93bd35 update tests.systemd CLI options to be the same than retry command
14aa43a405 new feature to re-run failed spread tests (#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: f640ac72e316a598138a6b437d41441c7975d916
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from f640ac72e3..b8d20c1d5b
b8d20c1d5b fix snaps.name test with correct siffix spelling
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: b8d20c1d5b874c2f29da2f281b425124fea1f527
* Add more tests with details
* adding one more test
* adding missing tests
* adding more test details
* Update tests/lib/external/snapd-testing-tools/tests/remote.retry/task.yaml
Co-authored-by: Andrew Phelps <136256549+andrewphelpsj@users.noreply.github.com>
* Update tests/lib/tools/suite/tests.invariant/task.yaml
Co-authored-by: Andrew Phelps <136256549+andrewphelpsj@users.noreply.github.com>
* Update task.yaml
* Update task.yaml
* Update task.yaml
---------
Co-authored-by: Andrew Phelps <136256549+andrewphelpsj@users.noreply.github.com>
* enable nested tests for uc24
* fix issue related to cloud init
* some test fixes
* Add missing details for tests
* Adding missing details for nested tests
* addin support for ubuntu-24.04 in upload-snapd-to-gce
* updated the password for external users in spread.yaml
* tests: first set of test fixes for uc24
* add details and 2 more fixes
* fix 2 more tests
* gix user-state test
* fix lp-1813365
* Add missing details
* fix listing test
this changes needs to be done like this until the os.query is-core-* is
fixed
* fix shellcheck
* fix merge saving jobs
* spread.yaml: use lxd edge with shmounts fix
* tests: use snapcraft edge with fix to handle lxd version naming change
* tests: use lxd 5.20/stable with snapcraft 4.x that cannot handle lxd version naming change
* tests: added details for nested/manual/devmode-snaps-can-run-other-snaps
When we build the snapd snap using snapcraft, preseed.json is copied directly
from the source code. Do the same when repacking the deb built from source tree
to the snap.
Signed-off-by: Maciej Borzecki <maciej.borzecki@canonical.com>
* new full-run test
* update tests to support pre-built images
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..74082c0c34
74082c0c34 Tests improve remote wait (#49)
5121bfb659 remove support for opensuse leap 15.4 (#48)
30df700d08 Add new systems support (#47)
1f08938925 Support check amazon linux version (#46)
43533bdd97 Change the exit value checking for test formats (#45)
3c88244c04 Update check-test-format to support a dir and a list of files (#44)
510d95f429 add extra check for error in auto-refresh detection function
3289d4031b Try open the log with latin-1 encoding when utf-8 is not working
9db785499f improved how the tools are waiting for system reboot
2a5c4414a3 fix shellcheck errors
5e7b63883d Fixes for osquery and tests pkgs (#43)
4c9145e2ac support reboot waiting for auto-refresh
45768f5188 show changes in unknown status after refresh
8013c30c2a Remove support for ubuntu 22.10
b32b80bf54 Fix remote.rait-for test in bionic
5675c625e9 Enable fedora 38
55f4471957 Support for new oss
f2e88b357c New tool used to query spread json reports
cacd35ede0 utils/spread-shellcheck: explain disabled warnings (#42)
c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs
b84eea92e2 spread-shellcheck: fix quotes in environment variables (#41)
ab1e51c29f New comparison in os-query for core systems (#40)
e5ae22a5d4 systemd units can be overwritten
63540b845a Fix error messages in remote pull and push
75e8a426a5 make sure the unit is removed in tests.systemd test
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
162e93bd35 update tests.systemd CLI options to be the same than retry command
14aa43a405 new feature to re-run failed spread tests (#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 74082c0c345a7707ca360d2177a23ee958c880bd
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 74082c0c34..5ef5dcbe8f
5ef5dcbe8f Tests use artifacts in spread tests (#51)
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 5ef5dcbe8f376e3958019fffcd54fd6add8117a4
* update args order in new test
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 5ef5dcbe8f..b89b4f8647
b89b4f8647 fix artifacts name
d30cee6da0 Merge remote-tracking branch 'upstream/main'
555c43d2ab Support auto-refresh with Do instead of Doing
96c2b0c19c remove tests support for ubuntu 23.04 (EoL)
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: b89b4f86471b76eae9d5ac1b1f55728598da544e
* improbe checks in nested.sh
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from b89b4f8647..4cf8e635bf
4cf8e635bf fix os.query test after merge
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4cf8e635bf65f974dbcc5b303f890ac9078ed78a
* removing duplicated debian-11
* remove another merge error
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4cf8e635bf..3a0fc57e1e
3a0fc57e1e add explanation about why we check for ( Do | Doing )
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 3a0fc57e1eaf13b189a83982835cf289b3dfad3c
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 3a0fc57e1e..2e4a3153a2
2e4a3153a2 1 more comment
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 2e4a3153a27529092156fbe723cdfef221467bd4
* increase limit waiting for log changes because check to ntp server breaks
* increase time to wait for log changes
* wait more time waiting for changes in the log
It gets stuck more than 60 seconds waiting for ntp sync
* o/snapstatetest, o/devicestate, overlord: add param to InstallSnap and InstallEssentialSnaps for setting up gadget files
* o/devicestate: allow remodeling on hybrid models
When remodeling on a hybrid model, check to make sure that we have a
seed partition before creating a new recovery system as a part of the
remodel. Creating the new system is skipped if we don't see a seed
partition.
* tests/nested/manual/hybrid-remodel: new test for remodeling on a hybrid model
* o/snapstate/snapstatetest: forward gadget files to InstallSnap for gadget
* o/devicestate: add check for if the system is core-boot in undoCreateRecoverySystem
* gadget: add Info.HasRole method that checks for a role in an Info struct
* o/devicestate: replace manual role lookup with gadget.Info.HasRole incovation
* tests/nested/manual/hybrid-remodel: add TODO about testing system seeds on hybrid systems
* tests/nested/manual/hybrid-remodel: properly wait for reboot
* tests/nested/manual/hybrid-remodel: use change_id to make assertions more specific
* o/devicestate: do not attempt to finalize seed install when there is no seed partition
* tests: put muinstaller use into script for use in multiple tests
* o/devicestate: add comment explaining reason for explicitly checking classic systems
* tests: rename muinstaller.sh to setup_nested_hybrid_system.sh to clarify functionality
* tests/lib/tools/setup_nested_hybrid_system.sh: tell shellcheck where to find sourced files
* tests/lib/tools: rename --fake-disk to --disk in setup_nested_hybrid_system.sh
* tests/nested/manual/muinstaller-real: fix missing usage of $label variable in setup_nested_hybrid_system.sh
* tests/nested/manual/muinstaller-real: fix check for pre-existing gpt partition
* tests/nested/manual/muinstaller-real: use proper quoting on path with escaped chars
* tests: use load generator to generate load in perf tests
Also it is included a new test to check snap connections for snap
provided interfaces and another to check many installations in parallel.
* fix shellchecks
* fix misspelling
* update parallel install numbers
* Target the perf tests for ubuntu core >=18
* tests: add new performance and load test suite
This is the initial change to support performance tests.
These new tests have to be able to run either in nested vms and in
external devices.
The idea is to run nightly and on demand on nested vms and also as part
of regular runs on specific devices.
The initial test is about connecting and disconnecting interfaces,
covering a specific percentage of those and also test how the system
behaves when all are connected together.
* fix shellcheck
* fix minimal-smoke test
* add details to minimal-smoke test
* use ubuntu-image snap in xenial
* Fix system-snap-refresh in uc20
Simplify the test and validate the base snap during restore is the same
revision than the initial one (no need to remove).
Also it is included a new function to determine the revision of a snap
and the test is updated accordingly.
* fix shellcheck
* Update tests/core/system-snap-refresh/task.yaml
Co-authored-by: Andrew Phelps <136256549+andrewphelpsj@users.noreply.github.com>
* Fix spread error in MATCH
* make sure not extra output is displayed showing rev
---------
Co-authored-by: Andrew Phelps <136256549+andrewphelpsj@users.noreply.github.com>
Some tests fail before the fake store is configured so the dir
"STORE_DIR" is nor created. When the test is restored, the test fails to
restore because the tool store-state is returning 1 in case the
STORE_DIR dir does not exist.
This change makes more robust the store-state tool to manage this
scenario and continue in case the fake store was not configured
properly.
Before it would result it being added to install mode as well, when we only set up the minimal set of snaps (gadget, kernel, base, snapd). If the validation-set contained any snaps besides these, it would fail. Wait until run mode, where we set up any non-essential snaps, and after this we expect all snaps to be present in system.
* Test fedora-38 and remove fedora-36
* update the number of workers for fedora 38
* update tests.session to support selinux
* tests: fix shell quoting
* disable shellcheck SC2086
this is needed to avoid error:
systemd-run: unrecognized option '--property
SELinuxContext=unconfined_u:unconfined_r:unconfined_t:s0'
* Updated tests.session tool to use --pipe with systemd-run
Also was moved the pipes creation to be done just when bustctl is used
intead of systemd-run
---------
Co-authored-by: Michael Vogt <mvo@ubuntu.com>
