* First list of changes to make nested tests run
* avoid using OVMF with 4M
* Check for cloud init status in uc24
* removed OVMF_SIZE var which is not used anymore
* fix merge issue
* update gadget.yaml for test-snapd-remove-pc-24 snap
* remove mbr and source: shim.efi.signed from gadget.yaml
* restoring source: shim.efi.signed in seed partition
* Fixed the incorrect uuid of the core24 snap
* fixed the snapcraft.yaml for jq-core24
* enable nested tests for uc24
* fix issue related to cloud init
* some test fixes
* Add missing details for tests
* Adding missing details for nested tests
* addin support for ubuntu-24.04 in upload-snapd-to-gce
* updated the password for external users in spread.yaml
* tests: use load generator to generate load in perf tests
Also it is included a new test to check snap connections for snap
provided interfaces and another to check many installations in parallel.
* fix shellchecks
* fix misspelling
* update parallel install numbers
* Target the perf tests for ubuntu core >=18
* tests: add new performance and load test suite
This is the initial change to support performance tests.
These new tests have to be able to run either in nested vms and in
external devices.
The idea is to run nightly and on demand on nested vms and also as part
of regular runs on specific devices.
The initial test is about connecting and disconnecting interfaces,
covering a specific percentage of those and also test how the system
behaves when all are connected together.
* fix shellcheck
* fix minimal-smoke test
* add details to minimal-smoke test
* use ubuntu-image snap in xenial
Before it would result it being added to install mode as well, when we only set up the minimal set of snaps (gadget, kernel, base, snapd). If the validation-set contained any snaps besides these, it would fail. Wait until run mode, where we set up any non-essential snaps, and after this we expect all snaps to be present in system.
This is the first part of the improvements done for nested tests.
It includes:
. new remote.wait-for tool
. several improvements to muinstaller tests
. new secboot-remove-signature command
. other small changes
* Start using remote tools in nested tests
This change introduces the new tools for remote commands from the
snapd-testing-tools project
This change starts using the remote.exec which replaces the tests.nested
exec and the remote.push by the tests.nested copy
The new remote commands have the same implementation then the
tests.nested but those include tests and are being used in other
projects successfully.
* fix inclusive wording
* adding missing placeholder file
* add binaries for remote tools
* Fix how remote tools are linked
* replace nested_exec by remote.exec
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 3b56339b88..fea2bac306
fea2bac306 fix shellcheck
0b5d8cfe51 fix wrong spelling
05fd783a65 Allow to use a custom config file for the remote configuration
b438b3ef03 Skip remote tests on kinetic because no sshpass
f323ee2c9c Add support for ubuntu 22.10 and remove 21.10
6fe0fa0e74 Fix shellcheck in tests.pkgs
a1f7a72ce1 Add disable refresh
507f8bd7c7 improve refresh and wait-for for remote tools
5e7e210e52 Fix refresh_core_base function
e798cb8cca Disable refreshes to make the full refresh
e5a20b9945 initial wait for ssh before checking refreshes
a20c608202 Adding more logs for refresh status and fix error waiting for ssh
f57cd75f09 New default values for wait-for
efdc966b7d More logs in remote refresh
83703b516d Both refreshes unified and more details in logs
32df7bfcc3 Reboot detection for refreshes
cf7fea5f57 Improving remote.wait-for and remote.refresh tools
35b2afcd19 adding more logging for refresh and wait-for remote tools
20547c2347 New remote and systemd tools with tests (#30)
b8eb20d069 Merge pull request #31 from snapcore/run-without-spread
5b8f214d03 Fix spelling
4fe5d64a5c Include a comment in the setup.sh script
03feb2ce2b fix bash error
84e70d2ac5 Make tools work when are not in the spread env
54909c304d Merge pull request #29 from snapcore/udpate-os-query-new-systems
cc1729c2f4 preserve size for amazon-linux
bb3d8db60e preserve size for centos
d1089d5ce3 Fixing the tests
47793a6465 Fix auto-merge problem
f50be4ac5e Merge branch 'main' into udpate-os-query-new-systems
1a40a224c9 os.query tool cli simplified
9b7b4e9bf1 Merge pull request #28 from snapcore/add-centos-9-support
7fe2087423 Add support for centos-9
64a830b933 Merge pull request #27 from snapcore/improve-log-analyzer
7fe27d4aea Improve log analyzer utility
207536268e Merge pull request #19 from snapcore/new-spread-manager
2f2ff2e282 Update spread manager to support csv
33a44ca3be Merge branch 'main' into new-spread-manager
b4654950d4 Merge pull request #26 from snapcore/support-csv-for-expressions
3250bbd885 Support expressions with comma separation
2540135b90 Merge pull request #25 from snapcore/add-indent-to-log-parser
2536b0f070 Minor improvements in log-parser and log-analyzer based con review comments
84dc8092b1 Merge pull request #24 from snapcore/improve-log-parser
515770b3bf Add support for fedora-35
875c29b5ce Updated results with latest log-parser changes
d27f2bcdb7 Fix log-parser
b2cce1fcce fix wording
14d15e4fe4 Fixes for log-parser and changes for log analyzer
438d92d241 Log analyzer updated to support reexecute in all the scenarios
1c9dff58ff Merge pull request #23 from snapcore/improve-log-parser
cc7ee488d1 Fix shellcheck
324b99e719 revert change in log-analyzer test
f746f40ebe Fix shellcheck
2d7dbbe1bd Fix spelling
728dd64c2c Last set of changes for log analizer tools
bf389dcd01 New fixes for log parses
6b2b56afc3 Fix another shellcheck
56163e170b Fix shellcheck
d96ab8094f Merge branch 'main' into new-spread-manager
60fb99f02f new dir task5
259a7e188c Fix spread test
e674234454 New spread-manager tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: fea2bac306a463f4c97f79a6e934e85ec00e804c
* Using a custom remote configuration file
This is to avoid issues when the helpers try to use the remote tools
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from fea2bac306..4a994e96b3
4a994e96b3 Fix new config path in tests
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4a994e96b33d5966053645ec87563d0ea8fd019b
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4a994e96b3..53e6c8f315
53e6c8f315 Use the new ubuntu kinetic image
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 53e6c8f315fbdd379b59d6458eb869e68f34c9b3
* fix tests with wrong use of the new tools
* Update test with remote.push command
This test seems to be updated during last merge.
* Update sprad.yaml which fails on debug
* Update the number of workers for nested tests on uc20 and uc22
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 53e6c8f315..4d6088ec8c
4d6088ec8c Fix shellcheck in spread-manager util
edb4f3f51b fix shell check errors in remote tools
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4d6088ec8c5d43e7b4d45faf904978175c418120
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4d6088ec8c..f46dccb464
f46dccb464 Fix sshpass command in remote.exec
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: f46dccb464f3a82932c0a89ecabffb6ef84ec6df
* gadget/device: introduce package which provides helpers for locations of things
Various helpers for getting location of things were spread out or doubled in the
code base. This package aggregates them in one place.
Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
* boot,secboot: reset DA lockout counter after successful boot
The TPM DictionaryAttack lockout counter is incremented after each
unclean shutdown according to Chris Coulson in LP:1979185.
This means that eventually enough unclean shutdowns the system
fails to boot and asks for the recovery key. To fix this snapd
needs to clean the DA lockout counter after each successful
run.
* device: add missing doc string for TpmLockoutAuthUnder
* secboot: add missing tests around MarkSuccessful
* tests: add nested core20-da-lockout test
This commits adds a nested core20 test that triggers a bunch of
unclean shutdowns in a nested VM. On a system that does not have
the new `secboot.MarkSuccessful()` code this will eventually
trigger a recovery key prompt as each unclean reboot increases
the DA lockout counter by one.
```
google-nested:ubuntu-20.04-64 /tmp/work-dir/logs# tail -n1 /tmp/work-dir/logs/serial.log
Please enter the recovery key for disk /dev/disk/by-partuuid/3a1bacae-5d46-ce4b-960a-4074d18a8c05: (press TAB for no echo)
```
* devicestate: fix go fmt error
* tests: set nested/manual/core20-da-lockout to manual and improve comment/description
* secboot: improve docstrings
* secboot: shuffle code around so that tests with `-tags nosecboot` work correctly
* boot,device: extract StampSealedKeys,SealedKeysMethod from boot to device
* secboot: use new device.SealedKeysMethod()
* tests: make core20-da-lockout faster by reading the da lockout counter directly (thanks to Chris Coulson)
* tests: add new 'test.nested vm unclean-reboot` and use in da lockout test
* tests: retry on lockout counter reading
The lockout counter is cleared during snapd startup so we need
to retry for a bit to see it cleared.
* tests: fix whitespace
* secboot: improve test around testMarkSuccessfulEncrypted and also check authValue
* secboot: fix silly typo (thanks to Samuele)
Co-authored-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
* New set of nested tests for uc22
This change includes:
. Updates for the tests to support uc22
. New models for uc22
. New snap needed to run uc22 tests
* New support for system user asserts
Still missing to sign the assert for uc22
* fix shellcheck
* Fixing style shellchecks
* Fix shell check on tests
* New fix for shell check
* Created the auto-import.assert for developer1
* Generate the auto-import.assert for the developer1
* Fixed error on tests
* Fix interfaces-custom-devices test
* Update variable names
* Fix the nested remodel tests
Currently the test is failing in core 18 because the core18 snap is
being autore-freshed breaking the connection with the host machine.
The idea of the change is to use the tests.nested tool which is widely
used on nested tests and repacks the core18 snap avoiding
auto-refreshes.
This change also includes a new wait-for event used to wait until the
device is initialized and has a serial. By doing that it is not needed
anymore to source the nested.sh lib.
* Change call to nested_get_boot_id by tests.nested boot-id
* Same change on boot id for test remodel-cross-store
* Update to use the use the gadget snap from the store
* Minor fixes to use the env vars to download the gadget and kernel snaps
* Allow skip repack for kernel, base and gadget
This is needed to make remodel tests work. This is also neede for other
tests to be able to use kernel and gadget from the store
* Improve cleanup of tests
* Removing snaps installed in other tests
This is the continuation of the nested tool changes:
1. Most of the tested tests are not using the nested helper anymore
2. Many new functinos added to the nested tool
3. Loops updated to use retry instead
4. Small fixes
The change includes:
. Many functions added to tests.nested tool
. is-nested
. get
. boot-id
. secboot-sign
. Restructured the tests.nested tool command to simplify the cli
. exec
. vm (before was start-vm, stop-vm, and remove-vm)
The tests were update to use the tests.nested tool instead of the
nested.sh helper.
Next step is to reorganize the helper in 2 parts.
This are the second part of the commands for the tests.nested tool
The new commands are:
copy: copies a file to the vm
wait-for: waits for an specified event (ssh. no-ssh,
snap-command and reboot)
Tests have been updated as well
