* Start using remote tools in nested tests
This change introduces the new tools for remote commands from the
snapd-testing-tools project
This change starts using the remote.exec which replaces the tests.nested
exec and the remote.push by the tests.nested copy
The new remote commands have the same implementation then the
tests.nested but those include tests and are being used in other
projects successfully.
* fix inclusive wording
* adding missing placeholder file
* add binaries for remote tools
* Fix how remote tools are linked
* replace nested_exec by remote.exec
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 3b56339b88..fea2bac306
fea2bac306 fix shellcheck
0b5d8cfe51 fix wrong spelling
05fd783a65 Allow to use a custom config file for the remote configuration
b438b3ef03 Skip remote tests on kinetic because no sshpass
f323ee2c9c Add support for ubuntu 22.10 and remove 21.10
6fe0fa0e74 Fix shellcheck in tests.pkgs
a1f7a72ce1 Add disable refresh
507f8bd7c7 improve refresh and wait-for for remote tools
5e7e210e52 Fix refresh_core_base function
e798cb8cca Disable refreshes to make the full refresh
e5a20b9945 initial wait for ssh before checking refreshes
a20c608202 Adding more logs for refresh status and fix error waiting for ssh
f57cd75f09 New default values for wait-for
efdc966b7d More logs in remote refresh
83703b516d Both refreshes unified and more details in logs
32df7bfcc3 Reboot detection for refreshes
cf7fea5f57 Improving remote.wait-for and remote.refresh tools
35b2afcd19 adding more logging for refresh and wait-for remote tools
20547c2347 New remote and systemd tools with tests (#30)
b8eb20d069 Merge pull request #31 from snapcore/run-without-spread
5b8f214d03 Fix spelling
4fe5d64a5c Include a comment in the setup.sh script
03feb2ce2b fix bash error
84e70d2ac5 Make tools work when are not in the spread env
54909c304d Merge pull request #29 from snapcore/udpate-os-query-new-systems
cc1729c2f4 preserve size for amazon-linux
bb3d8db60e preserve size for centos
d1089d5ce3 Fixing the tests
47793a6465 Fix auto-merge problem
f50be4ac5e Merge branch 'main' into udpate-os-query-new-systems
1a40a224c9 os.query tool cli simplified
9b7b4e9bf1 Merge pull request #28 from snapcore/add-centos-9-support
7fe2087423 Add support for centos-9
64a830b933 Merge pull request #27 from snapcore/improve-log-analyzer
7fe27d4aea Improve log analyzer utility
207536268e Merge pull request #19 from snapcore/new-spread-manager
2f2ff2e282 Update spread manager to support csv
33a44ca3be Merge branch 'main' into new-spread-manager
b4654950d4 Merge pull request #26 from snapcore/support-csv-for-expressions
3250bbd885 Support expressions with comma separation
2540135b90 Merge pull request #25 from snapcore/add-indent-to-log-parser
2536b0f070 Minor improvements in log-parser and log-analyzer based con review comments
84dc8092b1 Merge pull request #24 from snapcore/improve-log-parser
515770b3bf Add support for fedora-35
875c29b5ce Updated results with latest log-parser changes
d27f2bcdb7 Fix log-parser
b2cce1fcce fix wording
14d15e4fe4 Fixes for log-parser and changes for log analyzer
438d92d241 Log analyzer updated to support reexecute in all the scenarios
1c9dff58ff Merge pull request #23 from snapcore/improve-log-parser
cc7ee488d1 Fix shellcheck
324b99e719 revert change in log-analyzer test
f746f40ebe Fix shellcheck
2d7dbbe1bd Fix spelling
728dd64c2c Last set of changes for log analizer tools
bf389dcd01 New fixes for log parses
6b2b56afc3 Fix another shellcheck
56163e170b Fix shellcheck
d96ab8094f Merge branch 'main' into new-spread-manager
60fb99f02f new dir task5
259a7e188c Fix spread test
e674234454 New spread-manager tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: fea2bac306a463f4c97f79a6e934e85ec00e804c
* Using a custom remote configuration file
This is to avoid issues when the helpers try to use the remote tools
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from fea2bac306..4a994e96b3
4a994e96b3 Fix new config path in tests
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4a994e96b33d5966053645ec87563d0ea8fd019b
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4a994e96b3..53e6c8f315
53e6c8f315 Use the new ubuntu kinetic image
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 53e6c8f315fbdd379b59d6458eb869e68f34c9b3
* fix tests with wrong use of the new tools
* Update test with remote.push command
This test seems to be updated during last merge.
* Update sprad.yaml which fails on debug
* Update the number of workers for nested tests on uc20 and uc22
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 53e6c8f315..4d6088ec8c
4d6088ec8c Fix shellcheck in spread-manager util
edb4f3f51b fix shell check errors in remote tools
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4d6088ec8c5d43e7b4d45faf904978175c418120
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4d6088ec8c..f46dccb464
f46dccb464 Fix sshpass command in remote.exec
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: f46dccb464f3a82932c0a89ecabffb6ef84ec6df
* Nested test for core20 preseeding using ubuntu-image.
* Disable some shellcheck errors.
* Exclude the new test from shellcheck.
* Fix section order.
* Fix unbound variable error.
* Remove snap/.gnupg in restore.
* Restore connections-after-failed-refresh test.
* Fix typo
Co-authored-by: Maciej Borzecki <maciek.borzecki@gmail.com>
* Remove newline.
* Point at task.yaml.
* Fix shellcheck exclude.
* Fix exclude.
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 53ca1b3b70..3b56339b88
3b56339b88 Merge pull request #22 from snapcore/tests-fix-spread-shellcheck
488408d792 fix shellcheck test
b9340d265c Fix spread-shellcheck and update the test with new scenario
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 3b56339b88040d0b5b7abaf7ca6dbf33936e3cec
* Improve nested.sh (thanks mardy!).
* Disable SC2191
Co-authored-by: Maciej Borzecki <maciek.borzecki@gmail.com>
* Allow run failed tests on google-nested backend
This change is to allow running just the failed tests for google-nested
backend when the "Run failed" tag is set
* Make sure no new lines in results
* fix core22-basic to deal with recovery-keys change
* filter by backend to make sure external subtests are not included
* Fix core22-basic disabling recovery keys checks
* Fix recovery keys for core20-basic test now
* Update the key for nested cache
It's --show-keys, so the test was not really exercising the API operation as a
non-root user.
Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
The change includes:
. Many functions added to tests.nested tool
. is-nested
. get
. boot-id
. secboot-sign
. Restructured the tests.nested tool command to simplify the cli
. exec
. vm (before was start-vm, stop-vm, and remove-vm)
The tests were update to use the tests.nested tool instead of the
nested.sh helper.
Next step is to reorganize the helper in 2 parts.
* devicestate: add support to save timings from install mode
Using a quick and not super nice way to store debug timings of
the install to the ubuntu-data partition.
* devicestate: rework timing log writing (thanks to Samuele)
* devicestate: rework because we can not call into snapd when snapd tries to reboot
* devicestate: improve unit test
* devicestate: snap debug timings --ensure=seed needs to run on its own
* devicestate: write the logs *after* the timings so that any failure from the timing writing is captured
* devicestate: fix comment
* tests: remove silly debug output
* tests: test install-timings.txt.gz more
* devicestate: add extra \n after each debug change output
* devicestate: move state unlock into writeTimings() and skip the "seed" change, we get the info via "snap debug timings --ensure=seed"
* devicestate: add clear information what is happening in writeTimings (thanks to Maciej)
* devicestate: update test to latest code changes
* tests: remove stray newline
This change includes:
. a new nested_retry command for nested tests
. tests updates to start using the new command intead of a loop
. replaced the existent retry functions using the new retry command
The change updates the layout of the tests following a proposed order
which should make the test easier to read and understand
The proposed order is:
summary
details
backends
systems
manual
priority
warn-timeout
kill-timeout
environment
prepare
restore
debug
execute
This change includes:
. core20 and core nested suites are unified into core suite. This
avoids tests duplication
. Now the tpm, secboot and kvm variables have defaults which are used
through functions defined in nested.sh
. By default the uc20 is build with secboot and tpm enabled.
. By detauld uc16 and uc18 are not using neither secboot nor tpm
. By default kvm is always used
. To define a different behaviour, the test needs to define the
variable in its environment.
