* new full-run test
* update tests to support pre-built images
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..74082c0c34
74082c0c34 Tests improve remote wait (#49)
5121bfb659 remove support for opensuse leap 15.4 (#48)
30df700d08 Add new systems support (#47)
1f08938925 Support check amazon linux version (#46)
43533bdd97 Change the exit value checking for test formats (#45)
3c88244c04 Update check-test-format to support a dir and a list of files (#44)
510d95f429 add extra check for error in auto-refresh detection function
3289d4031b Try open the log with latin-1 encoding when utf-8 is not working
9db785499f improved how the tools are waiting for system reboot
2a5c4414a3 fix shellcheck errors
5e7b63883d Fixes for osquery and tests pkgs (#43)
4c9145e2ac support reboot waiting for auto-refresh
45768f5188 show changes in unknown status after refresh
8013c30c2a Remove support for ubuntu 22.10
b32b80bf54 Fix remote.rait-for test in bionic
5675c625e9 Enable fedora 38
55f4471957 Support for new oss
f2e88b357c New tool used to query spread json reports
cacd35ede0 utils/spread-shellcheck: explain disabled warnings (#42)
c82afb2dee Support --no-install-recommends parameter when installing dependencies with tests.pkgs
b84eea92e2 spread-shellcheck: fix quotes in environment variables (#41)
ab1e51c29f New comparison in os-query for core systems (#40)
e5ae22a5d4 systemd units can be overwritten
63540b845a Fix error messages in remote pull and push
75e8a426a5 make sure the unit is removed in tests.systemd test
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
162e93bd35 update tests.systemd CLI options to be the same than retry command
14aa43a405 new feature to re-run failed spread tests (#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 74082c0c345a7707ca360d2177a23ee958c880bd
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 74082c0c34..5ef5dcbe8f
5ef5dcbe8f Tests use artifacts in spread tests (#51)
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 5ef5dcbe8f376e3958019fffcd54fd6add8117a4
* update args order in new test
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 5ef5dcbe8f..b89b4f8647
b89b4f8647 fix artifacts name
d30cee6da0 Merge remote-tracking branch 'upstream/main'
555c43d2ab Support auto-refresh with Do instead of Doing
96c2b0c19c remove tests support for ubuntu 23.04 (EoL)
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: b89b4f86471b76eae9d5ac1b1f55728598da544e
* improbe checks in nested.sh
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from b89b4f8647..4cf8e635bf
4cf8e635bf fix os.query test after merge
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4cf8e635bf65f974dbcc5b303f890ac9078ed78a
* removing duplicated debian-11
* remove another merge error
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4cf8e635bf..3a0fc57e1e
3a0fc57e1e add explanation about why we check for ( Do | Doing )
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 3a0fc57e1eaf13b189a83982835cf289b3dfad3c
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 3a0fc57e1e..2e4a3153a2
2e4a3153a2 1 more comment
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 2e4a3153a27529092156fbe723cdfef221467bd4
* increase limit waiting for log changes because check to ntp server breaks
* increase time to wait for log changes
* wait more time waiting for changes in the log
It gets stuck more than 60 seconds waiting for ntp sync
* New tool to manage initial env in spread tests
This is used to know in spread tests the initial value for the vars and
be able to repeat not needed steps
* test fixed
* Minor fixes con docs and env var removed
* improve tests.env tool
* Repleace the systemd.sh helper by the tests.systemd tool in tests/lib
This is the first part of the migration to start using the tests.systemd
tool instead of the systemd.sh helper
* Fix shellcheck issues
* move function to get active snapd units to state.sh
It is the only place where the function is used
* Update missing functions in nested.sh
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..14aa43a405
14aa43a405 new feature to re-run failed spread tests (#39)
604cb782db Fix shellcheck in systemd tool
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 14aa43a405f344b056db4fec7aee78709c41959a
* use -n for tests.systemd wait-for-service
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 14aa43a405..162e93bd35
162e93bd35 update tests.systemd CLI options to be the same than retry command
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 162e93bd356f202b68408d0391b34e93a4f98755
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 162e93bd35..01a2a83b4b
01a2a83b4b Update tests.systemd to have stop units as systemd.sh
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 01a2a83b4b58262e13ca2578ac286a41b780fa7d
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 01a2a83b4b..9089ff5c02
9089ff5c02 Update tests to use the new tests.systemd stop-unit
44ecd5e56a Move tests.systemd stop-units to stop-unit
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 9089ff5c02a1cf074a05fb7c58af93274f63a446
* Use the new tests.systemd stop-unit command
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 9089ff5c02..75e8a426a5
75e8a426a5 make sure the unit is removed in tests.systemd test
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 75e8a426a59fb02ee74d5bb7c60c19403c9146a0
* Fix error in test muinstaller-real
the vm needs to be removed before creating a new one
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 75e8a426a5..63540b845a
63540b845a Fix error messages in remote pull and push
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 63540b845a542ee0a71257e0a726a618201ebb5d
* update how stop is used in muinstaller-real nested test
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 63540b845a..e5ae22a5d4
e5ae22a5d4 systemd units can be overwritten
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: e5ae22a5d42b55d6b33c3f692968e72c62a619dc
* Remove quiet helper
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1c8efb77e1..8a2d0a99df
8a2d0a99df Adding quiet tool and removing set +-x from tests.pkgs
d90935d2a4 A comment explaining about the default values for wait-for
3232c5dba7 Add support for ubuntu 23.04
a7164fba07 remove fedora 35 support, add fedora 37 support
89b9eb5301 Update systems supported
92bb6a0664 Include snap-sufix in the snaps.name tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 8a2d0a99df94f086f4f40b79471b35443c0aff5f
* Adding missing binary
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 8a2d0a99df..bfc71082c8
bfc71082c8 Update the tests.systemd to allow parameters waiting for service status
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: bfc71082c85c10f8940ee4def18f576f1ed273ca
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from bfc71082c8..604cb782db
604cb782db Fix shellcheck in systemd tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 604cb782db444110108cd212f00aef0050e3a5a2
* Start using remote tools in nested tests
This change introduces the new tools for remote commands from the
snapd-testing-tools project
This change starts using the remote.exec which replaces the tests.nested
exec and the remote.push by the tests.nested copy
The new remote commands have the same implementation then the
tests.nested but those include tests and are being used in other
projects successfully.
* fix inclusive wording
* adding missing placeholder file
* add binaries for remote tools
* Fix how remote tools are linked
* replace nested_exec by remote.exec
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 3b56339b88..fea2bac306
fea2bac306 fix shellcheck
0b5d8cfe51 fix wrong spelling
05fd783a65 Allow to use a custom config file for the remote configuration
b438b3ef03 Skip remote tests on kinetic because no sshpass
f323ee2c9c Add support for ubuntu 22.10 and remove 21.10
6fe0fa0e74 Fix shellcheck in tests.pkgs
a1f7a72ce1 Add disable refresh
507f8bd7c7 improve refresh and wait-for for remote tools
5e7e210e52 Fix refresh_core_base function
e798cb8cca Disable refreshes to make the full refresh
e5a20b9945 initial wait for ssh before checking refreshes
a20c608202 Adding more logs for refresh status and fix error waiting for ssh
f57cd75f09 New default values for wait-for
efdc966b7d More logs in remote refresh
83703b516d Both refreshes unified and more details in logs
32df7bfcc3 Reboot detection for refreshes
cf7fea5f57 Improving remote.wait-for and remote.refresh tools
35b2afcd19 adding more logging for refresh and wait-for remote tools
20547c2347 New remote and systemd tools with tests (#30)
b8eb20d069 Merge pull request #31 from snapcore/run-without-spread
5b8f214d03 Fix spelling
4fe5d64a5c Include a comment in the setup.sh script
03feb2ce2b fix bash error
84e70d2ac5 Make tools work when are not in the spread env
54909c304d Merge pull request #29 from snapcore/udpate-os-query-new-systems
cc1729c2f4 preserve size for amazon-linux
bb3d8db60e preserve size for centos
d1089d5ce3 Fixing the tests
47793a6465 Fix auto-merge problem
f50be4ac5e Merge branch 'main' into udpate-os-query-new-systems
1a40a224c9 os.query tool cli simplified
9b7b4e9bf1 Merge pull request #28 from snapcore/add-centos-9-support
7fe2087423 Add support for centos-9
64a830b933 Merge pull request #27 from snapcore/improve-log-analyzer
7fe27d4aea Improve log analyzer utility
207536268e Merge pull request #19 from snapcore/new-spread-manager
2f2ff2e282 Update spread manager to support csv
33a44ca3be Merge branch 'main' into new-spread-manager
b4654950d4 Merge pull request #26 from snapcore/support-csv-for-expressions
3250bbd885 Support expressions with comma separation
2540135b90 Merge pull request #25 from snapcore/add-indent-to-log-parser
2536b0f070 Minor improvements in log-parser and log-analyzer based con review comments
84dc8092b1 Merge pull request #24 from snapcore/improve-log-parser
515770b3bf Add support for fedora-35
875c29b5ce Updated results with latest log-parser changes
d27f2bcdb7 Fix log-parser
b2cce1fcce fix wording
14d15e4fe4 Fixes for log-parser and changes for log analyzer
438d92d241 Log analyzer updated to support reexecute in all the scenarios
1c9dff58ff Merge pull request #23 from snapcore/improve-log-parser
cc7ee488d1 Fix shellcheck
324b99e719 revert change in log-analyzer test
f746f40ebe Fix shellcheck
2d7dbbe1bd Fix spelling
728dd64c2c Last set of changes for log analizer tools
bf389dcd01 New fixes for log parses
6b2b56afc3 Fix another shellcheck
56163e170b Fix shellcheck
d96ab8094f Merge branch 'main' into new-spread-manager
60fb99f02f new dir task5
259a7e188c Fix spread test
e674234454 New spread-manager tool
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: fea2bac306a463f4c97f79a6e934e85ec00e804c
* Using a custom remote configuration file
This is to avoid issues when the helpers try to use the remote tools
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from fea2bac306..4a994e96b3
4a994e96b3 Fix new config path in tests
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4a994e96b33d5966053645ec87563d0ea8fd019b
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4a994e96b3..53e6c8f315
53e6c8f315 Use the new ubuntu kinetic image
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 53e6c8f315fbdd379b59d6458eb869e68f34c9b3
* fix tests with wrong use of the new tools
* Update test with remote.push command
This test seems to be updated during last merge.
* Update sprad.yaml which fails on debug
* Update the number of workers for nested tests on uc20 and uc22
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 53e6c8f315..4d6088ec8c
4d6088ec8c Fix shellcheck in spread-manager util
edb4f3f51b fix shell check errors in remote tools
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 4d6088ec8c5d43e7b4d45faf904978175c418120
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 4d6088ec8c..f46dccb464
f46dccb464 Fix sshpass command in remote.exec
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: f46dccb464f3a82932c0a89ecabffb6ef84ec6df
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 872b64e05b..1a986df8d5
1a986df8d5 Merge pull request #17 from snapcore/new-snaps-cleanup-tool
29600af53a Fixing os.paths test for arch linux
e5cdb06053 Minor fixes based on comment reviews
89a0073b88 Run cleanup tests just on systems with snapd pre-installed
19d216a1b9 New tools to get os paths, snap names and cleanup the snaps
a36ee8451a Merge pull request #16 from snapcore/add-codespell
f837691b62 Adding codespell tool to github workflow
58418c3f7e Merge pull request #15 from snapcore/testing-jammy
f015a34899 Fix issue about how info is saved for Debug output
1ee4240e79 Adding jammy to the test systems in spread.yaml
61a4c3142d Merge pull request #14 from snapcore/include-support-s390x
3a1a65f6e1 Update the comments about shellchek
5ddfb7e1db New support for s390x arch in os.query tool
575f5d2016 Merge pull request #13 from snapcore/new-utils-section
4701612434 avoid installing and removing snapd in ubuntu focal
4d0317633d shellcheck in xenial is not needed
d95cb2e627 Moving some tools to the new utils section
839a0ce808 Merge pull request #10 from snapcore/new-spread-shellcheck-tool
5d5efd6fbe Merge pull request #12 from snapcore/run-shellchek-in-old-version
497b7d691f Force testing shellchek with versions 0.8 and 0.7
b3cb5a5a0a Update the copyright
5cfa7cb84d New shellcheck for spread tasks
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 1a986df8d5e66459ec9aa2498a1e9c7d7e419284
* Using new snaps.name and snaps.cleanup tools
These tools are imported from snapd-testing-tools
* cleanup tool will not be in bin
* Fix shellcheck
* Update missing tests
* Fix shellcheck
* Squashed 'tests/lib/external/snapd-testing-tools/' changes from 1a986df8d5..53ca1b3b70
53ca1b3b70 Merge pull request #21 from snapcore/fix-some-shellcheks
d380a8065e Fix shellchecks
ad410d44cd Merge pull request #20 from snapcore/fixes-on-remote-tools
80bfd6d85b Updates based on review comments in snapd project
5c77d93bd6 Merge pull request #18 from snapcore/tests-new-remote-tools
3cef4b69b4 Updating timeouts
2c6bf4bd48 Using local vars and removing wait-for tool
8642893803 Test fixes
a14ec4ea88 New remote tools
4545a5e5be Merge branch 'main' into tests-new-remote-tools
a791ae89cb New remote tools
git-subtree-dir: tests/lib/external/snapd-testing-tools
git-subtree-split: 53ca1b3b709e685b7910e4b559f182dd78039029
* Make snaps.name executable
Add a tool for dumping or otherwise controlling device cgroup of a snap application.
Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
This tool expose package management of the underlying system with a
a basic install / remove interface. Debian package names are re-mapped
to the convention of the system.
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
The new tool is used for backup and restore the test fs.
The meachnism is already being used by the prepare/restore for tests on
main suite, but now it is extended to be used in nested tests as well.
The idea of the tool is to backup the test directory revious to the test
execution and restore the backup after hte test is restored.
Several of our tests used a trick where a "defer.sh" script was appended
to, collecting more and more commands to execute. The restore section
then used tac(1) to reverse file order and execute the result,
implementing a simple cleanup stack in shell.
This patch takes that idea and polishes the rough edges. The new
tests.cleanup tool comes with basic but sufficient functionality, tests
and documentation to complete the same task.
Using this pattern can simplify restore logic as the conditions that
lead to prepare or execute customizing something do not have to be
repeated.
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Internal test tools are now separated into two classes, common tools are
symlinked to tests/bin and are on PATH, remaining tools are invoked with
full path.
The README file had some stale TODO entries that were changed earlier,
those are now marked as done.
Lastly any-python is now on PATH, so that scripts can invoke it via #!
mechanism.
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
Note that this tool is used somewhat more often so it is placed on PATH
with a symbolic link in tests/bin.
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
In addition, tests.invariant will be on PATH, by placing as symlink in
the tests/bin directory. I was thinking about this for a while but I
think it's useful to use "tests.invariant check" in a debug session, to
see if anything is fishy.
Signed-off-by: Zygmunt Krynicki <me@zygoon.pl>
