22 Commits

Author SHA1 Message Date
Zygmunt Bazyli Krynicki
6da6dd1472 tests: document or tweak documentation of interface tests (#13896)
* tests: fix typo: case

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-audio-playback-record

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-avahi-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-cli

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-content-circular

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-content-empty-content-attr

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-cups-control-autoconect

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-cups-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-content-mkdir-writable

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-custom-device-app-slot

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-desktop-document-portal

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-firewall-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-fuse-support

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: default

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-hardware-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-hardware-random-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-hardware-random-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-home

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-hooks

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-hooks-plug-with-number

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-kernel-module-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-libvirt

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-locale-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-location-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-log-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-mount-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: rewrite documentation of interfaces-mount-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-netlink-connector

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-network

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: rewrite documentation of interfaces-network-bind

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-network-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-network-manager

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat and tweak details of interfaces-network-status

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-packagekit-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-password-manager-service

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-personal-files

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: rewrite details of interfaces-polkit

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-pulseaudio

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-snapd-control

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-snapd-control-with-manage

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-system-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: reformat details of interfaces-upower-observe

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document interfaces-wayland

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document known

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document known-remote

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: correct the summary for known-remote test

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests: fix typo: bound

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document install-local-w-metadata

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

* tests/main: document login

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>

---------

Signed-off-by: Zygmunt Krynicki <zygmunt.krynicki@canonical.com>
2024-04-26 15:10:48 +02:00
Alberto Mardegan
3c21c1c1ae tests: temporary hack until the test snap has been updated
TODO: revert this when no longer needed :-)
2022-11-15 16:04:18 +03:00
Sergio Cazzolato
0a8b0a71d2 tests: run spread tests on arm64 instances in google cloud (#12067)
* New machines running in arm64 servers

* Fix for tests and workflow update to support a different backend for arm

* Use spread-arm when the system is arm64

* Fix matrix.system comparison

* Fix save results step in github worflow

* fix error in mount-order-regression test

* Skip upgrade-from-release on arm machines

* skip preseed-core20 test on arm

The model is for amd64, so it fails to prepare the image in arm

* Adding more logging to the run tests workflow job

* Fix uc20-create-partitions for pc-amd64 arch

* Update spread run command

This is because it failed to re-execute failed

* Run just on focal

This was discussed to reduce the number of machines we create on each
workflow execution
2022-09-07 16:06:23 -03:00
Sergio Cazzolato
6bfe99e648 tests: add executions for ubuntu 22.04
* Add executions for Ubuntu Jammy

There are some tests that are not been executed in Jammy

This change adds ubuntu 2* to supported systems on spread tests

* Fixes for tests

* revert change in test interfaces-opengl-nvidia

This needs some changes in the code and it will be done in a following
pr

* revert change to  fake-netplan-apply

Co-authored-by: Michael Vogt <mvo@ubuntu.com>
2022-07-14 17:11:34 +02:00
Sergio Cazzolato
75c9f46508 Back test to focal 2021-09-02 08:17:09 -03:00
Sergio Cazzolato
52973116b9 Merge remote-tracking branch 'upstream/master' into tests-fix-libvirt-test 2021-09-02 08:16:25 -03:00
Sergio Cazzolato
e6a314029b Fix interfaces-libvirt test
The test is failing after it has been moved to focal, the change
includes:
 . Add new dependency for focal
 . Use the new group libvirt instead of the old one libvirtd
2021-09-01 14:31:32 -03:00
Michael Vogt
4ae738d488 tests: move interfaces-libvirt test back to 16.04
This test was moved to run on `20.04` but there are at least two
problems:
1. The `libvirtd` group got renamed to `libvirt`
2. The <kernel>test_ping_serve.virtio</kernel> errors when it
   comes from a read-only medium like a snap

To unblock us short term this commit just reverts back to 16.04 to
unbreak master and then we can work on a proper fix.
2021-09-01 15:54:29 +02:00
Sergio Cazzolato
e41c6793b8 Revert some changes that are not needed 2021-08-27 13:01:44 -03:00
Sergio Cazzolato
cc6266051b Migrate tests that are only executed on xenial to bionic
Start using bionic to run tests that were executed just on xenial.

The idea is to start moving out from xenial which is not longer
supported.
2021-08-24 22:50:16 -03:00
Sergio Cazzolato
464050fa15 Replace "not MATCH" by NOMATCH in tests 2021-06-10 09:02:50 -03:00
Sergio Cazzolato
f880e5a51a Fixing the tests layout
The list of errors:

Checking tests formatting
Checks failed for task ./tests/regression/lp-1813963/task.yaml
- Keys 'systems' and 'backends' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/regression/lp-1597839/task.yaml
- Keys 'systems' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/lib/tools/suite/tests.session/task.yaml
- Keys 'execute' and 'restore' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/config-versions/task.yaml
- Keys 'systems' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task
./tests/main/classic-prepare-image-no-core/task.yaml
- Keys 'environment' and 'kill-timeout' do not follow the desired
order: ['summary', 'details', 'backends', 'systems', 'manual',
'priority', 'warn-timeout', 'kill-timeout', 'environment', 'prepare',
'restore', 'debug', 'execute']
Checks failed for task ./tests/main/classic-prepare-image/task.yaml
- Keys 'environment' and 'kill-timeout' do not follow the desired
order: ['summary', 'details', 'backends', 'systems', 'manual',
'priority', 'warn-timeout', 'kill-timeout', 'environment', 'prepare',
'restore', 'debug', 'execute']
Checks failed for task ./tests/main/lxd-postrm-purge/task.yaml
- Keys 'kill-timeout' and 'priority' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/interfaces-fuse-support/task.yaml
- Keys 'systems' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/lxd/task.yaml
- Keys 'kill-timeout' and 'priority' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/lxd-try/task.yaml
- Keys 'restore' and 'prepare' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/classic-custom-device-reg/task.yaml
- Keys 'environment' and 'kill-timeout' do not follow the desired
order: ['summary', 'details', 'backends', 'systems', 'manual',
'priority', 'warn-timeout', 'kill-timeout', 'environment', 'prepare',
'restore', 'debug', 'execute']
Checks failed for task ./tests/main/interfaces-network-control/task.yaml
- Keys 'systems' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/interfaces-libvirt/task.yaml
- Keys 'systems' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task
./tests/main/parallel-install-interfaces-content/task.yaml
- Keys 'backends' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task ./tests/main/interfaces-gpg-keys/task.yaml
- Keys 'systems' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
Checks failed for task
./tests/nested/manual/core20-custom-kernel-commandline/task.yaml
- Keys 'environment' and 'details' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']
- Keys 'debug' and 'prepare' do not follow the desired order:
['summary', 'details', 'backends', 'systems', 'manual', 'priority',
'warn-timeout', 'kill-timeout', 'environment', 'prepare', 'restore',
'debug', 'execute']

Crushing failure and despair.
Error: Process completed with exit code 1.
2021-04-20 09:21:11 -03:00
Maciej Borzecki
84aac46c55 tests: fix use of MATCH -v
MATCH is a wrapper around grep -E -q, thus MATCH -v ends up being grep -E -q -v.
However, grep -v is peculiar as far as exit codes are concerned, and returns 0
when lines not matching the pattern were found in the input.

For example, this command returns 1:

$ cat <<EOF | grep -E -v foo
foo
foo
foo
EOF

but this one returns 0, even though the pattern we don't want is in the input:

cat <<EOF | grep -E -v foo; echo $?
foo
bar
baz
EOF

In the tests, MATCH -v is used to check that the pattern does not appear in the
input, which is incorrect.

Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
2019-12-17 12:11:57 +01:00
Sergio Cazzolato
9aaaa933ec Remove PWD use and normalize interfaces tests 2019-01-22 18:18:15 -03:00
Maciej Borzecki
d356f9a33b tests: fix tests to not assume a permissive umask
When default umask is strict, eg. 0077, the
/home/gopath/src/github.com/snapcore/snapd hierarchy will have 0700 permissions,
meaning attempts to write to a place under that path as a `test` user will fail.
This has come up with Fedora 29 where the default umask was changed to a more
strict one.

Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
2018-12-06 08:48:25 +01:00
Maciej Borzecki
d3bc0fcf8f tests: make the match pattern for unconnected interface more specific
Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
2018-06-20 07:47:01 +02:00
Maciej Borzecki
e5affe0d85 tests: shellchecks part 4
Signed-off-by: Maciej Borzecki <maciej.zenon.borzecki@canonical.com>
2018-06-18 15:32:02 +02:00
sergio-j-cazzolato
90c6f913fa Checking interfaces declaring the specific interface
This is done in the tests which are checking the interfaces information
in order to make them more robusts, making the test independent of other
plug:slots declared such it is happening in gadgets.
2018-04-24 08:59:15 -03:00
sergio-j-cazzolato
20c8ab9e1c Update interface tests to remove extra checks and normalize tests
This change includes:
. Remove all the checks that are done after a connection and
disconnection (this is already checked as part of the snap_connect and
snap_disconnect tests).
. install_local function used to pack and install local snaps
. Remove not needed echo
. Change the check of interfaces default behaviour for the
autoconnection to support the scenario when there are other snaps also
connected to the interface (that happens when testing caracalla gadget)
. Add new lines between task tags
2018-02-22 11:30:19 -03:00
Sergio Cazzolato
0c0b605db5 tests: dependency packages installed during project prepare
The objective of this change is to provide an environment for all the
tests as similar as possible, so all the dependencies are installed
during the project setup.

The packages are not removed at the end of the project, so it will help
in case the machine needs to be used to debug.

Openvswitch test is moved to manual due to the dependencies install is
stuck trying to install openvswitch-switch package.
2017-07-18 11:40:52 -03:00
Federico Gimenez
b5f1b008b9 tests: prefer ipv4 over ipv6 (#3418)
apt update is hanging on security.ubuntu.com with IPv6, we are also getting timeouts on other apt operations that might be related to the same problem.

Instead of disabling it and enabling after concrete apt calls, with these changes we give weight to IPv4 over IPv6 at the prepare stage of the suite and this setting is kept during the whole execution.
2017-06-02 11:24:02 -03:00
Federico Gimenez
755f1c9d5d tests: add libvirt interface spread test (#2829) 2017-02-16 15:11:39 -02:00