mirror of
https://gitlab.winehq.org/wine/wine-gecko.git
synced 2024-09-13 09:24:08 -07:00
154 lines
4.6 KiB
ReStructuredText
154 lines
4.6 KiB
ReStructuredText
.. _test_manifests:
|
|
|
|
==============
|
|
Test Manifests
|
|
==============
|
|
|
|
Many test suites have their test metadata defined in files called
|
|
**test manifests**.
|
|
|
|
Test manifests are divided into two flavors: :ref:`manifest_destiny_manifests`
|
|
and :ref:`reftest_manifests`.
|
|
|
|
.. _manifest_destiny_manifests:
|
|
|
|
Manifest Destiny Manifests
|
|
==========================
|
|
|
|
Manifest destiny manifests are essentially ini files that conform to a basic
|
|
set of assumptions.
|
|
|
|
The `reference documentation <http://mozbase.readthedocs.org/en/latest/manifestdestiny.html>`_
|
|
for manifest destiny manifests describes the basic format of test manifests.
|
|
|
|
In summary, manifests are ini files with section names describing test files::
|
|
|
|
[test_foo.js]
|
|
[test_bar.js]
|
|
|
|
Keys under sections can hold metadata about each test::
|
|
|
|
[test_foo.js]
|
|
skip-if = os == win
|
|
|
|
There is a special **DEFAULT** section whose keys/metadata apply to all
|
|
sections/tests::
|
|
|
|
[DEFAULT]
|
|
property = value
|
|
|
|
[test_foo.js]
|
|
|
|
In the above example, **test_foo.js** inherits the metadata **property = value**
|
|
from the **DEFAULT** section.
|
|
|
|
Recognized Metadata
|
|
-------------------
|
|
|
|
Test manifests can define some common keys/metadata to influence behavior.
|
|
Those keys are as follows:
|
|
|
|
head
|
|
List of files that will be executed before the test file. (Used in
|
|
xpcshell tests.)
|
|
|
|
tail
|
|
List of files that will be executed after the test file. (Used in
|
|
xpcshell tests.)
|
|
|
|
support-files
|
|
List of additional files required to run tests. This is typically
|
|
defined in the **DEFAULT** section.
|
|
|
|
Unlike other file lists, *support-files* supports a globbing mechanism
|
|
to facilitate pulling in many files with minimal typing. This globbing
|
|
mechanism is activated if an entry in this value contains a ``*``
|
|
character. A single ``*`` will wildcard match all files in a directory.
|
|
A double ``**`` will descend into child directories. For example,
|
|
``data/*`` will match ``data/foo`` but not ``data/subdir/bar`` where
|
|
``data/**`` will match ``data/foo`` and ``data/subdir/bar``.
|
|
|
|
generated-files
|
|
List of files that are generated as part of the build and don't exist in
|
|
the source tree.
|
|
|
|
The build system assumes that each manifest file, test file, and file
|
|
listed in **head**, **tail**, and **support-files** is static and
|
|
provided by the source tree (and not automatically generated as part
|
|
of the build). This variable tells the build system not to make this
|
|
assumption.
|
|
|
|
This variable will likely go away sometime once all generated files are
|
|
accounted for in the build config.
|
|
|
|
If a generated file is not listed in this key, a clobber build will
|
|
likely fail.
|
|
|
|
dupe-manifest
|
|
Record that this manifest duplicates another manifest.
|
|
|
|
The common scenario is two manifest files will include a shared
|
|
manifest file via the ``[include:file]`` special section. The build
|
|
system enforces that each test file is only provided by a single
|
|
manifest. Having this key present bypasses that check.
|
|
|
|
The value of this key is ignored.
|
|
|
|
run-if
|
|
Run this test only if the specified condition is true.
|
|
See :ref:`manifest_filter_language`.
|
|
|
|
skip-if
|
|
Skip this test if the specified condition is true.
|
|
See :ref:`manifest_filter_language`.
|
|
|
|
fail-if
|
|
Expect test failure if the specified condition is true.
|
|
See :ref:`manifest_filter_language`.
|
|
|
|
run-sequentially
|
|
If present, the test should not be run in parallel with other tests.
|
|
|
|
Some test harnesses support parallel test execution on separate processes
|
|
and/or threads (behavior varies by test harness). If this key is present,
|
|
the test harness should not attempt to run this test in parallel with any
|
|
other test.
|
|
|
|
By convention, the value of this key is a string describing why the test
|
|
can't be run in parallel.
|
|
|
|
.. _manifest_filter_language:
|
|
|
|
Manifest Filter Language
|
|
------------------------
|
|
|
|
Some manifest keys accept a special filter syntax as their values. These
|
|
values are essentially boolean expressions that are evaluated at test
|
|
execution time.
|
|
|
|
See
|
|
`the source <https://hg.mozilla.org/mozilla-central/file/default/testing/mozbase/manifestdestiny/manifestparser/manifestparser.py>`_.
|
|
|
|
.. todo::
|
|
|
|
Document manifest filter language.
|
|
|
|
.. _manifest_file_installation:
|
|
|
|
File Installation
|
|
-----------------
|
|
|
|
Files referenced by manifests are automatically installed into the object
|
|
directory into paths defined in
|
|
:py:func:`mozbuild.frontend.emitter.TreeMetadataEmitter._process_test_manifest`.
|
|
|
|
Referenced files in the manifest not in the same directory tree as the manifest
|
|
file are **not** installed.
|
|
|
|
.. _reftest_manifests:
|
|
|
|
Reftest Manifests
|
|
=================
|
|
|
|
See `MDN <https://developer.mozilla.org/en-US/docs/Creating_reftest-based_unit_tests>`_.
|