This adds support for web-platform-tests to mach try. It changes the implementation
so that instead of passing paths to manifests, the user passes arbitary paths in the
source tree, and tests under that path are run, with test discovery mainly left to
the harness.
Previously, we had a single make target and rule for generating all test
archives. These tasks can be performed in parallel. This commit
refactors the make file to add multiple targets for each archive and
thus enables test archives to be generated concurrently.
On my MacBook Pro, this reduces `make package-tests -j8` from ~78s to
~50s, a reduction of ~28s, or ~36%. Reduction on machines without SSDs
(like many builders in automation) will likely be less. Although, the
page cache should service most file reads during archiving since these
files were just staged, so hopefully the gains are in the same ballpark.
Upcoming work will introduce multiple targets for building test
archives. To prepare for this, we introduce a phony target that
tracks the staging of all test files so each target can gate on a common
prerequisite.
If a Buildbot test job is scheduled through TaskCluster (The Buildbot Bridge supports this),
then the generated Buildbot Change associated to a test job does not have the installer and
test url necessary to Mozharness to run the test job.
Since we can't modify how a test job is called on Buildbot (we can't switch from
--read-builbot-config to --installer-url and --test-url), we have to detect that there is
a 'taskId' defined for the test job (this indicates that the job was scheduled through the BBB)
and based on suc 'taskID' we can determine the parent task and the artifacts it uploaded.
Changes to ScriptMixin:
* Refactor _retry_download_file() to _retry_download()
* If no file is specified when calling_retry_download() we call _urlopen() instead of _download_file()
* Add load_json_url() method to fetch the contents of a json file without writing to disk
Changes to TestingMixin:
* If the job is triggered through Buildbot we look for the Changes object, otherwise, we look
for artifacts of the parent task
* Added functions find_artifacts_from_buildbot_changes (original behaviour)
and find_artifacts_from_taskcluster (functionality via TaskClusterArtifactsFinderMixin)
* Call self.exception() instead of raising exceptions + minor fixes
New TaskClusterArtifactsFinderMixin:
* It allows any inheriting class to find the artifacts of the build job which triggers this test job
This stops exposing ANDROID_BUILD_TOOLS and ANDROID_PLATFORM_TOOLS via
AC_SUBST. We expose most tools already, and this adds EMULATOR, and
consumes it (and ADB) where appropriate.
This performs the update check for system add-ons. It runs as part of the daily
add-on update checks similar to hotfix checks. Currently no URL is set so builds
won't actually start checking yet.
I've taken a few shortcuts here by only staging updates and needing a restart to
install as well as always downloading updates rather than using existing local
copies. At least the latter probably needs fixing before turning this on but
it makes more sense to iterate on those in tree.
This commit is us getting out of our own way. We were specifying
-classpath twice, once in $(JAVAC) and once in java-build.mk. Only
the latter of these is active. This a problem for ANDROID_EXTRA_JARS
-- those JARs should be on the classpath and input to $(DX) -- and
JARs that should be on the classpath but *not* input to $(DX). This
commit removes the global flags to $(JAVAC) and adds
JAVA_{BOOT}CLASSPATH_JARS. This required some hijinkery moving
wildcards to moz.build files, but everything seems to work.
As well as clarifying some parts of the build, part 2 uses this work
to modify the classpath.
