Since bug 1239217, artifacts builds are using a hybrid build system that
uses the fastermake rules to copy files to dist/bin, which means
artifact files are not removed by the processing of the dist/bin install
manifest. This means we don't need to add them to the recursivemake
install manifest anymore.
Currenlty --enable-artifact builds will take artifacts from fx-team regardless of the
state of the current working directory. This can lead to broken builds if someone
updates to a tree other than fx-team.
This commit changes the default behavior from tracking fx-team to finding all recent
pushheads and finding the closest to the working parent that has artifacts available.
This also fixes a mis-match between tree names according to mozext and branch names in
the taskcluster index preventing artifact download from common integration branches.
clang based tools typically choke when they encounter a lot of -XClang
arguments, and since those args are passed to the cc1 driver they're
unnecessary for such tools anyways.
This helps people who have things like --enable-clang-plugin in their
mozconfig use clang based tools without having to remove these arguments
manually.
Downloaded from https://pypi.python.org/packages/source/v/virtualenv/virtualenv-14.0.1.tar.gz
and extracted without any modifications.
This appears to speed up `mach configure` for artifact builds by ~3.3s on
my Linux machine (~16.3s to 13.0s). A significant part of this appears
to be newer version of pip caching and reusing wheels after first
install.
This should be a rubber stamp review, as all changes are from trusted
upstream packages.
Calling error() in moz.build files to indicate unsupported configurations is
useful, but readers using EmptyConfig will trigger them currently. This patch
allows a Config to have an `error_is_fatal` attribute, which will make
error emit a warning instead.
This is two straightforward optimizations in FileCopier: avoiding a redundant iteration
over the directory structure to find destination files (which includes a
call to normpath) and avoiding redundant calls to determine directories to preserve
when remove_unaccounted is not specified (which include a call to dirname).
Running a no-op install of _tests with this patch results in a reduction of about
25,000 calls to normpath and remove about 220,000 calls to dirname, resulting in
an overall speedup of 10-20%.
Listing the tests in all-tests.json is prerequisite to adding support
for Marionette tests to `./mach test FILE`.
Marionette tests will be run from the source directory, as they do not
currently need packaging.
The current rule is only for "backend.RecursiveMakeBackend", but, with
the current default of generating both the RecursiveMake and FasterMake
backends, the command creates/refreshes both backends. This is, in fact,
how the FasterMake backend is refreshed in most cases.
Moreover, with an hybrid backends, the generated file is not
"backend.RecursiveMakeBackend" anymore, so we need a more generic way to
handle this.
Furthermore, it's not necessarily desirable for all backends to have a
dependency file to handle the dependencies to refresh the backend, so
generate a plain list instead. This has the side effect of making `mach
build-backend --diff` more readable for changes to that file.
Finally, make the backend.* files created like any other backend file,
such that its diff appears in the `mach build-backend --diff` output.
The FasterMake build system is meant to be invoked through `mach build
faster`, which does it already, or, in the near future, as part of an
hybrid build system, which will deal with it as well. People doing
`make -C objdir/faster` won't have the backend automatically refreshed,
but that's not a supported way to use it anyways.
Install manifests are not empty in normal conditions, apart a few
exceptions where they are only used for a "magic" `rm -rf`.
However, we're going to introduce changes that will empty some of
the install manifests and make their work happen from a different
backend, in which case we don't want them to correspond to a `rm -rf`.
When doing build system changes affecting backend-generated files, I
often use `mach build-backend --diff`. But most of the time I end up
wanting to look at the full diff again when doing further changes, which
leads me to stash my changes away, run `mach build-backend` to get the
initial state again, unstash and rerun `mach build-backend --diff`.
This has been a time drain for long enough :)
The code doing this was added before we had install manifests, back when
they were purge manifests, and before we tracked all files created by
the backend. Nowadays, if an install manifest is removed, it will be
removed in BuildBackend.consume.
In fact, purging the install manifests in the backend itself breaks the
deleted files count displayed after reticunating splines.
XPI_NAME affects no tier on its own, as it merely changes paths where
things end up that are defined by other variables.
FILES_PER_UNIFIED_FILE had an erroneous value.
Those are the worst offenders of affected_tiers. The rules to handle
them are all in directories that is not necessarily related to where
the variables are defined, each of which has a Makefile.in for those
rules, which forces export to go through them.
They are respectively using PP_TARGETS and INSTALL_TARGETS. Both affect
the misc tier since bug 1240671, per the *_TARGET value they set in the
backend.
This has the nice side effect of now skipping directories where test
harness files are handled by install manifests.
JAR_MANIFESTS affects the libs tiers through config/rules.mk rules.
While we could move the rules in the backend, they are too complex to
just do that now.
GENERATED_FILES impacts the export tier through the config/rules.mk
definitions, now moved to the backend itself, so that everything is
close to each other.
Those are non-passthru variables with the same property as
ANDROID_GENERATED_RESFILES, ANDROID_APK_NAME and ANDROID_APK_PACKAGE, in
that they don't affect tiers through the backend code itself, but from
the Makefile.in along the moz.build they are defined in.
The are passthru variables that don't actually affect any tier per the
backend itself. They do affect the `export` tier by way of the Makefile.in
along the moz.build defining them, and the existence of those
Makefile.in already guarantees those directories not to be skipped for
`export`.
This initiates a move off affected_tiers in VARIABLES definition to
explicit in-backend handling, which will hopfully make things clearer.
HAS_MISC_RULE is currently used to opt-in to the misc tier in a few
directories with a misc:: rule. It is, in fact, mostly used for custom
xpi creation, which will be separately addressed in bug 1240676, so it
will eventually go away entirely, but in the meantime, we send it as a
throwaway passthru.
Historically, all tiers were handled as opt-out (may_skip), until we
added the first opt-in tier (no_skip). It doesn't make much sense to
differentiate them anymore, so handle them all as opt-in.
When adding a backend, we currently have to add them in three different
places so that they appear in the right places.
Instead, keep the list in a single place.
Currently |./mach artifact| installs an old version of the taskcluster client,
which installs an old version of requests that's incompatible with commonly
installed python versions. This bumps to a version of taskcluster client that
accepts and requests version < 3, so we pick up the in-tree version instead of
installing 2.4.3.
