The hglib Mercurial finder was nice. But it is somewhat slow, as it
involves a separate Mercurial process.
This commit introduces a native Mercurial finder that speaks directly
to a Mercurial repository instance. It is significantly faster.
The new code is isolated to its own file because it imports Mercurial
code, which is GPL.
This commit adds support for specifying a Mercurial revision with `mach
file-info`. Aside from being a potentially useful feature, it proves
that MercurialRevisionFinder works with BuildReader.
The moz.build reader uses absolute paths when referencing moz.build
files. *Finder classes expect paths to be relative to a base. When we
switched the reader to use *Finder instances for I/O, we simply provided
a default FileFinder based at the filesystem root. This was quick and
easy. Things worked.
Unfortunately, that solution isn't ideal because not all *Finder
instances can accept absolute paths like that. The
MercurialRevisionFinder is one of them.
Changing the moz.build reader to talk in terms of relative paths is a
lot of work. While this would be ideal, it is significantly easier to
defer the problem until later and hack up MercurialRevisionFinder to
accept absolute paths. This commit does exactly that.
Bug 1171069 has been filed to track converting BuildReader to relative
paths.
Now that moz.build files use finders for I/O, we can start reading
moz.build data from other sources.
Version control is essentially a filesystem. We implement a finder
that speaks to Mercurial to obtain file data. It is able to obtain
file data from a specific revision in the repository.
We use the hglib package (which uses the Mercurial command server) for
speaking with Mercurial. This adds overhead compared to consuming the
raw Mercurial APIs. However, it also avoids GPL side-effects of
importing Mercurial's Python modules.
Testing shows that performance is good but not great. A follow-up
commit will introduce a GPL licensed Mercurial finder. For now, get
the base functionality in place.
Sometimes moz.build data may not come from the local filesystem. To
support defining moz.build data in other backing stores, we switch
moz.build reading to use mozpack's *Finder classes for I/O.
The use of a FileFinder bound to / is sub-optimal. We should ideally
be creating a finder bound to topsrcdir. However, doing this would
require refactoring a lot of path handling in the reader, as that code
makes many assumptions that paths are absolute. This would be excellent
follow-up work.
While I was here, I cleaned up some linter warnings for unused imports.
On my machine, this commit results in a slight slowdown of moz.build
reading. Before, `mach build-backend` was consistently reporting 1.99s
for reading 2,572 moz.build files. After, it is consistently reporting
2.07s, an increase of 4%. This is likely due to a) function call
overhead b) the cost of instantiating a few thousand File instances
c) FileFinder performing an os.path.exists() before returning File
instances. I believe the regression is tolerable.
Passing raw file handles around is a bit dangerous because it could lead
to leaking file descriptors. Add a read() method that handles the simple
case of obtaining the full contents of a File instance.
This should ideally be a method on BaseFile. But this would require
extra work and isn't needed. So we've deferred it until bug 1170329.
Today, the *Finder classes are optimized for doing matching and
returning multiple results. However, sometimes only looking for a
single file is wanted.
This commit implements the "get" method on all the *Finder classes.
It returns a BaseFile or None.
FileFinder._find_file has been refactored into FileFinder.get
to avoid code duplication.
The current mode of execution of templates doesn't allow them to more advanced
control flow, like returning early, returning or yielding values, because that
mode of execution is equivalent to running the code at the top level of a .py
file.
Making the templates executed through a function call, although trickier,
allows those control flows, which will be useful for template as context
managers.
inspect.getsourcelines() and inspect.getfile() involve I/O out of our control.
Our use of those functions, however, doesn't require all their smarts. In fact,
we only use them on function objects, for which we can just do the work
ourselves without involving inspect functions that trigger I/O.
According to the Baseline Requirements, root certificates MUST NOT
have the extendedKeyUsage extension. The extension is optional for
intermediates and required for end-entity certificates. This change
modifies the test certificates so they're more in line with the BRs.
The one change this patch makes in functionality is that it properly computes minimumUse in more cases, in anticipation of it being used for hazards (in addition to its current use for unnecessary roots).
