mirror of
https://github.com/uutils/uutils.github.io.git
synced 2026-10-16 03:24:02 -07:00
Sylvestre Ledru
e7ae2ac985
The "Build website artifacts" job spends most of its time here: the
l10n step builds ~36 language books one after another, each taking its
turn at mutating the shared coreutils checkout (swap the locale into
en-US.ftl, run uudoc, build mdbook, restore en-US.ftl). Nothing in it
uses more than one core.
Everything uudoc touches is relative to its working directory, so give
each language a throwaway copy of the checkout to work in and run
DOC_JOBS of them at a time (default nproc). The checkout itself becomes
read-only, which is what makes this safe and also removes the need to
restore en-US.ftl between languages. Copying the checkout costs ~0.2s
per language; target/, .git/ and docs/book*/ are left out, which is
almost all of its weight.
The per-language tldr download and repack moves into the worker too:
it was a second serial loop, ~36 archives fetched one at a time before
any book was built.
Measured on a 30-locale tree with DOC_JOBS=4, to match a GitHub runner:
10m04s at 107% CPU -> 1m59s at 387%. The 30 resulting books are
byte-for-byte identical to the ones the serial script produces.
Two notes on what this deliberately avoids, both found by running it on
a machine where uutils shadows GNU coreutils/findutils:
- The fan-out is a bash job pool rather than `xargs -P`. uutils' xargs
accepts -P and silently ignores it ("[NOT IMPLEMENTED]"), so the
parallel build would quietly stay serial there.
- The tldr merge keeps the original direction (English base, translated
pages overlaid on top). Overlaying English onto the translation with
`cp -rn` reads as equivalent, but uutils' cp stops merging a directory
after the first existing file and still exits 0 — it copied 296 of
4673 pages, and the Arabic book lost every Examples section.