A --build run now reports how long each phase took, alongside the existing
resolve time from the progress spinner:
augment : libdisplay-info staged (42.1s)
build : 1 backend(s) in 63.4s
This makes emb's cache-hit speedup measurable in CI (a warm consume build shows
resolve in seconds, no download/extract) and surfaces where cold time goes.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The publish skip-on-exists keyed only on sysroot_key, which omits the baked
toolset (the Dockerfile's base image, apt list, and slim prune list). A toolset
change with an unchanged sysroot (e.g. adding build-essential) left the old
image in place — skip-on-exists matched and never rebuilt.
The published image's primary tag is now `<sysrootKey>-<toolsetHash>`, where the
toolset hash covers the emitted Dockerfile + .dockerignore (ToolchainImage.
tagFor/imageTag). An emitter/toolset change yields a new tag, so publish
rebuilds and re-points the aliases. The baked in-image paths stay keyed by
sysrootKey, so a consume build still cache-hits; the build matrix references the
moving :<os> alias, so it always gets the latest image.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The toolchain image shipped cmake/ninja/meson but no native C/C++ compiler.
Rebuilding an `augment` library (libdisplay-info via meson) at consume time
then failed at `meson setup` — meson needs a build-machine compiler to
configure even a cross build. (Local validation missed it: that sysroot already
had the augment baked in, so meson was skipped.)
- dockerfile_emitter: add build-essential (gcc/g++/libc6-dev/make). Confirmed
in a debian:bookworm-slim container: meson setup fails without it, succeeds
with it.
- overlay_builder._check: surface stdout as well as stderr — meson/cmake write
their diagnostics to stdout, so failures previously showed only a bare
"meson setup failed (exit 1)".
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The arm-gnu provider resolved the -dev closure only from one-line `deb …`
sources (sources.list + sources.list.d/*.list). Trixie (Debian 13) and current
raspios ship their sources in deb822 format (sources.list.d/*.sources with
Types/URIs/Suites/Components), so resolution failed with "no apt sources …
(cannot resolve -dev)".
- aptIndexUrls now parses both formats: one-line entries and deb822 stanzas
(multi-value URIs/Suites/Components, trailing-slash URIs normalized,
deb-src and Enabled:no skipped).
- _readAptSources also reads sources.list.d/*.sources and separates files with
a blank line so one-line and deb822 stanzas never merge.
Fixes the publish-trixie job; publish-bookworm (one-line sources) already
worked. Tests cover deb822 and mixed one-line+deb822 inputs.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The skip-on-exists probe targets the primary tag. When a caller passed only a
moving alias (e.g. --tag bookworm), a changed manifest would find the alias
already present and skip — leaving consumers on a stale image.
Always publish the content-addressed sysroot_key as the primary (skip-checked)
tag, with --tag values added as aliases on top. A changed manifest yields a new
key that doesn't exist yet, so it rebuilds and re-points the aliases; an
unchanged manifest is correctly skipped. Enables CI to reference a stable
per-OS alias (`:bookworm`/`:trixie`) while staying correct across changes.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Factor the hardware/OS facts out of project manifests into a board library
that ships with emb, and let manifests `extends` it. Resolution merges the
layers: emb board (hardware) <- project (embedder) [<- app, future].
- boards/raspberry-pi.emb.yaml: hardware-only base for rpi{5,4,zero-2w} x
{bookworm,trixie} — provider, triple, per-OS toolchain+image+partition,
per-model cpu_flags, base GUI/input/Vulkan stack. Located at runtime via the
emb_cli package_config (override with EMB_BOARDS_DIR or the resolver arg).
- cross.extends: <board> in a target deep-merges the project layer over the
named board, recursing for chains. Merge rules: maps deep-merge; cpu_flags
and backends replace (per-model / complete statement); sysroot.dev_packages
union (board base + project additions, deduped).
- examples/cross/raspberry-pi-family.emb.yaml now `extends` the boards and
carries only the project layer (backends, augment, plugin packages) — the
hardware duplication is gone.
- Fix cross_command_test target names for the expanded example (#51).
Validated: extends resolves in the real CLI via package_config auto-discovery;
--list-targets + --dry-run show hardware-from-board + project overlay; new
resolver tests cover inherit/override/union/replace/unknown-board. 294 pass.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Expand the rpi family example from a single bookworm/drm-kms-egl board into the
six targets the ivi-homescreen PiOS CI needs: {rpi5, rpi4, rpi-zero-2w} x
{bookworm, trixie}.
- OS variant drives toolchain + sysroot (and thus the container image):
bookworm -> 12.3.rel1 + raspios-bookworm; trixie -> 15.2.rel1 + raspios-trixie.
bookworm cross-builds libdisplay-info 0.2.0 (ships 0.1.1, drm-cxx needs >=0.2.0)
and omits the apt libdisplay-info-dev so the static .a isn't shadowed; trixie
keeps it. Vulkan headers are vendored by ivi-homescreen + drm-cxx, so neither
OS stages them.
- Board only changes -mcpu (a76/a72/a53); all three share one sysroot per OS.
- Backends: rpi5 builds all five (incl. drm-kms-vulkan); rpi4 and zero-2w build
the four without it. Plugins on (PLUGINS_DIR=../ivi-homescreen-plugins).
Validated: parses, --list-targets shows the right per-model backend sets, and
--dry-run resolves each target's toolchain/image/cpu/augment. Example tests pass.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Closes the gap left by --dockerfile (which only emits): --publish resolves,
emits, builds, and pushes the toolchain image to a registry, so CI pulls a
ready toolchain instead of resolving. Skips the build when the content-
addressed tag is already published.
Registry-agnostic by construction (serves GHCR, JFrog Artifactory, ECR, GAR,
Docker Hub): the --image prefix is free-form host[/path]/name, auth is left to
a prior `docker login`, and the existence probe is a plain Registry-v2 query.
Flags: --publish, --image <prefix>, --tag (repeatable, default sysroot_key),
--force, --no-push, --container-tool. The publish argv is a pure, unit-tested
ImagePublishPlan; the handler runs it through the injected process runner.
The existence probe prefers `skopeo inspect docker://<ref>` (works under both
podman and docker, single- or multi-arch), falling back to `<tool> manifest
inspect` on docker hosts without skopeo; an unprobeable host just rebuilds,
which is safe since the image is content-addressed.
Validated against a local registry: skip-on-exists short-circuits the build
(exit 0, no build); build + push invoked correctly.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The arm-gnu toolchain image baked the full device rootfs (~7.4 GB for the
radxa zero3). Prune it to a cross-build sysroot via .dockerignore: drop the
runtime data, bundled apps, docs, kernel/firmware, and target executables
(never used when cross-compiling on the build host); keep headers, libraries,
pkgconfig/cmake metadata, and the wayland protocol XMLs.
Image-only — the host sysroot stays full for local non-container builds.
.dockerignore (not a RUN rm) so the pruned content never enters a layer.
Validated on radxa zero3: sysroot 7.4 GB -> 4.7 GB (image ~8 GB -> ~6 GB),
with all three backends (wayland-egl, drm-kms-egl, software) still building.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Downloading + extracting an arm-gnu toolchain + sysroot is the slow part
of a cold build. --dockerfile resolves the target, then emits a Dockerfile
+ .dockerignore (into the platform dir) that bake the toolchain + sysroot
into an OCI image, so CI pulls a ready toolchain instead of re-resolving.
The image bakes them at a fixed workspace (/emb) under the same
cross-<triple>-<sysrootKey> path emb computes, so consuming it needs no
special flag: a build in the image resolves as a cache hit (providers skip
download/extract when the dir is populated; the toolchain file is re-emitted
for the in-image paths, and the sysroot's relativized symlinks relocate).
arm-gnu only (the Yocto providers don't lay out a self-contained dir to
bake); augment libs are rebuilt at consume time, not baked. The Dockerfile
generator is a pure, unit-tested function; --build is ignored when
--dockerfile is set. Validated live against the cached radxa toolchain.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
HostProvisioner gains availableUpdates() — package names with an update
available (null when the backend can't report it; empty = up to date).
PackageKit implements it via GetUpdates (the same data pkcon get-updates
uses); brew/winget return null for now.
emb doctor, after confirming the backend is available, prints the update
count (+ a short preview), labeled as reflecting the last cache refresh.
It stays read-only (no refresh) and never fails the command on the update
check.
Adds a doctor_command test (the feature had none): wiring, up-to-date,
unsupported, error-tolerance, and unavailable-skips-the-check. README +
example/README document the new line.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Turns the radxa_zero3 example into a build-validated manifest and adds
the two sysroot capabilities its bookworm-KDE image needs.
- apt_resolver: explicit dev_packages roots are always staged, even when
dpkg-status marks them installed — a device image often records a
package installed yet strips its files (e.g. linux-libc-dev). Transitive
deps keep the already-installed prune.
- cross.sysroot.symlinks: declare in-sysroot symlinks created after dev
staging (link path skipped if it already exists). Folded into sysrootKey.
- arm-gnu provider applies the symlinks after dev-package staging.
- radxa_zero3.emb.yaml: validated end-to-end — rootfs partition 3 (p2 is
boot), dev_packages, a drm->libdrm symlink (radxa's vendor linux-libc-dev
omits /usr/include/drm/), and the three board backends (wayland-egl,
drm-kms-egl, software). emb cross . --build --deb produced aarch64 ELFs
+ .debs for all three.
Tests: resolver root-always-staged, symlink parse + sysrootKey effect,
radxa example fields. examples/cross/README documents partition/symlinks.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Joel Winarske
Joel Winarske