Add a README subsection ("Layered manifests: extends, board → project → app")
covering both extends forms (<board> and <dir>#<target>) and the merge rules,
with project and app manifest snippets.
Add examples/cross/app-extends.emb.yaml: an app target that extends the rpi
family file's rpi5-bookworm target (which extends the board) and overlays an
app-specific dev package — a real three-layer chain. A cross_command test
--dry-runs it so it stays valid.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
`cross.extends` gains a second form alongside the board reference:
- <board> : a target in emb's shipped board library (hardware).
- <dir>#<target> : a target in another emb project at <dir> (the project
layer) — e.g. a Flutter app extending an ivi-homescreen
target, so plugin config (DISABLE_PLUGINS/PLUGINS_DIR,
plugin -dev packages) lives with the app.
This completes the layering: app -> project -> board. <dir> is resolved relative
to the extending manifest's project root (the .emb/ parent in .emb/ mode, else
the manifest's dir); the referenced project's own extends chain is applied via
resolve(), and the same merge rules apply (maps deep-merge, backends/cpu_flags
replace, sysroot.dev_packages union). Reference cycles across projects are
detected and reported.
Tests cover the three-layer merge (hardware from board, backends from project,
plugins from app, dev_packages unioned across all three) plus unknown-target
and missing-project errors.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The image tag is content-addressed by the manifest alone (sysroot key + toolset
hash), so the "already published?" check no longer needs a resolved profile.
Move it ahead of provider.resolve(): when the image exists, --publish now skips
the toolchain/sysroot download + extract entirely instead of resolving just to
discover it can skip.
This is why a cache-evicted publish job previously spent ~2 min re-resolving a
sysroot only to skip; it now returns in seconds regardless of the sandbox cache.
The post-resolve skip-check in _publishImage is removed (the early check is
authoritative and emits the same tag); --force / --no-push paths are unchanged.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
libdisplay-info 0.2.0's meson build reads /usr/share/hwdata/pnp.ids on the build
machine to bake the PNP-ID table into the library; without it `meson setup`
fails ("File /usr/share/hwdata/pnp.ids does not exist"). Add hwdata to the image
toolset alongside build-essential. Toolset change re-keys the image tag, so
publish rebuilds it automatically.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
A --build run now reports how long each phase took, alongside the existing
resolve time from the progress spinner:
augment : libdisplay-info staged (42.1s)
build : 1 backend(s) in 63.4s
This makes emb's cache-hit speedup measurable in CI (a warm consume build shows
resolve in seconds, no download/extract) and surfaces where cold time goes.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The publish skip-on-exists keyed only on sysroot_key, which omits the baked
toolset (the Dockerfile's base image, apt list, and slim prune list). A toolset
change with an unchanged sysroot (e.g. adding build-essential) left the old
image in place — skip-on-exists matched and never rebuilt.
The published image's primary tag is now `<sysrootKey>-<toolsetHash>`, where the
toolset hash covers the emitted Dockerfile + .dockerignore (ToolchainImage.
tagFor/imageTag). An emitter/toolset change yields a new tag, so publish
rebuilds and re-points the aliases. The baked in-image paths stay keyed by
sysrootKey, so a consume build still cache-hits; the build matrix references the
moving :<os> alias, so it always gets the latest image.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The toolchain image shipped cmake/ninja/meson but no native C/C++ compiler.
Rebuilding an `augment` library (libdisplay-info via meson) at consume time
then failed at `meson setup` — meson needs a build-machine compiler to
configure even a cross build. (Local validation missed it: that sysroot already
had the augment baked in, so meson was skipped.)
- dockerfile_emitter: add build-essential (gcc/g++/libc6-dev/make). Confirmed
in a debian:bookworm-slim container: meson setup fails without it, succeeds
with it.
- overlay_builder._check: surface stdout as well as stderr — meson/cmake write
their diagnostics to stdout, so failures previously showed only a bare
"meson setup failed (exit 1)".
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The arm-gnu provider resolved the -dev closure only from one-line `deb …`
sources (sources.list + sources.list.d/*.list). Trixie (Debian 13) and current
raspios ship their sources in deb822 format (sources.list.d/*.sources with
Types/URIs/Suites/Components), so resolution failed with "no apt sources …
(cannot resolve -dev)".
- aptIndexUrls now parses both formats: one-line entries and deb822 stanzas
(multi-value URIs/Suites/Components, trailing-slash URIs normalized,
deb-src and Enabled:no skipped).
- _readAptSources also reads sources.list.d/*.sources and separates files with
a blank line so one-line and deb822 stanzas never merge.
Fixes the publish-trixie job; publish-bookworm (one-line sources) already
worked. Tests cover deb822 and mixed one-line+deb822 inputs.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
The skip-on-exists probe targets the primary tag. When a caller passed only a
moving alias (e.g. --tag bookworm), a changed manifest would find the alias
already present and skip — leaving consumers on a stale image.
Always publish the content-addressed sysroot_key as the primary (skip-checked)
tag, with --tag values added as aliases on top. A changed manifest yields a new
key that doesn't exist yet, so it rebuilds and re-points the aliases; an
unchanged manifest is correctly skipped. Enables CI to reference a stable
per-OS alias (`:bookworm`/`:trixie`) while staying correct across changes.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Factor the hardware/OS facts out of project manifests into a board library
that ships with emb, and let manifests `extends` it. Resolution merges the
layers: emb board (hardware) <- project (embedder) [<- app, future].
- boards/raspberry-pi.emb.yaml: hardware-only base for rpi{5,4,zero-2w} x
{bookworm,trixie} — provider, triple, per-OS toolchain+image+partition,
per-model cpu_flags, base GUI/input/Vulkan stack. Located at runtime via the
emb_cli package_config (override with EMB_BOARDS_DIR or the resolver arg).
- cross.extends: <board> in a target deep-merges the project layer over the
named board, recursing for chains. Merge rules: maps deep-merge; cpu_flags
and backends replace (per-model / complete statement); sysroot.dev_packages
union (board base + project additions, deduped).
- examples/cross/raspberry-pi-family.emb.yaml now `extends` the boards and
carries only the project layer (backends, augment, plugin packages) — the
hardware duplication is gone.
- Fix cross_command_test target names for the expanded example (#51).
Validated: extends resolves in the real CLI via package_config auto-discovery;
--list-targets + --dry-run show hardware-from-board + project overlay; new
resolver tests cover inherit/override/union/replace/unknown-board. 294 pass.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Expand the rpi family example from a single bookworm/drm-kms-egl board into the
six targets the ivi-homescreen PiOS CI needs: {rpi5, rpi4, rpi-zero-2w} x
{bookworm, trixie}.
- OS variant drives toolchain + sysroot (and thus the container image):
bookworm -> 12.3.rel1 + raspios-bookworm; trixie -> 15.2.rel1 + raspios-trixie.
bookworm cross-builds libdisplay-info 0.2.0 (ships 0.1.1, drm-cxx needs >=0.2.0)
and omits the apt libdisplay-info-dev so the static .a isn't shadowed; trixie
keeps it. Vulkan headers are vendored by ivi-homescreen + drm-cxx, so neither
OS stages them.
- Board only changes -mcpu (a76/a72/a53); all three share one sysroot per OS.
- Backends: rpi5 builds all five (incl. drm-kms-vulkan); rpi4 and zero-2w build
the four without it. Plugins on (PLUGINS_DIR=../ivi-homescreen-plugins).
Validated: parses, --list-targets shows the right per-model backend sets, and
--dry-run resolves each target's toolchain/image/cpu/augment. Example tests pass.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Closes the gap left by --dockerfile (which only emits): --publish resolves,
emits, builds, and pushes the toolchain image to a registry, so CI pulls a
ready toolchain instead of resolving. Skips the build when the content-
addressed tag is already published.
Registry-agnostic by construction (serves GHCR, JFrog Artifactory, ECR, GAR,
Docker Hub): the --image prefix is free-form host[/path]/name, auth is left to
a prior `docker login`, and the existence probe is a plain Registry-v2 query.
Flags: --publish, --image <prefix>, --tag (repeatable, default sysroot_key),
--force, --no-push, --container-tool. The publish argv is a pure, unit-tested
ImagePublishPlan; the handler runs it through the injected process runner.
The existence probe prefers `skopeo inspect docker://<ref>` (works under both
podman and docker, single- or multi-arch), falling back to `<tool> manifest
inspect` on docker hosts without skopeo; an unprobeable host just rebuilds,
which is safe since the image is content-addressed.
Validated against a local registry: skip-on-exists short-circuits the build
(exit 0, no build); build + push invoked correctly.
Signed-off-by: Joel Winarske <joel.winarske@gmail.com>
Joel Winarske
Joel Winarske