mirror of https://github.com/Dasharo/heads.git synced 2026-06-13 10:16:29 -07:00

Files

T

Thierry Laurion a2a027f661 circleci: implement hierarchical cache model with fork-specific caches

Implements a new cache-based pipeline that dramatically improves build times
through hierarchical caching with fork-specific caches.

Key changes:
- Add executor, commands, and job definitions for cache-based builds
- Add create_hashes job to generate cache digest files
- Add x86_blobs job for blob downloads with cache support
- Add x86_musl_cross_make job for toolchain builds with cache save
- Add x86_coreboot job per fork, each saves both modules and coreboot caches
- Add ppc64_musl_cross_make and ppc64_coreboot jobs (decoupled from single job)
- Add glossary documenting fan-in, workspace chain, cache layers

Fixes for cache invalidation issues:
- Remove .circleci/config.yml from cache key hashes (prevents cache invalidation
  on CI config changes - was causing full rebuilds on every pipeline)
- Fix musl-cross-make module to auto-detect existing crossgcc using wildcard check
- Exclude .circleci/config.yml from all_modules_and_patches.sha256sums and
  coreboot_musl-cross-make.sha256sums

Test results (multiple pipeline runs):

Pipeline 3789 (first run, cold cache):
- x86-musl-cross-make: 30 min
- ppc64-musl-cross-make: 16 min
- Result: Cache saved

Pipeline 3790 (second run, cache hit):
- x86-musl-cross-make: 4.5 min (6.6x faster than first run)
- ppc64-musl-cross-make: 4.5 min (3.5x faster than first run)
- Result: Beats baseline (14.5 min) by 3.2x

Pipeline 3791 (third run, cache hit):
- x86-musl-cross-make: ~6 min (27s Make Board + spin up variance)
- ppc64-musl-cross-make: 4.5 min
- Result: Still beats baseline

The wildcard fix for musl-cross-make module detects existing crossgcc from
cache and skips rebuild entirely (Make Board takes only 27s vs 26 min cold).

Add tests/circle-ci-simulation/ for local cache behavior verification:
- test_cache_hash.sh, test_musl_skip.sh, simulate_cold_cache.sh, etc.

Signed-off-by: Thierry Laurion <insurgo@riseup.net>

2026-04-20 21:10:10 -04:00

4.2 KiB

Raw Permalink Blame History

Development Workflow

Commit Conventions

All commits to linuxboot/heads must be:

git commit -S -s -m "component: short description"

-S — GPG-sign the commit (required; see CONTRIBUTING.md)
-s — add Signed-off-by: trailer for DCO compliance (required; CI enforces this)

Message Format

component: short imperative description (72 chars max)

Optional body explaining the why, not the what.  Wrap at 72 chars.
Reference issues or PRs with #NNN.

Signed-off-by: Your Name <email@example.com>

Subject line: imperative mood ("fix", "add", "remove", not "fixed"/"adds")
Component prefix: the file or subsystem changed (oem-factory-reset, tpmr, gui-init, Makefile, doc, etc.)
Body: explain motivation and context; the diff shows what changed

`Co-Authored-By`

Add a Co-Authored-By: trailer only on commits whose primary content is collaborative documentation (doc/*.md writing). Never add it to code fixes, features, or refactors.

Co-Authored-By: Name <email@example.com>

Documentation: `doc/*.md` vs `heads-wiki`

Location	Purpose	Signing required
`doc/*.md` in this repo	Developer-facing: architecture, patterns, internals, build conventions	Yes (same as all commits)
`linuxboot/heads-wiki`	User-facing: installation, configuration, how-to guides published at osresearch.net	No (lower bar for contribution)

Content should live in doc/*.md when it describes how the code works or how to build/develop. Content should live in heads-wiki when it describes how a user installs, configures, or operates a Heads-equipped device.

Over time, doc/*.md and the wiki may overlap; the canonical user-facing source is the wiki.

For CI internals, cache layering, and workspace-vs-cache behavior, see circleci.md. Use the maintainer checklist there when changing .circleci/config.yml.

Build Artifacts

See build-artifacts.md for the full ROM filename convention. Quick reference:

# Release build (clean tag, e.g. v0.2.1):
heads-x230-v0.2.1.rom

# Development build (any other state):
heads-x230-20260327-202007-my-feature-branch-v0.2.1-42-g0b9d8e4-dirty.rom
#              ^timestamp  ^branch name           ^git describe

The timestamp sorts builds chronologically. The branch name identifies which PR or feature a binary corresponds to without consulting git.

When testing a development build, the ROM filename is your primary build identifier — include it verbatim in bug reports and PR comments.

Testing Checklist

When touching provisioning code (oem-factory-reset, seal-hotpkey, gui-init):

Run a full OEM Factory Reset / Re-Ownership with custom identity (name + email)
Verify gpg --card-status reflects cardholder name and login data
Verify dongle branding shows correctly for the attached device
Verify TOTP/HOTP sealing succeeds after reset
Check /boot signing succeeds with the new GPG key

When touching the Makefile or build system:

Verify dev build filename includes timestamp + branch
Verify a locally-tagged clean commit produces the short filename
Verify .zip package extracts and sha256sum -c passes
If changing .circleci/config.yml, verify the documented cache/workspace behavior in circleci.md still matches the pipeline

Coding Conventions

Shell scripts

All user-visible output through logging helpers: STATUS, STATUS_OK, INFO, NOTE, WARN, ERROR, DEBUG (see logging.md)
Interactive prompts via INPUT only — never raw read
All interactive text output routed through >"${HEADS_TTY:-/dev/stderr}" to avoid interleaving with DO_WITH_DEBUG buffered stdout
Terminology: passphrase for TPM/LUKS secrets; PIN for GPG smartcard (OpenPGP spec); never "password" in user-facing text
Diceware references when prompting users to choose passphrases

UX patterns

See ux-patterns.md for INPUT, STATUS/STATUS_OK, DO_WITH_DEBUG, HEADS_TTY routing, and PIN caching conventions.

4.2 KiB Raw Permalink Blame History