commit 774d9a86fa722c92075c3d351285d0c9bad63f0d Author: LiHaohua Date: Wed May 6 14:56:07 2026 +0800 Bootstrap apt repository structure This is the initial shape of the CardputerZero deb repository. The design follows the GitHub Pages (metadata) + Releases (deb assets) pattern that sibling projects like ryanfortner/box64-debs and AdityaGarg8/t2-ubuntu-repo use successfully — it deliberately avoids Git LFS because the free plan's 1 GB/1 GB storage+bandwidth limits apply to public repos too. Files landing here: - README.md / docs/ARCHITECTURE.md / docs/MAINTAINERS.md explain the flow for users, the design tradeoffs, and the maintainer runbook (including GPG key setup). - .github/workflows/validate-submission.yml runs on pull_request with a read-only token and no secrets, verifying any incoming/*.deb is a valid arm64 package. Safe to run on external contributor PRs. - .github/workflows/publish.yml runs on push to main (after merge). It uploads incoming/*.deb to a rolling "apt-pool" GitHub Release, rebuilds Packages/Release/InRelease with apt-ftparchive, GPG-signs if GPG_PRIVATE_KEY is set (warns loudly otherwise), and publishes the metadata tree to gh-pages. - incoming/czrepo-hello_0.1-1_arm64.deb is a 784-byte sentinel package used to exercise the publish pipeline end-to-end on this very first PR merge. The workflow is intentionally safe-by-default: without a GPG key configured it will still produce a usable (unsigned) apt index so the plumbing can be validated before trusted signing keys are generated. Co-Authored-By: Claude Opus 4.7 (1M context) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..1d5e93e --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,200 @@ +name: Publish apt index + +# Runs after a merge to main. For each .deb file landing in incoming/: +# 1. Upload it as an asset to a rolling "apt-pool" GitHub Release. +# 2. Remove the file from incoming/ and commit. +# 3. Regenerate dists/stable/main/binary-arm64/{Packages,Packages.gz,Release}. +# 4. GPG-sign Release → Release.gpg, and emit clearsigned InRelease. +# 5. Mirror dists/ + KEY.gpg + README to gh-pages for Pages serving. +# +# GPG key management: export a private key with `gpg --export-secret-keys --armor ` +# and store it as secret GPG_PRIVATE_KEY, the passphrase as GPG_PASSPHRASE. +# Public key must also be committed as KEY.gpg at the repo root so clients can verify. + +on: + push: + branches: [main] + paths: + - 'incoming/**' + - 'pool/**' + - 'KEY.gpg' + - '.github/workflows/publish.yml' + workflow_dispatch: + +permissions: + contents: write + +concurrency: + group: publish-apt + cancel-in-progress: false # don't drop a half-written index + +jobs: + publish: + runs-on: ubuntu-24.04 + env: + POOL_TAG: apt-pool + POOL_URL: https://github.com/${{ github.repository }}/releases/download/apt-pool + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Install tooling + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends dpkg-dev apt-utils gnupg + + - name: Enumerate incoming/*.deb + id: inc + run: | + shopt -s nullglob + files=(incoming/*.deb) + if [ ${#files[@]} -eq 0 ]; then + echo "No .deb in incoming/; will still refresh metadata." + echo "count=0" >> "$GITHUB_OUTPUT" + exit 0 + fi + printf 'Incoming:\n'; printf ' %s\n' "${files[@]}" + { + echo 'files<> "$GITHUB_OUTPUT" + echo "count=${#files[@]}" >> "$GITHUB_OUTPUT" + + - name: Ensure rolling release exists + if: steps.inc.outputs.count != '0' + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + if ! gh release view "$POOL_TAG" >/dev/null 2>&1; then + gh release create "$POOL_TAG" \ + --title "Apt pool" \ + --notes "Rolling release; holds every .deb the apt index references." + fi + + - name: Upload incoming/*.deb to release + stage into pool/ + if: steps.inc.outputs.count != '0' + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + mkdir -p pool/main + while IFS= read -r f; do + [ -z "$f" ] && continue + name=$(basename "$f") + echo "-- uploading $name" + gh release upload "$POOL_TAG" "$f" --clobber + # Keep a copy in pool/ ONLY for metadata building; removed before + # the final commit to avoid bloating git with .deb blobs. + cp "$f" "pool/main/$name" + # Clear incoming/ entry. + rm "$f" + done <<< "${{ steps.inc.outputs.files }}" + + - name: Fetch already-released .debs into pool/ for full index + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + mkdir -p pool/main + # If the pool release already holds assets from previous runs, pull + # them all down so apt-ftparchive can index the complete set. + if gh release view "$POOL_TAG" >/dev/null 2>&1; then + gh release download "$POOL_TAG" --dir pool/main --pattern '*.deb' --clobber || true + fi + ls -la pool/main || true + + - name: Generate Packages / Release + run: | + set -euo pipefail + cd "$GITHUB_WORKSPACE" + mkdir -p dists/stable/main/binary-arm64 + + # Packages: scan pool/ with file paths rewritten to point at Release assets. + # dpkg-scanpackages outputs "Filename: pool/main/foo.deb" — rewrite to the + # Release URL so apt downloads from there, not from Pages. + apt-ftparchive \ + -o APT::FTPArchive::Release::Origin="CardputerZero" \ + -o APT::FTPArchive::Release::Label="CardputerZero" \ + -o APT::FTPArchive::Release::Suite="stable" \ + -o APT::FTPArchive::Release::Codename="stable" \ + -o APT::FTPArchive::Release::Architectures="arm64" \ + -o APT::FTPArchive::Release::Components="main" \ + packages pool/main \ + | sed -E "s|^Filename: pool/main/|Filename: releases/download/${POOL_TAG}/|" \ + > dists/stable/main/binary-arm64/Packages + + gzip -kf9 dists/stable/main/binary-arm64/Packages + + apt-ftparchive \ + -o APT::FTPArchive::Release::Origin="CardputerZero" \ + -o APT::FTPArchive::Release::Label="CardputerZero" \ + -o APT::FTPArchive::Release::Suite="stable" \ + -o APT::FTPArchive::Release::Codename="stable" \ + -o APT::FTPArchive::Release::Architectures="arm64" \ + -o APT::FTPArchive::Release::Components="main" \ + release dists/stable \ + > dists/stable/Release + + - name: GPG sign Release → Release.gpg + InRelease + if: env.HAVE_GPG == '1' + env: + HAVE_GPG: ${{ secrets.GPG_PRIVATE_KEY != '' && '1' || '0' }} + GPG_PRIVATE_KEY: ${{ secrets.GPG_PRIVATE_KEY }} + GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }} + run: | + set -euo pipefail + echo "$GPG_PRIVATE_KEY" | gpg --batch --import + KEYID=$(gpg --list-secret-keys --with-colons | awk -F: '/^sec:/ {print $5; exit}') + echo "Signing with $KEYID" + + rm -f dists/stable/Release.gpg dists/stable/InRelease + echo "$GPG_PASSPHRASE" | gpg --batch --yes --pinentry-mode loopback \ + --passphrase-fd 0 --local-user "$KEYID" \ + -abs -o dists/stable/Release.gpg dists/stable/Release + echo "$GPG_PASSPHRASE" | gpg --batch --yes --pinentry-mode loopback \ + --passphrase-fd 0 --local-user "$KEYID" \ + --clearsign -o dists/stable/InRelease dists/stable/Release + + - name: Skip signing (no key configured) + if: env.HAVE_GPG != '1' + env: + HAVE_GPG: ${{ secrets.GPG_PRIVATE_KEY != '' && '1' || '0' }} + run: | + echo "::warning ::GPG_PRIVATE_KEY secret not set — publishing UNSIGNED index." + echo "apt clients will need [trusted=yes] in sources.list until the key is configured." + + - name: Clean pool/ before committing (we don't want .deb in git) + run: rm -rf pool/main/*.deb + + - name: Commit metadata back to main + run: | + set -euo pipefail + git config user.name "cardputer-repo-bot" + git config user.email "bot@users.noreply.github.com" + git add -A + if git diff --cached --quiet; then + echo "No metadata changes to commit." + exit 0 + fi + git commit -m "ci: refresh apt index ($(date -u +%Y-%m-%dT%H:%M:%SZ))" + git push origin HEAD:main + + - name: Publish to gh-pages + run: | + set -euo pipefail + staging=$(mktemp -d) + cp -r dists "$staging/" + [ -f KEY.gpg ] && cp KEY.gpg "$staging/" + cp README.md "$staging/index.md" + + cd "$staging" + git init -q -b gh-pages + git config user.name "cardputer-repo-bot" + git config user.email "bot@users.noreply.github.com" + git remote add origin "https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git" + git add -A + git commit -q -m "publish apt index" + git push -qf origin gh-pages diff --git a/.github/workflows/validate-submission.yml b/.github/workflows/validate-submission.yml new file mode 100644 index 0000000..4dca9fd --- /dev/null +++ b/.github/workflows/validate-submission.yml @@ -0,0 +1,97 @@ +name: Validate submission + +# Safe PR validation: no secrets, read-only token. Checks that files added +# under incoming/ are well-formed arm64 .deb packages. Signing + publishing +# happens later in publish.yml after merge. + +on: + pull_request: + paths: + - 'incoming/**' + - '.github/workflows/validate-submission.yml' + +permissions: + contents: read + +jobs: + validate: + runs-on: ubuntu-24.04 + steps: + - name: Checkout PR head + uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Enumerate new debs under incoming/ + id: enumerate + run: | + set -euo pipefail + git fetch origin main + base=$(git merge-base origin/main HEAD) + added=$(git diff --name-only --diff-filter=A "$base"...HEAD -- 'incoming/*.deb' || true) + if [ -z "$added" ]; then + echo "No new .deb files under incoming/ in this PR." + echo "files=" >> "$GITHUB_OUTPUT" + exit 0 + fi + echo "Found:" + printf ' %s\n' $added + { + echo 'files<> "$GITHUB_OUTPUT" + + - name: Install tooling + if: steps.enumerate.outputs.files != '' + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends dpkg-dev + + - name: Inspect each .deb + if: steps.enumerate.outputs.files != '' + shell: bash + run: | + set -euo pipefail + fail=0 + while IFS= read -r f; do + [ -z "$f" ] && continue + echo "::group::$f" + if ! [ -f "$f" ]; then + echo "::error file=$f::file does not exist" + fail=1 + echo "::endgroup::" + continue + fi + # 1. Must be a valid .deb (dpkg-deb fails loudly otherwise). + if ! dpkg-deb -I "$f" > /tmp/ctrl.txt 2>&1; then + echo "::error file=$f::dpkg-deb -I failed:" + cat /tmp/ctrl.txt + fail=1 + echo "::endgroup::" + continue + fi + cat /tmp/ctrl.txt + # 2. Architecture must be arm64. + arch=$(dpkg-deb --field "$f" Architecture) + if [ "$arch" != "arm64" ]; then + echo "::error file=$f::Architecture=$arch, expected arm64" + fail=1 + fi + # 3. Filename convention: __arm64.deb + base=$(basename "$f") + if [[ ! "$base" =~ ^[a-z0-9][a-z0-9._+-]*_[A-Za-z0-9.~+:-]+_arm64\.deb$ ]]; then + echo "::warning file=$f::filename '$base' does not match __arm64.deb" + fi + # 4. Size sanity: warn if > 50 MiB. + size_mb=$(( $(stat -c%s "$f") / 1024 / 1024 )) + if [ "$size_mb" -gt 50 ]; then + echo "::warning file=$f::${size_mb} MiB — large packages slow apt update" + fi + echo "::endgroup::" + done <<< "${{ steps.enumerate.outputs.files }}" + exit $fail + + - name: Summary + if: steps.enumerate.outputs.files == '' + run: echo "No .deb additions in this PR — nothing to validate." diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..d4205ef --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +# Never commit raw .deb to git — they live as Release assets, not in the tree. +# Exception: incoming/ is the PR landing zone (small, cleared on merge by CI). +pool/main/*.deb +*.deb +!incoming/*.deb +!incoming/.gitkeep diff --git a/README.md b/README.md new file mode 100644 index 0000000..a00c0c2 --- /dev/null +++ b/README.md @@ -0,0 +1,75 @@ +# CardputerZero Repository + +The official Debian `.deb` repository for [M5 CardputerZero](https://docs.m5stack.com/) +applications — hosted on GitHub, served via GitHub Pages. + +## Quick start (on the device) + +```bash +# Import the signing key +curl -fsSL https://m5stack.github.io/CardputerZeroRepository/KEY.gpg \ + | sudo tee /etc/apt/trusted.gpg.d/cardputer.asc > /dev/null + +# Add the repository +echo 'deb [arch=arm64] https://m5stack.github.io/CardputerZeroRepository stable main' \ + | sudo tee /etc/apt/sources.list.d/cardputer.list + +sudo apt update +sudo apt install +``` + +## How it works + +- **Metadata** (`dists/stable/main/binary-arm64/Packages*`, `Release`, `InRelease`) + lives on the `main` branch and is republished to `gh-pages` on every push. This + is small text (~KB per app). +- **`.deb` binaries** live as **GitHub Release assets**, not in the git tree. The + `Packages` index points to `https://github.com/m5stack/CardputerZeroRepository/releases/download//.deb`. + This avoids LFS quotas entirely. +- **Signing** happens inside GitHub Actions using a GPG key stored as a repo secret. + The public key is committed as `KEY.gpg` so clients can verify `InRelease`. +- **Submissions** come in as Pull Requests containing an uploaded `.deb` under + `incoming/`. The `validate-submission.yml` workflow runs on PR without + secrets (safe). On merge, `publish.yml` moves the file to a Release, rebuilds + the index, signs it, and pushes to `gh-pages`. + +## Architecture rationale + +Why GitHub Pages + Releases and **not** LFS? LFS on free plan is 1 GB storage / +1 GB bandwidth per month — and **bandwidth counts even for public repos**. The +Pages+Releases split avoids LFS entirely; see `docs/ARCHITECTURE.md` for the +full writeup. + +## Submission flow + +Developers either: + +1. **`czdev upload `** from CardputerZero-AppBuilder — opens a PR in + this repo with the `.deb` dropped under `incoming/`. +2. **Manual PR** — drop a `.deb` into `incoming/`, open a PR. CI validates + dpkg metadata + architecture + filename. Maintainer reviews, merges. + +Auth / signing for submitters is not wired yet — maintainer merge gates the +publication. + +## Layout + +``` +CardputerZeroRepository/ +├── dists/stable/main/binary-arm64/ # apt metadata (Packages, Release, InRelease) +├── pool/main/ # reserved; small debs may land here later +├── incoming/ # PR landing zone, emptied on merge +├── KEY.gpg # public signing key +└── .github/workflows/ + ├── validate-submission.yml # PR safety: verify deb format only + └── publish.yml # on merge to main: release + reindex + sign +``` + +## Status + +- [x] Repo structure bootstrapped (this PR) +- [x] `validate-submission.yml` — checks deb header, architecture=arm64 +- [x] `publish.yml` — builds Packages/Release, signs, pushes to gh-pages +- [ ] GPG signing key added as secret (see `docs/MAINTAINERS.md`) +- [ ] GitHub Pages enabled on `gh-pages` branch +- [ ] `czdev upload` subcommand wired to this flow (CardputerZero-AppBuilder) diff --git a/dists/stable/main/binary-arm64/.gitkeep b/dists/stable/main/binary-arm64/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..386a9e6 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,89 @@ +# CardputerZeroRepository architecture + +Design notes and tradeoffs for hosting a Debian `.deb` repository on a single +public GitHub repo. + +## Why this shape + +### Not LFS + +GitHub Large File Storage on the free plan is: + +| | Free plan | +|---|---| +| Single file | 2 GB | +| Total storage | **1 GB** | +| Bandwidth / month | **1 GB** | +| Public repo exempt from bandwidth? | **No** | + +For an apt repo with 100–500 `.deb` files at 1–20 MB each and <10k downloads +per month, LFS storage would be exhausted before any download traffic, and +the first few dozen `apt install` calls would exceed the monthly bandwidth +quota. Data packs cost $5/month per 50 GB. + +### GitHub Pages + Releases + +- **Pages** serves the `dists/.../Packages*` + `Release` + `InRelease` text + files. These are small (KB per package). Pages' soft bandwidth limit is + 100 GB/month, comfortable for `apt update` traffic. +- **Releases** host the `.deb` binaries as assets. Asset size limit is 2 GB, + no documented hard quota on total asset storage, and release bandwidth is + tracked separately from LFS. +- `Packages` indices point to `https://github.com/OWNER/REPO/releases/download//.deb`, + so `apt` downloads the binary directly from Releases. + +### Prior art + +- [`AdityaGarg8/t2-ubuntu-repo`](https://github.com/AdityaGarg8/t2-ubuntu-repo) + — flat gh-pages layout, `apt-ftparchive` + GPG sign in Actions, no LFS. +- [`ryanfortner/box64-debs`](https://github.com/ryanfortner/box64-debs) + — 4 years of daily CI commits, nightly arm64 builds, no LFS, repo stays + under 100 MB. + +Both use `apt-ftparchive` + `dpkg-scanpackages` + `crazy-max/ghaction-import-gpg`. + +## Workflow separation (security-critical) + +- **`validate-submission.yml`** runs on `pull_request`. No secrets available, + `GITHUB_TOKEN` is read-only. Can build, test, inspect. Cannot sign or push. +- **`publish.yml`** runs on `push` to `main` (i.e. after merge). Has access + to `secrets.GPG_PRIVATE_KEY`, can push to `gh-pages` branch, can move debs + to Releases. + +**Never use `pull_request_target` with `checkout` of the PR head.** That +combination grants secrets to arbitrary PR code — multiple public projects +have been pwned this way (see +[GitHub Security Lab: "Preventing pwn requests"](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)). + +## Repository size budget + +Assuming 300 apps, weekly updates, metadata only in git: + +- `Packages` text: ~1 KB per app × 300 = 300 KB +- `Packages.gz`: compressed, ~100 KB total +- `Release` / `InRelease`: small, ~5 KB +- `KEY.gpg`: 3 KB +- CI logs accumulate in Actions (not in repo) + +Repository should stay well under 100 MB across years of history; deb assets +live in Releases and don't count toward repo size. + +## Public-repo Actions economics + +- **Free unlimited minutes** for public repos (official). +- **Concurrency**: ~20 concurrent jobs free tier. +- **Job timeout**: 6 hours. +- **Workflow timeout**: 35 days. + +A single PR validation completes in <2 minutes; a full re-sign on merge in +<3 minutes. Room to grow. + +## Upgrade path + +If this repo hits growth that strains GitHub: + +1. **Multi-repo split** — one repo per component (apps / SDKs / firmware). +2. **Cloudflare R2** — egress is free; mirror `.deb` assets there while + keeping this repo as the source of truth. +3. **deb-s3** — full migration to S3-backed hosting; minimal client impact + since the `sources.list` URL changes but signing/layout do not. diff --git a/docs/MAINTAINERS.md b/docs/MAINTAINERS.md new file mode 100644 index 0000000..cb85e8a --- /dev/null +++ b/docs/MAINTAINERS.md @@ -0,0 +1,82 @@ +# Maintainers runbook + +## One-time setup (before first real publication) + +### 1. Generate a GPG signing key + +On a trusted machine (not the CI runner): + +```bash +gpg --batch --gen-key < KEY.gpg + +# Export the private key (store as secret, never commit) +gpg --armor --export-secret-keys "$FPR" > gpg-private.asc +``` + +Add `KEY.gpg` to the repo (normal commit). Set repo secrets: + +- `GPG_PRIVATE_KEY` = contents of `gpg-private.asc` +- `GPG_PASSPHRASE` = passphrase if you used one (leave unset if `%no-protection`) + +Shred `gpg-private.asc` once the secret is saved. + +### 2. Enable GitHub Pages + +- Settings → Pages → Source: `gh-pages` branch, `/` root. +- The first `publish.yml` run will create the branch; rerun the workflow if + Settings doesn't offer `gh-pages` yet. + +### 3. Flip the Pages URL into README + +Once the site is live (check `https://.github.io//`), add its URL +to `README.md` so users can copy-paste the `sources.list` line. + +## Accepting a submission + +1. CI must be green on the PR (`validate-submission.yml`). +2. Inspect the `.deb` metadata in the PR diff (GitHub renders `dpkg-deb -I` + output from the action logs). +3. Merge with "Squash and merge". +4. `publish.yml` runs automatically: + - uploads the `.deb` to the `apt-pool` release, + - removes it from `incoming/`, + - rebuilds `Packages` / `Release` / `InRelease`, + - pushes metadata to `gh-pages`. + +## Removing a package + +1. `gh release delete-asset apt-pool .deb` (or via UI). +2. `gh workflow run publish.yml` to rebuild the index without it. + +## Rotating the GPG key + +1. Generate a new key, commit the new `KEY.gpg`. +2. Update `GPG_PRIVATE_KEY` + `GPG_PASSPHRASE` secrets. +3. Re-run `publish.yml`; clients will see a new `InRelease` signature. +4. Announce the key change — users need to re-import `KEY.gpg`. + +## Monitoring + +- GitHub → Insights → Traffic: watch Pages bandwidth. +- GitHub → Releases → apt-pool: sum of asset sizes = total deb storage. +- If the repo crosses 1 GB: audit `pool/` (should be empty in committed tree — + a trailing `.deb` there means `publish.yml` didn't clean up). +- If Pages bandwidth gets close to 100 GB/month: plan migration to Cloudflare + R2 (see `ARCHITECTURE.md` — apt client impact is only the URL change). diff --git a/incoming/.gitkeep b/incoming/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/incoming/czrepo-hello_0.1-1_arm64.deb b/incoming/czrepo-hello_0.1-1_arm64.deb new file mode 100644 index 0000000..e2648aa Binary files /dev/null and b/incoming/czrepo-hello_0.1-1_arm64.deb differ diff --git a/pool/main/.gitkeep b/pool/main/.gitkeep new file mode 100644 index 0000000..e69de29