Bootstrap apt repository structure

This is the initial shape of the CardputerZero deb repository. The design
follows the GitHub Pages (metadata) + Releases (deb assets) pattern that
sibling projects like ryanfortner/box64-debs and AdityaGarg8/t2-ubuntu-repo
use successfully — it deliberately avoids Git LFS because the free plan's
1 GB/1 GB storage+bandwidth limits apply to public repos too.

Files landing here:

- README.md / docs/ARCHITECTURE.md / docs/MAINTAINERS.md explain the flow
  for users, the design tradeoffs, and the maintainer runbook (including
  GPG key setup).
- .github/workflows/validate-submission.yml runs on pull_request with a
  read-only token and no secrets, verifying any incoming/*.deb is a valid
  arm64 package. Safe to run on external contributor PRs.
- .github/workflows/publish.yml runs on push to main (after merge). It
  uploads incoming/*.deb to a rolling "apt-pool" GitHub Release, rebuilds
  Packages/Release/InRelease with apt-ftparchive, GPG-signs if
  GPG_PRIVATE_KEY is set (warns loudly otherwise), and publishes the
  metadata tree to gh-pages.
- incoming/czrepo-hello_0.1-1_arm64.deb is a 784-byte sentinel package
  used to exercise the publish pipeline end-to-end on this very first
  PR merge.

The workflow is intentionally safe-by-default: without a GPG key
configured it will still produce a usable (unsigned) apt index so the
plumbing can be validated before trusted signing keys are generated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
LiHaohua
2026-05-06 14:56:07 +08:00
co-authored by Claude Opus 4.7
commit 774d9a86fa
10 changed files with 549 additions and 0 deletions
+200
View File
@@ -0,0 +1,200 @@
name: Publish apt index
# Runs after a merge to main. For each .deb file landing in incoming/:
# 1. Upload it as an asset to a rolling "apt-pool" GitHub Release.
# 2. Remove the file from incoming/ and commit.
# 3. Regenerate dists/stable/main/binary-arm64/{Packages,Packages.gz,Release}.
# 4. GPG-sign Release → Release.gpg, and emit clearsigned InRelease.
# 5. Mirror dists/ + KEY.gpg + README to gh-pages for Pages serving.
#
# GPG key management: export a private key with `gpg --export-secret-keys --armor <fpr>`
# and store it as secret GPG_PRIVATE_KEY, the passphrase as GPG_PASSPHRASE.
# Public key must also be committed as KEY.gpg at the repo root so clients can verify.
on:
push:
branches: [main]
paths:
- 'incoming/**'
- 'pool/**'
- 'KEY.gpg'
- '.github/workflows/publish.yml'
workflow_dispatch:
permissions:
contents: write
concurrency:
group: publish-apt
cancel-in-progress: false # don't drop a half-written index
jobs:
publish:
runs-on: ubuntu-24.04
env:
POOL_TAG: apt-pool
POOL_URL: https://github.com/${{ github.repository }}/releases/download/apt-pool
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Install tooling
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends dpkg-dev apt-utils gnupg
- name: Enumerate incoming/*.deb
id: inc
run: |
shopt -s nullglob
files=(incoming/*.deb)
if [ ${#files[@]} -eq 0 ]; then
echo "No .deb in incoming/; will still refresh metadata."
echo "count=0" >> "$GITHUB_OUTPUT"
exit 0
fi
printf 'Incoming:\n'; printf ' %s\n' "${files[@]}"
{
echo 'files<<EOF'
printf '%s\n' "${files[@]}"
echo 'EOF'
} >> "$GITHUB_OUTPUT"
echo "count=${#files[@]}" >> "$GITHUB_OUTPUT"
- name: Ensure rolling release exists
if: steps.inc.outputs.count != '0'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
if ! gh release view "$POOL_TAG" >/dev/null 2>&1; then
gh release create "$POOL_TAG" \
--title "Apt pool" \
--notes "Rolling release; holds every .deb the apt index references."
fi
- name: Upload incoming/*.deb to release + stage into pool/
if: steps.inc.outputs.count != '0'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
mkdir -p pool/main
while IFS= read -r f; do
[ -z "$f" ] && continue
name=$(basename "$f")
echo "-- uploading $name"
gh release upload "$POOL_TAG" "$f" --clobber
# Keep a copy in pool/ ONLY for metadata building; removed before
# the final commit to avoid bloating git with .deb blobs.
cp "$f" "pool/main/$name"
# Clear incoming/ entry.
rm "$f"
done <<< "${{ steps.inc.outputs.files }}"
- name: Fetch already-released .debs into pool/ for full index
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
mkdir -p pool/main
# If the pool release already holds assets from previous runs, pull
# them all down so apt-ftparchive can index the complete set.
if gh release view "$POOL_TAG" >/dev/null 2>&1; then
gh release download "$POOL_TAG" --dir pool/main --pattern '*.deb' --clobber || true
fi
ls -la pool/main || true
- name: Generate Packages / Release
run: |
set -euo pipefail
cd "$GITHUB_WORKSPACE"
mkdir -p dists/stable/main/binary-arm64
# Packages: scan pool/ with file paths rewritten to point at Release assets.
# dpkg-scanpackages outputs "Filename: pool/main/foo.deb" — rewrite to the
# Release URL so apt downloads from there, not from Pages.
apt-ftparchive \
-o APT::FTPArchive::Release::Origin="CardputerZero" \
-o APT::FTPArchive::Release::Label="CardputerZero" \
-o APT::FTPArchive::Release::Suite="stable" \
-o APT::FTPArchive::Release::Codename="stable" \
-o APT::FTPArchive::Release::Architectures="arm64" \
-o APT::FTPArchive::Release::Components="main" \
packages pool/main \
| sed -E "s|^Filename: pool/main/|Filename: releases/download/${POOL_TAG}/|" \
> dists/stable/main/binary-arm64/Packages
gzip -kf9 dists/stable/main/binary-arm64/Packages
apt-ftparchive \
-o APT::FTPArchive::Release::Origin="CardputerZero" \
-o APT::FTPArchive::Release::Label="CardputerZero" \
-o APT::FTPArchive::Release::Suite="stable" \
-o APT::FTPArchive::Release::Codename="stable" \
-o APT::FTPArchive::Release::Architectures="arm64" \
-o APT::FTPArchive::Release::Components="main" \
release dists/stable \
> dists/stable/Release
- name: GPG sign Release → Release.gpg + InRelease
if: env.HAVE_GPG == '1'
env:
HAVE_GPG: ${{ secrets.GPG_PRIVATE_KEY != '' && '1' || '0' }}
GPG_PRIVATE_KEY: ${{ secrets.GPG_PRIVATE_KEY }}
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
run: |
set -euo pipefail
echo "$GPG_PRIVATE_KEY" | gpg --batch --import
KEYID=$(gpg --list-secret-keys --with-colons | awk -F: '/^sec:/ {print $5; exit}')
echo "Signing with $KEYID"
rm -f dists/stable/Release.gpg dists/stable/InRelease
echo "$GPG_PASSPHRASE" | gpg --batch --yes --pinentry-mode loopback \
--passphrase-fd 0 --local-user "$KEYID" \
-abs -o dists/stable/Release.gpg dists/stable/Release
echo "$GPG_PASSPHRASE" | gpg --batch --yes --pinentry-mode loopback \
--passphrase-fd 0 --local-user "$KEYID" \
--clearsign -o dists/stable/InRelease dists/stable/Release
- name: Skip signing (no key configured)
if: env.HAVE_GPG != '1'
env:
HAVE_GPG: ${{ secrets.GPG_PRIVATE_KEY != '' && '1' || '0' }}
run: |
echo "::warning ::GPG_PRIVATE_KEY secret not set — publishing UNSIGNED index."
echo "apt clients will need [trusted=yes] in sources.list until the key is configured."
- name: Clean pool/ before committing (we don't want .deb in git)
run: rm -rf pool/main/*.deb
- name: Commit metadata back to main
run: |
set -euo pipefail
git config user.name "cardputer-repo-bot"
git config user.email "bot@users.noreply.github.com"
git add -A
if git diff --cached --quiet; then
echo "No metadata changes to commit."
exit 0
fi
git commit -m "ci: refresh apt index ($(date -u +%Y-%m-%dT%H:%M:%SZ))"
git push origin HEAD:main
- name: Publish to gh-pages
run: |
set -euo pipefail
staging=$(mktemp -d)
cp -r dists "$staging/"
[ -f KEY.gpg ] && cp KEY.gpg "$staging/"
cp README.md "$staging/index.md"
cd "$staging"
git init -q -b gh-pages
git config user.name "cardputer-repo-bot"
git config user.email "bot@users.noreply.github.com"
git remote add origin "https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git"
git add -A
git commit -q -m "publish apt index"
git push -qf origin gh-pages
+97
View File
@@ -0,0 +1,97 @@
name: Validate submission
# Safe PR validation: no secrets, read-only token. Checks that files added
# under incoming/ are well-formed arm64 .deb packages. Signing + publishing
# happens later in publish.yml after merge.
on:
pull_request:
paths:
- 'incoming/**'
- '.github/workflows/validate-submission.yml'
permissions:
contents: read
jobs:
validate:
runs-on: ubuntu-24.04
steps:
- name: Checkout PR head
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Enumerate new debs under incoming/
id: enumerate
run: |
set -euo pipefail
git fetch origin main
base=$(git merge-base origin/main HEAD)
added=$(git diff --name-only --diff-filter=A "$base"...HEAD -- 'incoming/*.deb' || true)
if [ -z "$added" ]; then
echo "No new .deb files under incoming/ in this PR."
echo "files=" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "Found:"
printf ' %s\n' $added
{
echo 'files<<EOF'
printf '%s\n' $added
echo 'EOF'
} >> "$GITHUB_OUTPUT"
- name: Install tooling
if: steps.enumerate.outputs.files != ''
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends dpkg-dev
- name: Inspect each .deb
if: steps.enumerate.outputs.files != ''
shell: bash
run: |
set -euo pipefail
fail=0
while IFS= read -r f; do
[ -z "$f" ] && continue
echo "::group::$f"
if ! [ -f "$f" ]; then
echo "::error file=$f::file does not exist"
fail=1
echo "::endgroup::"
continue
fi
# 1. Must be a valid .deb (dpkg-deb fails loudly otherwise).
if ! dpkg-deb -I "$f" > /tmp/ctrl.txt 2>&1; then
echo "::error file=$f::dpkg-deb -I failed:"
cat /tmp/ctrl.txt
fail=1
echo "::endgroup::"
continue
fi
cat /tmp/ctrl.txt
# 2. Architecture must be arm64.
arch=$(dpkg-deb --field "$f" Architecture)
if [ "$arch" != "arm64" ]; then
echo "::error file=$f::Architecture=$arch, expected arm64"
fail=1
fi
# 3. Filename convention: <pkg>_<version>_arm64.deb
base=$(basename "$f")
if [[ ! "$base" =~ ^[a-z0-9][a-z0-9._+-]*_[A-Za-z0-9.~+:-]+_arm64\.deb$ ]]; then
echo "::warning file=$f::filename '$base' does not match <pkg>_<version>_arm64.deb"
fi
# 4. Size sanity: warn if > 50 MiB.
size_mb=$(( $(stat -c%s "$f") / 1024 / 1024 ))
if [ "$size_mb" -gt 50 ]; then
echo "::warning file=$f::${size_mb} MiB — large packages slow apt update"
fi
echo "::endgroup::"
done <<< "${{ steps.enumerate.outputs.files }}"
exit $fail
- name: Summary
if: steps.enumerate.outputs.files == ''
run: echo "No .deb additions in this PR — nothing to validate."
+6
View File
@@ -0,0 +1,6 @@
# Never commit raw .deb to git — they live as Release assets, not in the tree.
# Exception: incoming/ is the PR landing zone (small, cleared on merge by CI).
pool/main/*.deb
*.deb
!incoming/*.deb
!incoming/.gitkeep
+75
View File
@@ -0,0 +1,75 @@
# CardputerZero Repository
The official Debian `.deb` repository for [M5 CardputerZero](https://docs.m5stack.com/)
applications — hosted on GitHub, served via GitHub Pages.
## Quick start (on the device)
```bash
# Import the signing key
curl -fsSL https://m5stack.github.io/CardputerZeroRepository/KEY.gpg \
| sudo tee /etc/apt/trusted.gpg.d/cardputer.asc > /dev/null
# Add the repository
echo 'deb [arch=arm64] https://m5stack.github.io/CardputerZeroRepository stable main' \
| sudo tee /etc/apt/sources.list.d/cardputer.list
sudo apt update
sudo apt install <appname>
```
## How it works
- **Metadata** (`dists/stable/main/binary-arm64/Packages*`, `Release`, `InRelease`)
lives on the `main` branch and is republished to `gh-pages` on every push. This
is small text (~KB per app).
- **`.deb` binaries** live as **GitHub Release assets**, not in the git tree. The
`Packages` index points to `https://github.com/m5stack/CardputerZeroRepository/releases/download/<tag>/<pkg>.deb`.
This avoids LFS quotas entirely.
- **Signing** happens inside GitHub Actions using a GPG key stored as a repo secret.
The public key is committed as `KEY.gpg` so clients can verify `InRelease`.
- **Submissions** come in as Pull Requests containing an uploaded `.deb` under
`incoming/`. The `validate-submission.yml` workflow runs on PR without
secrets (safe). On merge, `publish.yml` moves the file to a Release, rebuilds
the index, signs it, and pushes to `gh-pages`.
## Architecture rationale
Why GitHub Pages + Releases and **not** LFS? LFS on free plan is 1 GB storage /
1 GB bandwidth per month — and **bandwidth counts even for public repos**. The
Pages+Releases split avoids LFS entirely; see `docs/ARCHITECTURE.md` for the
full writeup.
## Submission flow
Developers either:
1. **`czdev upload <file.deb>`** from CardputerZero-AppBuilder — opens a PR in
this repo with the `.deb` dropped under `incoming/`.
2. **Manual PR** — drop a `.deb` into `incoming/`, open a PR. CI validates
dpkg metadata + architecture + filename. Maintainer reviews, merges.
Auth / signing for submitters is not wired yet — maintainer merge gates the
publication.
## Layout
```
CardputerZeroRepository/
├── dists/stable/main/binary-arm64/ # apt metadata (Packages, Release, InRelease)
├── pool/main/ # reserved; small debs may land here later
├── incoming/ # PR landing zone, emptied on merge
├── KEY.gpg # public signing key
└── .github/workflows/
├── validate-submission.yml # PR safety: verify deb format only
└── publish.yml # on merge to main: release + reindex + sign
```
## Status
- [x] Repo structure bootstrapped (this PR)
- [x] `validate-submission.yml` — checks deb header, architecture=arm64
- [x] `publish.yml` — builds Packages/Release, signs, pushes to gh-pages
- [ ] GPG signing key added as secret (see `docs/MAINTAINERS.md`)
- [ ] GitHub Pages enabled on `gh-pages` branch
- [ ] `czdev upload` subcommand wired to this flow (CardputerZero-AppBuilder)
+89
View File
@@ -0,0 +1,89 @@
# CardputerZeroRepository architecture
Design notes and tradeoffs for hosting a Debian `.deb` repository on a single
public GitHub repo.
## Why this shape
### Not LFS
GitHub Large File Storage on the free plan is:
| | Free plan |
|---|---|
| Single file | 2 GB |
| Total storage | **1 GB** |
| Bandwidth / month | **1 GB** |
| Public repo exempt from bandwidth? | **No** |
For an apt repo with 100500 `.deb` files at 120 MB each and <10k downloads
per month, LFS storage would be exhausted before any download traffic, and
the first few dozen `apt install` calls would exceed the monthly bandwidth
quota. Data packs cost $5/month per 50 GB.
### GitHub Pages + Releases
- **Pages** serves the `dists/.../Packages*` + `Release` + `InRelease` text
files. These are small (KB per package). Pages' soft bandwidth limit is
100 GB/month, comfortable for `apt update` traffic.
- **Releases** host the `.deb` binaries as assets. Asset size limit is 2 GB,
no documented hard quota on total asset storage, and release bandwidth is
tracked separately from LFS.
- `Packages` indices point to `https://github.com/OWNER/REPO/releases/download/<tag>/<file>.deb`,
so `apt` downloads the binary directly from Releases.
### Prior art
- [`AdityaGarg8/t2-ubuntu-repo`](https://github.com/AdityaGarg8/t2-ubuntu-repo)
— flat gh-pages layout, `apt-ftparchive` + GPG sign in Actions, no LFS.
- [`ryanfortner/box64-debs`](https://github.com/ryanfortner/box64-debs)
— 4 years of daily CI commits, nightly arm64 builds, no LFS, repo stays
under 100 MB.
Both use `apt-ftparchive` + `dpkg-scanpackages` + `crazy-max/ghaction-import-gpg`.
## Workflow separation (security-critical)
- **`validate-submission.yml`** runs on `pull_request`. No secrets available,
`GITHUB_TOKEN` is read-only. Can build, test, inspect. Cannot sign or push.
- **`publish.yml`** runs on `push` to `main` (i.e. after merge). Has access
to `secrets.GPG_PRIVATE_KEY`, can push to `gh-pages` branch, can move debs
to Releases.
**Never use `pull_request_target` with `checkout` of the PR head.** That
combination grants secrets to arbitrary PR code — multiple public projects
have been pwned this way (see
[GitHub Security Lab: "Preventing pwn requests"](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)).
## Repository size budget
Assuming 300 apps, weekly updates, metadata only in git:
- `Packages` text: ~1 KB per app × 300 = 300 KB
- `Packages.gz`: compressed, ~100 KB total
- `Release` / `InRelease`: small, ~5 KB
- `KEY.gpg`: 3 KB
- CI logs accumulate in Actions (not in repo)
Repository should stay well under 100 MB across years of history; deb assets
live in Releases and don't count toward repo size.
## Public-repo Actions economics
- **Free unlimited minutes** for public repos (official).
- **Concurrency**: ~20 concurrent jobs free tier.
- **Job timeout**: 6 hours.
- **Workflow timeout**: 35 days.
A single PR validation completes in <2 minutes; a full re-sign on merge in
<3 minutes. Room to grow.
## Upgrade path
If this repo hits growth that strains GitHub:
1. **Multi-repo split** — one repo per component (apps / SDKs / firmware).
2. **Cloudflare R2** — egress is free; mirror `.deb` assets there while
keeping this repo as the source of truth.
3. **deb-s3** — full migration to S3-backed hosting; minimal client impact
since the `sources.list` URL changes but signing/layout do not.
+82
View File
@@ -0,0 +1,82 @@
# Maintainers runbook
## One-time setup (before first real publication)
### 1. Generate a GPG signing key
On a trusted machine (not the CI runner):
```bash
gpg --batch --gen-key <<EOF
%no-protection
Key-Type: RSA
Key-Length: 4096
Subkey-Type: RSA
Subkey-Length: 4096
Name-Real: CardputerZero Repository
Name-Email: repo@m5stack.local
Expire-Date: 3y
%commit
EOF
# Grab the fingerprint
FPR=$(gpg --list-keys --with-colons repo@m5stack.local | awk -F: '/^fpr/ {print $10; exit}')
echo "$FPR"
# Export the public key (commit this file)
gpg --armor --export "$FPR" > KEY.gpg
# Export the private key (store as secret, never commit)
gpg --armor --export-secret-keys "$FPR" > gpg-private.asc
```
Add `KEY.gpg` to the repo (normal commit). Set repo secrets:
- `GPG_PRIVATE_KEY` = contents of `gpg-private.asc`
- `GPG_PASSPHRASE` = passphrase if you used one (leave unset if `%no-protection`)
Shred `gpg-private.asc` once the secret is saved.
### 2. Enable GitHub Pages
- Settings → Pages → Source: `gh-pages` branch, `/` root.
- The first `publish.yml` run will create the branch; rerun the workflow if
Settings doesn't offer `gh-pages` yet.
### 3. Flip the Pages URL into README
Once the site is live (check `https://<owner>.github.io/<repo>/`), add its URL
to `README.md` so users can copy-paste the `sources.list` line.
## Accepting a submission
1. CI must be green on the PR (`validate-submission.yml`).
2. Inspect the `.deb` metadata in the PR diff (GitHub renders `dpkg-deb -I`
output from the action logs).
3. Merge with "Squash and merge".
4. `publish.yml` runs automatically:
- uploads the `.deb` to the `apt-pool` release,
- removes it from `incoming/`,
- rebuilds `Packages` / `Release` / `InRelease`,
- pushes metadata to `gh-pages`.
## Removing a package
1. `gh release delete-asset apt-pool <file>.deb` (or via UI).
2. `gh workflow run publish.yml` to rebuild the index without it.
## Rotating the GPG key
1. Generate a new key, commit the new `KEY.gpg`.
2. Update `GPG_PRIVATE_KEY` + `GPG_PASSPHRASE` secrets.
3. Re-run `publish.yml`; clients will see a new `InRelease` signature.
4. Announce the key change — users need to re-import `KEY.gpg`.
## Monitoring
- GitHub → Insights → Traffic: watch Pages bandwidth.
- GitHub → Releases → apt-pool: sum of asset sizes = total deb storage.
- If the repo crosses 1 GB: audit `pool/` (should be empty in committed tree —
a trailing `.deb` there means `publish.yml` didn't clean up).
- If Pages bandwidth gets close to 100 GB/month: plan migration to Cloudflare
R2 (see `ARCHITECTURE.md` — apt client impact is only the URL change).
View File
Binary file not shown.
View File