31 Commits

Author SHA1 Message Date
Igor Pecovnik
6d2d2e97a8 Fix repository management script for parallel execution
- Remove redundant run_cmd function, use run_aptly consistently
- Include EOS releases in repository processing (remove -ve 'eos' filter)
- Fix showall command to auto-discover repos from isolated databases
- Fix source file preservation in parallel mode (keep sources for workers)
- Fix snapshot management to handle published snapshots correctly
- Fix drop_unsupported_releases to only drop unsupported, not all repos
- Remove isolated DB cleanup that caused data loss on each run
- Remove inappropriate sudo usage from html and date commands
- Enable FORCE_PUBLISH by default for better workflow
- Improve common snapshot creation in isolated mode
- Add isolated database support to showall function

Signed-off-by: Igor Pecovnik <igor@armbian.com>
2026-01-05 01:48:26 +01:00
Igor Pecovnik
e70fab2a19 Revert few Rabbit's suggestions that broke repository management 2026-01-02 20:46:16 +01:00
Igor
4bba1d3c5e Repo tools: add parallel repository generation support (#9150)
**Repository tools: enable parallel repo generation and refactor repo.sh**

Add support for parallel repository generation using GitHub Actions, allowing
multiple workers to build different releases concurrently without database
locking conflicts.

### Highlights
- Add `-R/--single-release`, `update-main`, and `merge` for CI-level parallelism
- Use isolated per-release databases; remove local `-j/--parallel-jobs`
- Sign all Release files; optimize signing order
- Add `extract-repo.sh` for extracting and reorganizing packages from repositories
- Replace file-based logging with syslog (`logger`), removing sudo requirement
- Improve error handling, signing logic, and robustness
- Refactor repo.sh for better readability and documentation
- Apply CodeRabbit-recommended fixes

Signed-off-by: Igor Pecovnik <igor@armbian.com>
2026-01-02 16:48:06 +01:00
Jianfeng Liu
f17caa7104 repo: add loong64 support 2025-08-05 15:12:06 +02:00
Igor
028cdb9c39 Repository signing: add support for dual signing (#8320)
* Repository signing: add support for dual signing
2025-06-23 17:25:48 +02:00
Igor Pecovnik
0999ebfd69 Repository management: disable broken and not needed functionality Acquire-By-Hash
- faster repo generation
- cleaner repository
2024-12-13 11:46:07 +01:00
Igor Pecovnik
9041c08979 Implement config driven BSP package modification to set last good kernel for a single device
This will prevent upgrading to higher kernels which are known to be broken.
2024-08-24 15:36:19 +02:00
Igor Pecovnik
c91668e645 Cosmetic fix: do not show repository keys that doesn't exits 2024-08-03 17:51:52 +02:00
Igor Pecovnik
c958cdc684 Repo management: drop unused component 2024-08-03 16:09:15 +02:00
The-going
c4d458e2c6 mk_format_patch: fix shellcheck SC2045 (error) 2024-08-02 17:21:36 +02:00
The-going
82225ac5a1 mk_format_patch: Copy only those patches that have changes in chunks 2024-08-02 17:21:36 +02:00
Igor Pecovnik
239f3a5dc3 Repo management: rework to increase reliability and speed
- introduce common repo so we only add packages once
- publish via snapshots to increase reliability
- overwrite packages in case of conflicts
2024-07-25 16:42:23 +02:00
Igor Pecovnik
ae469d6503 Change the way repository is getting updated 2024-05-10 13:25:27 +01:00
Igor Pecovnik
abd4f458cc Unset hardcoded defaults from repo management tool 2024-03-07 12:09:29 +01:00
Igor Pecovnik
2da50b4172 Drop repository for unsupported Releases 2024-03-02 15:05:09 +01:00
Paolo Sabatino
562d96128b consolidate rk322x and rockchip 32 bit families
* merge patches from rockchip and rk322x families for current and edge
   kernels
 * adjust patches for tinkerboard to remove some cruft and overclocking
 * rework kernel configs
 * fix rk322x dmc to avoid lockup on rk3288
 * migrate rockchip-6.6 into patch series, rename all patches with
   more understandable names
 * add gen-series.sh script in tools directory (it is a naive tool to
   create patch series when you don't want to rebase everything)
2023-12-26 16:45:30 +01:00
Gunjan Gupta
58af873a43 Add extension to perform major kernel upgrade for allwinner kernels 2023-12-22 22:02:04 +05:30
Igor Pecovnik
39ee6545d9 Repository management - change control filename 2023-09-17 09:25:34 +02:00
Igor
f3736c990a Add functionality to delete packages from a list
Signed-off-by: Igor <igor@armbian.com>
2023-08-18 13:39:24 +02:00
Igor Pecovnik
c2f213a6f6 Internal: repository management will run on same host
Move executing check from the script
2023-08-12 09:59:53 +02:00
Igor
13ed5c0d62 Update repo 2023-08-10 22:43:22 +02:00
Igor
e492d3eeed Improve repository management
Signed-off-by: Igor <igor@armbian.com>
2023-08-10 22:43:22 +02:00
Ricardo Pardini
b92575381a pipeline: inventory all board vars; add not-eos-with-video; introduce TARGETS_FILTER_INCLUDE
> How to use:
>
> `./compile.sh inventory` - does just the board inventory; look for output in `output/info`
>
> `./compile.sh targets-dashboard` - does inventory, targets compositing, and images info; look for output in `output/info`, read the instructions output by the command if you want to load the OpenSearch dashboards.
>
> `./compile.sh targets` - does the full targets compositing and artifacts, look for output in `output/info`
>
> If you don't have a `userpatches/targets.yaml`, _one will be provided for you_ defaulting to Jammy minimal CLI
> and Jammy xfce desktop, for all boards in all branches. You can pass filters via `TARGETS_FILTER_INCLUDE=...` to narrow.
>

- board JSON inventory:
  - more generic regex parsing of variables from board files:
    - all top-level (non-indented) variables are parsed and included in the JSON board inventory
    - this allows us to add new variables to the board files without having to update the parser
    - variables can be bare, `export` or `declare -g`, but **_must_ be quoted** (single or double) and UPPER_CASE
  - some special treatment for certain variables:
    - `KERNEL_TARGET` is parsed as a _comma-separated_ list of valid BRANCH'es
    - `BOARD_MAINTAINER` is parsed as _space-separated_ list of valid maintainer GH usernames as `BOARD_MAINTAINERS: [...]` in the JSON
      - script complains if `BOARD_MAINTAINER` is not set in core boards. Empty is still allowed.
    - `HAS_VIDEO_OUTPUT="no"` causes `BOARD_HAS_VIDEO: false` in the JSON (for desktop-only inventorying, see below)
- introduce `not-eos-with-video` in `items-from-inventory` at the targets compositor
  - the same as `not-eos`, but with added `BOARD_HAS_VIDEO: true` filter, see above
- introduce `TARGETS_FILTER_INCLUDE` for targets compositor
  - this filters the targets _after_ compositing (but before getting image info), based on the board inventory data
  - it's a comma-separated list of `key:value` pairs, which are OR-ed together
  - new virtual info `BOARD_SLASH_BRANCH` post-compositing inventory for filtering of a specific BOARD/BRANCH combo (e.g. `odroidhc4/edge`)
  - some interesting possible filters:
    - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4"`: _only_ build a single board, all branches. JIRA [AR-1806]
    - `TARGETS_FILTER_INCLUDE="BOARD_SLASH_BRANCH:odroidhc4/current"`: _only_ build a single board/branch combo
    - `TARGETS_FILTER_INCLUDE="BOARD:odroidhc4,BOARD:odroidn2"`: _only_ build _two_ boards, all branches.
    - `TARGETS_FILTER_INCLUDE="BOARD_MAINTAINERS:rpardini"`: build all boards and branches where rpardini is a maintainer
    - `TARGETS_FILTER_INCLUDE="BOARDFAMILY:rockchip64"`: build all boards and branches in the rockchip64 family
  - image-info-only variables like `LINUXFAMILY` is **not** available for filtering at this stage
- rename `config/templates` `targets-all-cli.yaml` to `targets-default.yaml`
    - this is used when no `userpatches/targets.yaml` is found
    - new default includes all boards vs branches for non-EOS boards
        - also desktop for all boards that _don't_ have `HAS_VIDEO_OUTPUT='no``
- introduce simplified `targets-dashboard` CLI:
  - does only inventory, compositing, and image info, but not artifact reducing, etc.
  - ignore desktop builds in the OpenSearch indexer
  - update the OpenSearch Dashboards, including new information now available
- invert the logic used for `CLEAN_INFO` and `CLEAN_MATRIX`
    - defaults to `yes` now, so new users/CI don't get hit by stale caches by default
    - repo pipeline CLI stuff is usually run on saved/restored artifacts for `output/info`, so don't clean by default via the CLI
2023-07-26 15:15:02 +02:00
Ricardo Pardini
1fc13a57a7 armbian-next: json-info: cleanup info/json/csv/opensearch, add logging, add some very basic instructions; add Kibana dashboards & docker-compose to bring it OS+Kibana up 2023-02-18 07:45:27 -03:00
Konstantin Litvinov
8398dd1116 Added Debian 12 Bookworm. (#4740)
* Added Debian 12 Bookworm.

* Link identical

* Make minimal truly minimal again

Move netplan.io to CLI image only

* Add netplan.io to Bullseye standard CLI

Co-authored-by: Konstantin Litvinov <klitvinov@piesoft.us>
Co-authored-by: Igor <igor@armbian.com>
2023-01-26 10:04:17 +01:00