armbian-next: the great cli entrypoint (+docker) rewrite; introduce USE_LOCAL_APT_DEB_CACHE replacing apt-cacher-ng

- armbian-next: introduce `USE_LOCAL_APT_DEB_CACHE` (default `=yes`) as alternative/in addition to `apt-cacher-ng` (eg, in Docker)
  - this uses `cache/aptcache/${RELEASE}-${ARCH}` (in the host) for
      - apt cache, by bind-mounting it to `${SDCARD}/var/cache/apt` in the `chroot_sdcard_apt_get()` runner and its usages
      - debootstrap, by passing it `--cache-dir`
  - utility function to help understand what is happening to cache during usage
  - apt itself mantains this cache, removing old packages when new ones are installed. apt does this _by default_
      - introduce `DONT_MAINTAIN_APT_CACHE=yes` to skip out of automatic apt maintenance of apt cache, eg, during `remove`s
      - don't do `apt clean` and such if using local cache, that would clean the cache, not the chroot
  - clean up `install_deb_chroot()` a little, find an unrelated bug there
- WiP: the great cli entrypoint (+docker) rewrite, Phase 6: relaunching structure; re-pass ARMBIAN_BUILD_UUID; use ARMBIAN_COMMAND for log filename; fix for output/logs dir perms
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 4/x; better logging, check & force `DEST_LANG`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 3/x; don't write to stderr in generated Dockerfile
  - it's `drastic red` on non-buildx dockers
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 2/x, logging
- WiP: the great cli entrypoint (+docker) rewrite, Phase 5: cleanups 1/x
  - source configs in a logging section.
  - Docker: silent, fast retries to make sure `docker system df` works
  - shut-up `chown` (no `-v`) output related to  `SET_OWNER_TO_UID`
  - ask user to wait while `DESTIMG` is rsync'ed to `FINALDEST` -- it's potentially very slow
  - use green apple for Mac logging, instead of red apple which might imply error...
- WiP: the great cli entrypoint (+docker) rewrite, Phase 4: run as non-root, maybe-with-Docker
  - introduce `is_docker_ready_to_go()`; if it is, and we're not root, use Docker instead of sudo. <- GOOD IDEA? BAD IDEA? lol
  - introduce `SET_OWNER_TO_UID` var to be passed to Docker/sudo so written files are owned by the launching user, not root.
    - introduce `mkdir_recursive_and_set_uid_owner()` and `reset_uid_owner()` to reset owner based on `SET_OWNER_TO_UID`
    - use it for userpatches files created, logs, and output files, including images and debs.
  - @TODOs ref. `$SUDO_USER` which I think the old version of this?
  - add a lot of @TODOs, ref being able to relaunch something that's not `build` inside Docker, also add/change params and configs and command.
    - initially add `ARMBIAN_DOCKER_RELAUNCH_EXTRA_ARGS`
- WiP: the great cli entrypoint (+docker) rewrite, Phase 3: rpardini is demented, v3
- WiP: the great cli entrypoint (+docker) rewrite, Phase 2: rpardini is demented
- WiP: the great cli entrypoint (+docker) rewrite, Phase 1
- armbian-next: WiP: Docker: actually use the GHA-image as base; pull it every 24hs.
  - using image in my private repo.
  - this has significant speedup to "start building time" on the 1st run
  - move some Linux specific stuff to its own if
  - add comments and todo
- armbian-next: WiP: Docker, high-WiP, beginnings of Armbian mount dict, with linux/darwin preferences
- armbian-next: WiP: Docker, configure `BUILDKIT_COLORS`
- armbian-next: WiP: Docker, make docker image from Dockerfile more compact by flattening layers
- armbian-next: `logging`: add whale indicator if build running under Docker
- armbian-next: WiP: `docker`: working with `bookworm`, `sid`, and `jammy` on Darwin & Linux; works with `bullseye` on Linux only
- armbian-next: WiP: `docker`: force ARMBIAN_RUNNING_IN_CONTAINER both in Dockerfile and passed as `--env`; apt update and install in same layer; back to jammy
- armbian-next: introduce `armbian_is_running_in_container()` and `armbian_is_host_running_systemd()`, replacing `systemd-detect-virt` in multiple spots
- WiP: try with debian:bullseye -- can't detect docker at all
- armbian-next: WiP: 2nd stab at new Docker support; Darwin still works; Linux `docker.io` working
  - gen .dockerignore together with Dockerfile
  - split in funcs
  - hacks for Linux and `/dev/loop` stuff, CONTAINER_COMPAT=yes
  - mac still works, Linux stuff would break it but I if'fed
- armbian-next: the secrets of `CONTAINER_COMPAT` revealed; add size checking to check_loop_device() and avoid retry when `mknod`ing
  - this fails for the right reasons now, causing retries, which are then retried and work ;-)
  - this is related to building under Docker on Linux, using docker.io package (not docker-ce)
- armbian-next: remove `.dockerignore` and add it to `.gitignore`; it's going to be auto-generated
- armbian-next: `.dockerignore`: Docker context should only have minimal files and folders, to speed up Dockerfile build
  - IMPORTANT: `.dockerignore` is going to be generated from now on: so this is the last commit with changes before removal
-  armbian-next: WiP: initial stab at new Docker support; really run the passed cmdline; add Dockerfile to gitignore
-  armbian-next: WiP: initial stab at new Docker support; generate Dockerfile; introduce REQUIREMENTS_DEFS_ONLY
  - uses REQUIREMENTS_DEFS_ONLY
  - works on Docker Desktop on Mac;
  - linux TBA
- armbian-next: don't error out if `.git` not present; other small fixes
- armbian-next: general "work or at least don't misbehave when run on a very bare ubuntu:latest instance"
  - can't assume things, for example:
  - that `sudo` will be available; it might not, and might be already root, no reason to fail
  - that `/etc/timezone` will exist
  - that `systemd-detect-virt` will be available
  - that `git` will be available
  - that `locale-gen` will be available
This commit is contained in:
Ricardo Pardini
2022-10-09 17:58:23 +02:00
parent 2c6751f584
commit d24d3327a8
37 changed files with 1476 additions and 504 deletions

View File

@@ -1,4 +0,0 @@
### output directories
.tmp/
output/
cache/

4
.gitignore vendored
View File

@@ -25,3 +25,7 @@ ubuntu-*-cloudimg-console.log
# Mainly generated by merge tools like 'meld'
*.orig
# Dockerfile and .dockerignore is generated by docker.sh
Dockerfile
.dockerignore

View File

@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash
#
# Copyright (c) 2013-2021 Igor Pecovnik, igor.pecovnik@gma**.com
#
@@ -7,7 +7,7 @@
# warranty of any kind, whether express or implied.
#
# This file is a part of the Armbian build script
# https://github.com/armbian/build/
# https://github.com/armbian/build/
# DO NOT EDIT THIS FILE
# use configuration files like config-default.conf to set the build configuration

View File

@@ -1,151 +0,0 @@
# DO NOT EDIT THIS FILE
#
# This is a Docker launcher file. To set up the configuration, use command line arguments to compile.sh
# or use pass a config file as a parameter ./compile docker [example] BUILD_KERNEL="yes" ...
# Default values for Docker image
CUSTOM_PACKAGES="g++-11-arm-linux-gnueabihf libssl3 qemu"
BASE_IMAGE="ubuntu:jammy"
[[ ! -c /dev/loop-control ]] && display_alert "/dev/loop-control does not exist, image building may not work" "" "wrn"
# second argument can be a build parameter or a config file
# create user accessible directories and set their owner group and permissions
# if they are created from Docker they will be owned by root and require root permissions to change/delete
mkdir -p $SRC/{output,userpatches}
grep -q '^docker:' /etc/group && chgrp --quiet docker $SRC/{output,userpatches}
chmod --quiet g+w,g+s $SRC/{output,userpatches}
VERSION=$(cat $SRC/VERSION)
if grep -q $VERSION <(grep armbian <(docker images)); then
display_alert "Using existed a armbian Docker container"
else
# build a new container based on provided Dockerfile
display_alert "Docker container not found or out of date"
display_alert "Building a Docker container"
if ! docker build --build-arg CUSTOM_PACKAGES="$CUSTOM_PACKAGES" --build-arg BASE_IMAGE=$BASE_IMAGE -t armbian:$VERSION . ; then
STATUS=$?
# Adding a newline, so the alert won't be shown in the same line as the error
echo
display_alert "Docker container build exited with code: " "$STATUS" "err"
exit 1
fi
fi
DOCKER_FLAGS=()
# Running this container in privileged mode is a simple way to solve loop device access issues
# Required for USB FEL or when writing image directly to the block device, when CARD_DEVICE is defined
#DOCKER_FLAGS+=(--privileged)
# add only required capabilities instead (though MKNOD should be already present)
# CAP_SYS_PTRACE is required for systemd-detect-virt in some cases
DOCKER_FLAGS+=(--cap-add=SYS_ADMIN --cap-add=MKNOD --cap-add=SYS_PTRACE)
# mounting things inside the container on Ubuntu won't work without this
# https://github.com/moby/moby/issues/16429#issuecomment-217126586
DOCKER_FLAGS+=(--security-opt=apparmor:unconfined)
# remove resulting container after exit to minimize clutter
# bad side effect - named volumes are considered not attached to anything and are removed on "docker volume prune"
DOCKER_FLAGS+=(--rm)
# pass through loop devices
for d in /dev/loop*; do
DOCKER_FLAGS+=(--device=$d)
done
# accessing dynamically created devices won't work by default
# and --device doesn't accept devices that don't exist at the time "docker run" is executed
# https://github.com/moby/moby/issues/27886
# --device-cgroup-rule requires new Docker version
# Test for --device-cgroup-rule support. If supported, appends it
# Otherwise, let it go and let user know that only kernel and u-boot for you
if docker run --help | grep device-cgroup-rule > /dev/null 2>&1; then
# allow loop devices (not required)
DOCKER_FLAGS+=(--device-cgroup-rule='b 7:* rmw')
# allow loop device partitions
DOCKER_FLAGS+=(--device-cgroup-rule='b 259:* rmw')
# this is an ugly hack, but it is required to get /dev/loopXpY minor number
# for mknod inside the container, and container itself still uses private /dev internally
DOCKER_FLAGS+=(-v /dev:/tmp/dev:ro)
else
display_alert "Your Docker version does not support device-cgroup-rule" "" "wrn"
display_alert "and will be able to create only Kernel and u-boot packages (KERNEL_ONLY=yes)" "" "wrn"
fi
# Expose ports for NFS server inside docker container, required for USB FEL
#DOCKER_FLAGS+=(-p 0.0.0.0:2049:2049 -p 0.0.0.0:2049:2049/udp -p 0.0.0.0:111:111 -p 0.0.0.0:111:111/udp -p 0.0.0.0:32765:32765 -p 0.0.0.0:32765:32765/udp -p 0.0.0.0:32767:32767 -p 0.0.0.0:32767:32767/udp)
# Export usb device for FEL, required for USB FEL
#DOCKER_FLAGS+=(-v /dev/bus/usb:/dev/bus/usb:ro)
# map source to Docker Working dir.
DOCKER_FLAGS+=(-v=$SRC/:/root/armbian/)
# map /tmp to tmpfs
DOCKER_FLAGS+=(--mount type=tmpfs,destination=/tmp)
# mount 2 named volumes - for cacheable data and compiler cache
DOCKER_FLAGS+=(-v=armbian-cache:/root/armbian/cache -v=armbian-ccache:/root/.ccache)
DOCKER_FLAGS+=(-e COLUMNS="`tput cols`" -e LINES="`tput lines`")
# pass other command line arguments like KERNEL_ONLY=yes, KERNEL_CONFIGURE=yes, etc.
# pass "docker-guest" as an additional config name that will be sourced in the container if exists
if [[ $SHELL_ONLY == yes ]]; then
display_alert "Running the container in shell mode" "" "info"
cat <<\EOF
Welcome to the docker shell of Armbian.
To build the whole thing using default profile, run:
./compile.sh
To build the U-Boot only, run:
# Optional: prepare the environment first if you had not run `./compile.sh`
./compile.sh 'prepare_host && compile_sunxi_tools && install_rkbin_tools'
# build the U-Boot only
./compile.sh compile_uboot
If you prefer to use profile, for example, `userpatches/config-my.conf`, try:
./compile.sh my 'prepare_host && compile_sunxi_tools && install_rkbin_tools'
./compile.sh my compile_uboot
EOF
docker run "${DOCKER_FLAGS[@]}" -it --entrypoint /usr/bin/env armbian:$VERSION "$@" /bin/bash
else
display_alert "Running the container" "" "info"
docker run "${DOCKER_FLAGS[@]}" -it armbian:$VERSION "$@"
fi
# Docker error treatment
STATUS=$?
# Adding a newline, so the message won't be shown in the same line as the error
echo
case $STATUS in
0)
# No errors from either Docker or build script
echo
;;
125)
display_alert "Docker command failed, check syntax or version support. Error code: " "$STATUS" "err"
;;
126)
display_alert "Failure when running containerd command. Error code: " "$STATUS" "err"
;;
127)
display_alert "containerd command not found. Error code: " "$STATUS" "err"
;;
137)
display_alert "Container exit from docker stop. Error code: " "$STATUS" "info"
;;
*)
# Build script exited with error, but the error message should have been already printed
echo
;;
esac
# don't need to proceed further on the host
exit $STATUS

View File

@@ -11,20 +11,20 @@
KERNEL_ONLY="" # leave empty to select each time, set to "yes" or "no" to skip dialog prompt
KERNEL_CONFIGURE="" # leave empty to select each time, set to "yes" or "no" to skip dialog prompt
CLEAN_LEVEL="debs,oldcache" # comma-separated list of clean targets:
: # "make-atf" = make clean for ATF, if it is built.
: # "make-uboot" = make clean for uboot, if it is built.
: # "make-kernel" = make clean for kernel, if it is built. very slow.
: # *important*: "make" by itself has disabled, since Armbian knows how to handle Make timestamping now.
: # "debs" = delete packages in "./output/debs" for current branch and family. causes rebuilds, hopefully cached.
: # "alldebs" = delete all packages in "./output/debs", "images" = delete "./output/images",
: # "cache" = delete "./output/cache", "sources" = delete "./sources"
: # "oldcache" = remove old cached rootfs except for the newest 8 files
: # --> "make-atf" = make clean for ATF, if it is built.
: # --> "make-uboot" = make clean for uboot, if it is built.
: # --> "make-kernel" = make clean for kernel, if it is built. very slow.
: # --> "debs" = delete packages in "./output/debs" for current branch and family. causes rebuilds, hopefully cached.
: # --> "alldebs" = delete all packages in "./output/debs", "images" = delete "./output/images",
: # --> "cache" = delete "./output/cache", "sources" = delete "./sources"
: # --> "oldcache" = remove old cached rootfs except for the newest 8 files
: # --> *important*: "make" by itself has disabled, since Armbian knows how to handle Make timestamping now.
REPOSITORY_INSTALL="" # comma-separated list of core modules which will be installed from repository
REPOSITORY_INSTALL="" # comma-separated list of core packages which will be installed from repository instead of built
# "u-boot", "kernel", "bsp", "armbian-config", "armbian-firmware"
# leave empty to build from sources or use local cache
DEST_LANG="en_US.UTF-8" # sl_SI.UTF-8, en_US.UTF-8
# DEST_LANG="en_US.UTF-8" # Example: "sl_SI.UTF-8" Default: "en_US.UTF-8"
# advanced
EXTERNAL_NEW="prebuilt" # compile and install or install prebuilt additional packages

View File

@@ -27,7 +27,7 @@ call_extension_method() {
# Then a sanity check, hook points should only be invoked after the manager has initialized.
if [[ ${initialize_extension_manager_counter} -lt 1 ]]; then
display_alert "Extension problem" "Call to call_extension_method() (in ${BASH_SOURCE[1]- $(get_extension_hook_stracktrace "${BASH_SOURCE[*]}" "${BASH_LINENO[*]}")}) before extension manager is initialized." "err"
display_alert "Extension problem" "Call to call_extension_method() ($*: in ${BASH_SOURCE[1]- $(get_extension_hook_stracktrace "${BASH_SOURCE[*]}" "${BASH_LINENO[*]}")}) before extension manager is initialized." "err"
fi
# With DEBUG_EXTENSION_CALLS, log the hook call. Users might be wondering what/when is a good hook point to use, and this is visual aid.

View File

@@ -0,0 +1,49 @@
function cli_standard_build_pre_run() {
declare -g ARMBIAN_COMMAND_REQUIRE_BASIC_DEPS="yes" # Require prepare_host_basic to run before the command.
# Super early handling. If no command and not root, become root by using sudo. Some exceptions apply.
if [[ "${EUID}" == "0" ]]; then # we're already root. Either running as real root, or already sudo'ed.
display_alert "Already running as root" "great" "debug"
else # not root.
# Pass the current UID to any further relaunchings (under docker or sudo).
ARMBIAN_CLI_RELAUNCH_PARAMS+=(["SET_OWNER_TO_UID"]="${EUID}") # add params when relaunched under docker
# We've a few options.
# 1) We could check if Docker is working, and do everything under Docker. Users who can use Docker, can "become" root inside a container.
# 2) We could ask for sudo (which _might_ require a password)...
# @TODO: GitHub actions can do both. Sudo without password _and_ Docker; should we prefer Docker? Might have unintended consequences...
if is_docker_ready_to_go; then
# add the current user EUID as a parameter when it's relaunched under docker. SET_OWNER_TO_UID="${EUID}"
display_alert "Trying to build, not root, but Docker is ready to go" "delegating to Docker" "debug"
ARMBIAN_CHANGE_COMMAND_TO="docker"
return 0
fi
# check if we're on Linux via uname. if not, refuse to do anything.
if [[ "$(uname)" != "Linux" ]]; then
display_alert "Not running on Linux; Docker is not available" "refusing to run" "err"
exit 1
fi
display_alert "This script requires root privileges; Docker is unavailable" "trying to use sudo" "wrn"
declare -g ARMBIAN_CLI_RELAUNCH_ARGS=()
produce_relaunch_parameters # produces ARMBIAN_CLI_RELAUNCH_ARGS
sudo --preserve-env "${SRC}/compile.sh" "${ARMBIAN_CLI_RELAUNCH_ARGS[@]}" # MARK: relaunch done here!
display_alert "AFTER SUDO!!!" "AFTER SUDO!!!" "warn"
fi
}
function cli_standard_build_run() {
# @TODO: then many other interesting possibilities like a REPL, which we lost somewhere along the way. docker-shell?
# configuration etc - it initializes the extension manager
prepare_and_config_main_build_single
# Allow for custom user-invoked functions, or do the default build.
if [[ -z $1 ]]; then
main_default_build_single
else
# @TODO: rpardini: check this with extensions usage?
eval "$@"
fi
}

View File

@@ -0,0 +1,9 @@
function cli_config_dump_pre_run() {
declare -g CONFIG_DEFS_ONLY='yes'
}
function cli_config_dump_run() {
# configuration etc - it initializes the extension manager
do_capturing_defs prepare_and_config_main_build_single # this sets CAPTURED_VARS
echo "${CAPTURED_VARS}" # to stdout!
}

View File

@@ -0,0 +1,37 @@
function cli_docker_pre_run() {
if [[ "${DOCKERFILE_GENERATE_ONLY}" == "yes" ]]; then
display_alert "Dockerfile generation only" "func cli_docker_pre_run" "debug"
return 0
fi
# make sure we're not _ALREADY_ running under docker... otherwise eternal loop?
if [[ "${ARMBIAN_RUNNING_IN_CONTAINER}" == "yes" ]]; then
display_alert "wtf" "asking for docker... inside docker; turning to build command" "warn"
# @TODO: wrong, what if we wanna run other stuff inside Docker? not build?
ARMBIAN_CHANGE_COMMAND_TO="build"
fi
}
function cli_docker_run() {
LOG_SECTION="docker_cli_prepare" do_with_logging docker_cli_prepare
if [[ "${DOCKERFILE_GENERATE_ONLY}" == "yes" ]]; then
display_alert "Dockerfile generated" "exiting" "info"
exit 0
fi
# Force showing logs here while bulding Dockerfile.
SHOW_LOG=yes LOG_SECTION="docker_cli_build_dockerfile" do_with_logging docker_cli_build_dockerfile
LOG_SECTION="docker_cli_prepare_launch" do_with_logging docker_cli_prepare_launch
ARMBIAN_CLI_RELAUNCH_PARAMS+=(["SET_OWNER_TO_UID"]="${EUID}") # fix the owner of files to our UID
ARMBIAN_CLI_RELAUNCH_PARAMS+=(["ARMBIAN_BUILD_UUID"]="${ARMBIAN_BUILD_UUID}") # pass down our uuid to the docker instance
ARMBIAN_CLI_RELAUNCH_PARAMS+=(["SKIP_LOG_ARCHIVE"]="yes") # launched docker instance will not cleanup logs.
declare -g SKIP_LOG_ARCHIVE=yes # Don't archive logs in the parent instance either.
declare -g ARMBIAN_CLI_RELAUNCH_ARGS=()
produce_relaunch_parameters # produces ARMBIAN_CLI_RELAUNCH_ARGS
docker_cli_launch "${ARMBIAN_CLI_RELAUNCH_ARGS[@]}" # MARK: this "re-launches" using the passed params.
}

View File

@@ -1,152 +0,0 @@
#!/usr/bin/env bash
function cli_entrypoint() {
# array, readonly, global, for future reference, "exported" to shutup shellcheck
declare -rg -x -a ARMBIAN_ORIGINAL_ARGV=("${@}")
if [[ "${ARMBIAN_ENABLE_CALL_TRACING}" == "yes" ]]; then
set -T # inherit return/debug traps
mkdir -p "${SRC}"/output/call-traces
echo -n "" > "${SRC}"/output/call-traces/calls.txt
trap 'echo "${BASH_LINENO[@]}|${BASH_SOURCE[@]}|${FUNCNAME[@]}" >> ${SRC}/output/call-traces/calls.txt ;' RETURN
fi
if [[ "${EUID}" == "0" ]] || [[ "${1}" == "vagrant" ]]; then
:
elif [[ "${1}" == docker || "${1}" == dockerpurge || "${1}" == docker-shell ]] && grep -q "$(whoami)" <(getent group docker); then
:
elif [[ "${CONFIG_DEFS_ONLY}" == "yes" ]]; then # this var is set in the ENVIRONMENT, not as parameter.
display_alert "No sudo for" "env CONFIG_DEFS_ONLY=yes" "debug" # not really building in this case, just gathering meta-data.
else
display_alert "This script requires root privileges, trying to use sudo" "" "wrn"
sudo "${SRC}/compile.sh" "$@"
fi
# Purge Armbian Docker images
if [[ "${1}" == dockerpurge && -f /etc/debian_version ]]; then
display_alert "Purging Armbian Docker containers" "" "wrn"
docker container ls -a | grep armbian | awk '{print $1}' | xargs docker container rm &> /dev/null
docker image ls | grep armbian | awk '{print $3}' | xargs docker image rm &> /dev/null
shift
set -- "docker" "$@"
fi
# Docker shell
if [[ "${1}" == docker-shell ]]; then
shift
SHELL_ONLY=yes
set -- "docker" "$@"
fi
handle_docker_vagrant "$@"
prepare_userpatches "$@"
if [[ -z "${CONFIG}" && -n "$1" && -f "${SRC}/userpatches/config-$1.conf" ]]; then
CONFIG="userpatches/config-$1.conf"
shift
fi
# using default if custom not found
if [[ -z "${CONFIG}" && -f "${SRC}/userpatches/config-default.conf" ]]; then
CONFIG="userpatches/config-default.conf"
fi
# source build configuration file
CONFIG_FILE="$(realpath "${CONFIG}")"
if [[ ! -f "${CONFIG_FILE}" ]]; then
display_alert "Config file does not exist" "${CONFIG}" "error"
exit 254
fi
CONFIG_PATH=$(dirname "${CONFIG_FILE}")
# DEST is the main output dir.
declare DEST="${SRC}/output"
if [ -d "$CONFIG_PATH/output" ]; then
DEST="${CONFIG_PATH}/output"
fi
display_alert "Output directory DEST:" "${DEST}" "debug"
# set unique mounting directory for this build.
# basic deps, which include "uuidgen", will be installed _after_ this, so we gotta tolerate it not being there yet.
declare -g ARMBIAN_BUILD_UUID
if [[ -f /usr/bin/uuidgen ]]; then
ARMBIAN_BUILD_UUID="$(uuidgen)"
else
display_alert "uuidgen not found" "uuidgen not installed yet" "info"
ARMBIAN_BUILD_UUID="no-uuidgen-yet-${RANDOM}-$((1 + $RANDOM % 10))$((1 + $RANDOM % 10))$((1 + $RANDOM % 10))$((1 + $RANDOM % 10))"
fi
display_alert "Build UUID:" "${ARMBIAN_BUILD_UUID}" "debug"
# Super-global variables, used everywhere. The directories are NOT _created_ here, since this very early stage.
export WORKDIR="${SRC}/.tmp/work-${ARMBIAN_BUILD_UUID}" # WORKDIR at this stage. It will become TMPDIR later. It has special significance to `mktemp` and others!
export SDCARD="${SRC}/.tmp/rootfs-${ARMBIAN_BUILD_UUID}" # SDCARD (which is NOT an sdcard, but will be, maybe, one day) is where we work the rootfs before final imaging. "rootfs" stage.
export MOUNT="${SRC}/.tmp/mount-${ARMBIAN_BUILD_UUID}" # MOUNT ("mounted on the loop") is the mounted root on final image (via loop). "image" stage
export EXTENSION_MANAGER_TMP_DIR="${SRC}/.tmp/extensions-${ARMBIAN_BUILD_UUID}" # EXTENSION_MANAGER_TMP_DIR used to store extension-composed functions
export DESTIMG="${SRC}/.tmp/image-${ARMBIAN_BUILD_UUID}" # DESTIMG is where the backing image (raw, huge, sparse file) is kept (not the final destination)
export LOGDIR="${SRC}/.tmp/logs-${ARMBIAN_BUILD_UUID}" # Will be initialized very soon, literally, below.
LOG_SECTION=entrypoint start_logging_section # This creates LOGDIR.
add_cleanup_handler trap_handler_cleanup_logging # cleanup handler for logs; it rolls it up from LOGDIR into DEST/logs
if [ "${OFFLINE_WORK}" == "yes" ]; then
display_alert "* " "You are working offline!"
display_alert "* " "Sources, time and host will not be checked"
else
# check and install the basic utilities.
LOG_SECTION="prepare_host_basic" do_with_logging prepare_host_basic
fi
# Source the extensions manager library at this point, before sourcing the config.
# This allows early calls to enable_extension(), but initialization proper is done later.
# shellcheck source=lib/extensions.sh
source "${SRC}"/lib/extensions.sh
display_alert "Using config file" "${CONFIG_FILE}" "info"
pushd "${CONFIG_PATH}" > /dev/null || exit
# shellcheck source=/dev/null
source "${CONFIG_FILE}"
popd > /dev/null || exit
[[ -z "${USERPATCHES_PATH}" ]] && USERPATCHES_PATH="${CONFIG_PATH}"
# Script parameters handling
while [[ "${1}" == *=* ]]; do
parameter=${1%%=*}
value=${1##*=}
shift
display_alert "Command line: setting $parameter to" "${value:-(empty)}" "info"
eval "$parameter=\"$value\""
done
##
## Main entrypoint.
##
# reset completely after sourcing config file
#set -o pipefail # trace ERR through pipes - will be enabled "soon"
#set -o nounset ## set -u : exit the script if you try to use an uninitialised variable - one day will be enabled
set -o errtrace # trace ERR through - enabled
set -o errexit ## set -e : exit the script if any statement returns a non-true return value - enabled
# configuration etc - it initializes the extension manager.
do_capturing_defs prepare_and_config_main_build_single # this sets CAPTURED_VARS
if [[ "${CONFIG_DEFS_ONLY}" == "yes" ]]; then
echo "${CAPTURED_VARS}" # to stdout!
else
unset CAPTURED_VARS
# Allow for custom user-invoked functions, or do the default build.
if [[ -z $1 ]]; then
main_default_build_single
else
# @TODO: rpardini: check this with extensions usage?
eval "$@"
fi
fi
# Build done, run the cleanup handlers explicitly.
# This zeroes out the list of cleanups, so it's not done again when the main script exits normally and trap = 0 runs.
run_cleanup_handlers
}

View File

@@ -0,0 +1,24 @@
function cli_requirements_pre_run() {
declare -g ARMBIAN_COMMAND_REQUIRE_BASIC_DEPS="yes" # Require prepare_host_basic to run before the command.
if [[ "$(uname)" != "Linux" ]]; then
display_alert "Not running on Linux" "refusing to run 'requirements'" "err"
exit 1
fi
if [[ "${EUID}" == "0" ]]; then # we're already root. Either running as real root, or already sudo'ed.
display_alert "Already running as root" "great" "debug"
else
# Fail, installing requirements is not allowed as non-root.
exit_with_error "This command requires root privileges - refusing to run"
fi
}
function cli_requirements_run() {
declare -g REQUIREMENTS_DEFS_ONLY='yes' # @TODO: decide, this is already set in ARMBIAN_COMMANDS_TO_VARS_DICT
declare -a -g host_dependencies=()
early_prepare_host_dependencies # tests itself for REQUIREMENTS_DEFS_ONLY=yes too
install_host_dependencies "for REQUIREMENTS_DEFS_ONLY=yes"
display_alert "Done with" "REQUIREMENTS_DEFS_ONLY" "cachehit"
}

View File

@@ -0,0 +1,16 @@
function cli_undecided_pre_run() {
# If undecided, run the 'build' command.
# 'build' will then defer to 'docker' if ran on Darwin.
# so save a trip, check if we're on Darwin right here.
if [[ "$(uname)" == "Linux" ]]; then
display_alert "Linux!" "func cli_undecided_pre_run go to build" "debug"
ARMBIAN_CHANGE_COMMAND_TO="build"
else
display_alert "Not under Linux; use docker..." "func cli_undecided_pre_run go to docker" "debug"
ARMBIAN_CHANGE_COMMAND_TO="docker"
fi
}
function cli_undecided_run() {
exit_with_error "Should never run the undecided command. How did this happen?"
}

View File

@@ -0,0 +1,7 @@
function cli_vagrant_pre_run() {
:
}
function cli_vagrant_run() {
:
}

View File

@@ -0,0 +1,49 @@
function armbian_register_commands() {
# More than one command can map to the same handler. In that case, use ARMBIAN_COMMANDS_TO_VARS_DICT for specific vars.
declare -g -A ARMBIAN_COMMANDS_TO_HANDLERS_DICT=(
["docker"]="docker" # thus requires cli_docker_pre_run and cli_docker_run
["docker-purge"]="docker" # idem
["dockerpurge"]="docker" # idem
["docker-shell"]="docker" # idem
["dockershell"]="docker" # idem
["generate-dockerfile"]="docker" # idem
["vagrant"]="vagrant" # thus requires cli_vagrant_pre_run and cli_vagrant_run
["requirements"]="requirements" # implemented in cli_requirements_pre_run and cli_requirements_run # @TODO
["config-dump"]="config_dump" # implemented in cli_config_dump_pre_run and cli_config_dump_run # @TODO
["configdump"]="config_dump" # idem
["build"]="standard_build" # implemented in cli_standard_build_pre_run and cli_standard_build_run
["undecided"]="undecided" # implemented in cli_undecided_pre_run and cli_undecided_run - relaunches either build or docker
)
# Vars to be set for each command. Optional.
declare -g -A ARMBIAN_COMMANDS_TO_VARS_DICT=(
["docker-purge"]="DOCKER_SUBCMD='purge'"
["dockerpurge"]="DOCKER_SUBCMD='purge'"
["docker-shell"]="DOCKER_SUBCMD='shell'"
["dockershell"]="DOCKER_SUBCMD='shell'"
["generate-dockerfile"]="DOCKERFILE_GENERATE_ONLY='yes'"
["requirements"]="REQUIREMENTS_DEFS_ONLY='yes'"
["config-dump"]="CONFIG_DEFS_ONLY='yes'"
["configdump"]="CONFIG_DEFS_ONLY='yes'"
)
# Override the LOG_CLI_ID to change the log file name.
# Will be set to ARMBIAN_COMMAND if not set after all pre-runs done.
declare -g ARMBIAN_LOG_CLI_ID
# Keep a running dict of params/variables. Can't repeat stuff here. Dict.
declare -g -A ARMBIAN_CLI_RELAUNCH_PARAMS=(["ARMBIAN_RELAUNCHED"]="yes")
# Keep a running array of config files needed for relaunch.
declare -g -a ARMBIAN_CLI_RELAUNCH_CONFIGS=()
}

View File

@@ -0,0 +1,158 @@
function cli_entrypoint() {
# array, readonly, global, for future reference, "exported" to shutup shellcheck
declare -rg -x -a ARMBIAN_ORIGINAL_ARGV=("${@}")
if [[ "${ARMBIAN_ENABLE_CALL_TRACING}" == "yes" ]]; then
set -T # inherit return/debug traps
mkdir -p "${SRC}"/output/call-traces
echo -n "" > "${SRC}"/output/call-traces/calls.txt
trap 'echo "${BASH_LINENO[@]}|${BASH_SOURCE[@]}|${FUNCNAME[@]}" >> ${SRC}/output/call-traces/calls.txt ;' RETURN
fi
# @TODO: allow for a super-early userpatches/config-000.custom.conf.sh to be loaded, before anything else.
# This would allow for custom commands and interceptors.
# Decide what we're gonna do. We've a few hardcoded, 1st-argument "commands".
declare -g -A ARMBIAN_COMMANDS_TO_HANDLERS_DICT ARMBIAN_COMMANDS_TO_VARS_DICT
armbian_register_commands # this defines the above two dictionaries
# Process the command line, separating params (XX=YY) from non-params arguments.
# That way they can be set in any order.
declare -A -g ARMBIAN_PARSED_CMDLINE_PARAMS=() # A dict of PARAM=VALUE
declare -a -g ARMBIAN_NON_PARAM_ARGS=() # An array of all non-param arguments
parse_cmdline_params "${@}" # which fills the above vars.
# Now load the key=value pairs from cmdline into environment, before loading config or executing commands.
# This will be done _again_ later, to make sure cmdline params override config et al.
apply_cmdline_params_to_env "early" # which uses ARMBIAN_PARSED_CMDLINE_PARAMS
# From here on, no more ${1} or stuff. We've parsed it all into ARMBIAN_PARSED_CMDLINE_PARAMS or ARMBIAN_NON_PARAM_ARGS and ARMBIAN_COMMAND.
declare -a -g ARMBIAN_CONFIG_FILES=() # fully validated, complete paths to config files.
declare -g ARMBIAN_COMMAND_HANDLER="" ARMBIAN_COMMAND="" ARMBIAN_COMMAND_VARS="" # only valid command and handler will ever be set here.
declare -g ARMBIAN_HAS_UNKNOWN_ARG="no" # if any unknown params, bomb.
for argument in "${ARMBIAN_NON_PARAM_ARGS[@]}"; do # loop over all non-param arguments, find commands and configs.
parse_each_cmdline_arg_as_command_param_or_config "${argument}" # sets all the vars above
done
# More sanity checks.
# If unknowns, bail.
if [[ "${ARMBIAN_HAS_UNKNOWN_ARG}" == "yes" ]]; then
exit_with_error "Unknown arguments found. Please check the output above and fix them."
fi
# @TODO: Have a config that is always included? "${SRC}/userpatches/config-default.conf" ?
# If we don't have a command decided yet, use the undecided command.
if [[ "${ARMBIAN_COMMAND}" == "" ]]; then
display_alert "No command found, using default" "undecided" "debug"
ARMBIAN_COMMAND="undecided"
fi
# If we don't have a command at this stage, we should default either to 'build' or 'docker', depending on OS.
# Give the chosen command a chance to refuse running, or, even, change the final command to run.
# This allows for example the 'build' command to auto-launch under docker, even without specifying it.
# Also allows for launchers to keep themselves when re-launched, yet do something diferent. (eg: docker under docker does build).
# Or: build under Darwin does docker...
# each _pre_run can change the command and vars to run too, so do it in a loop until it stops changing.
declare -g ARMBIAN_CHANGE_COMMAND_TO="${ARMBIAN_COMMAND}"
while [[ "${ARMBIAN_CHANGE_COMMAND_TO}" != "" ]]; do
display_alert "Still a command to pre-run, this time:" "${ARMBIAN_CHANGE_COMMAND_TO}" "debug"
ARMBIAN_COMMAND="${ARMBIAN_CHANGE_COMMAND_TO}"
armbian_prepare_cli_command_to_run "${ARMBIAN_COMMAND}"
ARMBIAN_CHANGE_COMMAND_TO=""
armbian_cli_pre_run_command
done
# IMPORTANT!!!: it is INVALID to relaunch compile.sh from here. It will cause logging mistakes.
# So the last possible moment to relaunch is in xxxxx_pre_run!
# Also form here, UUID will be generated, output created, logging enabled, etc.
# Init basic dirs.
declare -g DEST="${SRC}/output" USERPATCHES_PATH="${SRC}"/userpatches # DEST is the main output dir, and USERPATCHES_PATH is the userpatches dir.
mkdir -p "${DEST}" "${USERPATCHES_PATH}" # Create output and userpatches directory if not already there
display_alert "Output directory created! DEST:" "${DEST}" "debug"
# set unique mounting directory for this execution.
# basic deps, which include "uuidgen", will be installed _after_ this, so we gotta tolerate it not being there yet.
declare -g ARMBIAN_BUILD_UUID
if [[ "${ARMBIAN_BUILD_UUID}" != "" ]]; then
display_alert "Using passed-in ARMBIAN_BUILD_UUID" "${ARMBIAN_BUILD_UUID}" "debug"
else
if [[ -f /usr/bin/uuidgen ]]; then
ARMBIAN_BUILD_UUID="$(uuidgen)"
else
display_alert "uuidgen not found" "uuidgen not installed yet" "info"
ARMBIAN_BUILD_UUID="no-uuidgen-yet-${RANDOM}-$((1 + $RANDOM % 10))$((1 + $RANDOM % 10))$((1 + $RANDOM % 10))$((1 + $RANDOM % 10))"
fi
ARMBIAN_BUILD_UUID="$(uuidgen)"
display_alert "Generated ARMBIAN_BUILD_UUID" "${ARMBIAN_BUILD_UUID}" "debug"
fi
display_alert "Build UUID:" "${ARMBIAN_BUILD_UUID}" "debug"
# Super-global variables, used everywhere. The directories are NOT _created_ here, since this very early stage.
export WORKDIR="${SRC}/.tmp/work-${ARMBIAN_BUILD_UUID}" # WORKDIR at this stage. It will become TMPDIR later. It has special significance to `mktemp` and others!
export LOGDIR="${SRC}/.tmp/logs-${ARMBIAN_BUILD_UUID}" # Will be initialized very soon, literally, below.
# @TODO: These are used by actual build, move to its cli handler.
export SDCARD="${SRC}/.tmp/rootfs-${ARMBIAN_BUILD_UUID}" # SDCARD (which is NOT an sdcard, but will be, maybe, one day) is where we work the rootfs before final imaging. "rootfs" stage.
export MOUNT="${SRC}/.tmp/mount-${ARMBIAN_BUILD_UUID}" # MOUNT ("mounted on the loop") is the mounted root on final image (via loop). "image" stage
export EXTENSION_MANAGER_TMP_DIR="${SRC}/.tmp/extensions-${ARMBIAN_BUILD_UUID}" # EXTENSION_MANAGER_TMP_DIR used to store extension-composed functions
export DESTIMG="${SRC}/.tmp/image-${ARMBIAN_BUILD_UUID}" # DESTIMG is where the backing image (raw, huge, sparse file) is kept (not the final destination)
# Make sure ARMBIAN_LOG_CLI_ID is set, and unique.
# Pre-runs might change it, but if not set, default to ARMBIAN_COMMAND.
declare -g ARMBIAN_LOG_CLI_ID="${ARMBIAN_LOG_CLI_ID:-${ARMBIAN_COMMAND}}"
LOG_SECTION="entrypoint" start_logging_section # This creates LOGDIR. @TODO: also maybe causes a spurious group to be created in the log file
add_cleanup_handler trap_handler_cleanup_logging # cleanup handler for logs; it rolls it up from LOGDIR into DEST/logs @TODO: use the COMMAND in the filenames.
# @TODO: So gigantic contention point here about logging the basic deps installation.
if [[ "${ARMBIAN_COMMAND_REQUIRE_BASIC_DEPS}" == "yes" ]]; then
if [[ "${OFFLINE_WORK}" == "yes" ]]; then
display_alert "* " "You are working offline!"
display_alert "* " "Sources, time and host will not be checked"
else
# check and install the basic utilities;
LOG_SECTION="prepare_host_basic" do_with_logging prepare_host_basic # This includes the 'docker' case.
fi
fi
# Source the extensions manager library at this point, before sourcing the config.
# This allows early calls to enable_extension(), but initialization proper is done later.
# shellcheck source=lib/extensions.sh
source "${SRC}"/lib/extensions.sh
# Loop over the ARMBIAN_CONFIG_FILES array and source each. The order is important.
for config_file in "${ARMBIAN_CONFIG_FILES[@]}"; do
local config_filename="${config_file##*/}" config_dir="${config_file%/*}"
display_alert "Sourcing config file" "${config_filename}" "debug"
# use pushd/popd to change directory to the config file's directory, so that relative paths in the config file work.
pushd "${config_dir}" > /dev/null || exit_with_error "Failed to pushd to ${config_dir}"
# shellcheck source=/dev/null
LOG_SECTION="userpatches_config:${config_filename}" do_with_logging source "${config_file}"
# reset completely after sourcing config file
set -e
#set -o pipefail # trace ERR through pipes - will be enabled "soon"
#set -o nounset ## set -u : exit the script if you try to use an uninitialised variable - one day will be enabled
set -o errtrace # trace ERR through - enabled
set -o errexit ## set -e : exit the script if any statement returns a non-true return value - enabled
popd > /dev/null || exit_with_error "Failed to popd from ${config_dir}"
# Apply the params received from the command line _again_ after running the config.
# This ensures that params take precedence over stuff possibly defined in the config.
apply_cmdline_params_to_env "after config '${config_filename}'" # which uses ARMBIAN_PARSED_CMDLINE_PARAMS
done
display_alert "Executing final CLI command" "${ARMBIAN_COMMAND}" "debug"
armbian_cli_run_command
display_alert "Done Executing final CLI command" "${ARMBIAN_COMMAND}" "debug"
# Build done, run the cleanup handlers explicitly.
# This zeroes out the list of cleanups, so it"s not done again when the main script exits normally and trap = 0 runs.
run_cleanup_handlers
}

View File

@@ -1,87 +1,183 @@
#!/usr/bin/env bash
# Misc functions from compile.sh
function handle_docker_vagrant() {
# Check for Vagrant
if [[ "${1}" == vagrant && -z "$(command -v vagrant)" ]]; then
display_alert "Vagrant not installed." "Installing"
sudo apt-get update
sudo apt-get install -y vagrant virtualbox
fi
# Install Docker if not there but wanted. We cover only Debian based distro install. On other distros, manual Docker install is needed
if [[ "${1}" == docker && -f /etc/debian_version && -z "$(command -v docker)" ]]; then
DOCKER_BINARY="docker-ce"
# add exception for Ubuntu Focal until Docker provides dedicated binary
codename=$(cat /etc/os-release | grep VERSION_CODENAME | cut -d"=" -f2)
codeid=$(cat /etc/os-release | grep ^NAME | cut -d"=" -f2 | awk '{print tolower($0)}' | tr -d '"' | awk '{print $1}')
[[ "${codename}" == "debbie" ]] && codename="buster" && codeid="debian"
[[ "${codename}" == "ulyana" || "${codename}" == "jammy" || "${codename}" == "kinetic" || "${codename}" == "lunar" ]] && codename="focal" && codeid="ubuntu"
# different binaries for some. TBD. Need to check for all others
[[ "${codename}" =~ focal|hirsute ]] && DOCKER_BINARY="docker containerd docker.io"
display_alert "Docker not installed." "Installing" "Info"
sudo bash -c "echo \"deb [arch=$(dpkg --print-architecture)] https://download.docker.com/linux/${codeid} ${codename} stable\" > /etc/apt/sources.list.d/docker.list"
sudo bash -c "curl -fsSL \"https://download.docker.com/linux/${codeid}/gpg\" | apt-key add -qq - > /dev/null 2>&1 "
export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
sudo apt-get install -y -qq --no-install-recommends ${DOCKER_BINARY}
display_alert "Add yourself to docker group to avoid root privileges" "" "wrn"
"${SRC}/compile.sh" "$@"
exit $?
fi
# This is called like this:
# declare -A -g ARMBIAN_PARSED_CMDLINE_PARAMS=()
# declare -a -g ARMBIAN_NON_PARAM_ARGS=()
# parse_cmdline_params "${@}" # which fills the vars above, being global.
function parse_cmdline_params() {
declare -A -g ARMBIAN_PARSED_CMDLINE_PARAMS=()
declare -a -g ARMBIAN_NON_PARAM_ARGS=()
# loop over the arguments parse them out
local arg
for arg in "${@}"; do
if [[ "${arg}" == *=* ]]; then # contains an equal sign. it's a param.
local param_name param_value param_value_desc
param_name=${arg%%=*}
param_value=${arg##*=}
param_value_desc="${param_value:-(empty)}"
ARMBIAN_PARSED_CMDLINE_PARAMS["${param_name}"]="${param_value}" # For current run.
ARMBIAN_CLI_RELAUNCH_PARAMS["${param_name}"]="${param_value}" # For relaunch.
display_alert "Command line: parsed parameter '$param_name' to" "${param_value_desc}" "debug"
elif [[ "x${arg}x" != "xx" ]]; then # not a param, not empty, store it in the non-param array for later usage
local non_param_value="${arg}"
local non_param_value_desc="${non_param_value:-(empty)}"
display_alert "Command line: storing non-param argument" "${non_param_value_desc}" "debug"
ARMBIAN_NON_PARAM_ARGS+=("${non_param_value}")
fi
done
}
function prepare_userpatches() {
# Create userpatches directory if not exists
mkdir -p "${SRC}"/userpatches
# This can be called early on, or later after having sourced the config. Show what is happening.
# This is called:
# apply_cmdline_params_to_env "reason" # reads from global ARMBIAN_PARSED_CMDLINE_PARAMS
function apply_cmdline_params_to_env() {
declare -A -g ARMBIAN_PARSED_CMDLINE_PARAMS # Hopefully this has values
declare __my_reason="${1}"
shift
# Create example configs if none found in userpatches
if ! ls "${SRC}"/userpatches/{config-default.conf,config-docker.conf,config-vagrant.conf} 1> /dev/null 2>&1; then
# Loop over the dictionary and apply the values to the environment.
for param_name in "${!ARMBIAN_PARSED_CMDLINE_PARAMS[@]}"; do
local param_value param_value_desc current_env_value
# get the current value from the environment
current_env_value="${!param_name}"
current_env_value_desc="${current_env_value:-(empty)}"
# get the new value from the dictionary
param_value="${ARMBIAN_PARSED_CMDLINE_PARAMS[${param_name}]}"
param_value_desc="${param_value:-(empty)}"
# Migrate old configs
if ls "${SRC}"/*.conf 1> /dev/null 2>&1; then
display_alert "Migrate config files to userpatches directory" "all *.conf" "info"
cp "${SRC}"/*.conf "${SRC}"/userpatches || exit 1
rm "${SRC}"/*.conf
[[ ! -L "${SRC}"/userpatches/config-example.conf ]] && ln -fs config-example.conf "${SRC}"/userpatches/config-default.conf || exit 1
# Compare, log, and apply.
if [[ "${current_env_value}" != "${param_value}" ]]; then
display_alert "Applying cmdline param" "'$param_name': '${current_env_value_desc}' --> '${param_value_desc}' ${__my_reason}" "cmdline"
# use `declare -g` to make it global, we're in a function.
eval "declare -g $param_name=\"$param_value\""
else
# rpardini: strategic amount of spacing in log files show the kinda neuroticism that drives me.
display_alert "Skip cmdline param" "'$param_name': already set to '${param_value_desc}' ${__my_reason}" "info"
fi
done
}
display_alert "Create example config file using template" "config-default.conf" "info"
function armbian_prepare_cli_command_to_run() {
local command_id="${1}"
display_alert "Preparing to run command" "${command_id}" "debug"
ARMBIAN_COMMAND="${command_id}"
ARMBIAN_COMMAND_HANDLER="${ARMBIAN_COMMANDS_TO_HANDLERS_DICT[${command_id}]}"
ARMBIAN_COMMAND_VARS="${ARMBIAN_COMMANDS_TO_VARS_DICT[${command_id}]}"
# @TODO: actually set the vars...
# Create example config
if [[ ! -f "${SRC}"/userpatches/config-example.conf ]]; then
cp "${SRC}"/config/templates/config-example.conf "${SRC}"/userpatches/config-example.conf || exit 1
fi
local set_vars_for_command=""
if [[ "x${ARMBIAN_COMMAND_VARS}x" != "xx" ]]; then
# Loop over them, expanding...
for var_piece in ${ARMBIAN_COMMAND_VARS}; do
local var_decl="declare -g ${var_piece};"
display_alert "Command handler: setting variable" "${var_decl}" "debug"
set_vars_for_command+=" ${var_decl}"
done
fi
# Link default config to example config
if [[ ! -f "${SRC}"/userpatches/config-default.conf ]]; then
ln -fs config-example.conf "${SRC}"/userpatches/config-default.conf || exit 1
fi
local pre_run_function_name="cli_${ARMBIAN_COMMAND_HANDLER}_pre_run"
local run_function_name="cli_${ARMBIAN_COMMAND_HANDLER}_run"
# Create Docker config
if [[ ! -f "${SRC}"/userpatches/config-docker.conf ]]; then
cp "${SRC}"/config/templates/config-docker.conf "${SRC}"/userpatches/config-docker.conf || exit 1
fi
# Reset the functions.
function armbian_cli_pre_run_command() {
display_alert "No pre-run function for command" "${ARMBIAN_COMMAND}" "warn"
}
function armbian_cli_run_command() {
display_alert "No run function for command" "${ARMBIAN_COMMAND}" "warn"
}
# Create Docker file
if [[ ! -f "${SRC}"/userpatches/Dockerfile ]]; then
cp "${SRC}"/config/templates/Dockerfile "${SRC}"/userpatches/Dockerfile || exit 1
fi
# Materialize functions to call that specific command.
if [[ $(type -t "${pre_run_function_name}" || true) == function ]]; then
eval "$(
cat <<- EOF
display_alert "Setting up pre-run function for command" "${ARMBIAN_COMMAND}: ${pre_run_function_name}" "debug"
function armbian_cli_pre_run_command() {
# Set the variables defined in ARMBIAN_COMMAND_VARS
${set_vars_for_command}
display_alert "Calling pre-run function for command" "${ARMBIAN_COMMAND}: ${pre_run_function_name}" "debug"
${pre_run_function_name}
}
EOF
)"
fi
# Create Vagrant config
if [[ ! -f "${SRC}"/userpatches/config-vagrant.conf ]]; then
cp "${SRC}"/config/templates/config-vagrant.conf "${SRC}"/userpatches/config-vagrant.conf || exit 1
fi
# Create Vagrant file
if [[ ! -f "${SRC}"/userpatches/Vagrantfile ]]; then
cp "${SRC}"/config/templates/Vagrantfile "${SRC}"/userpatches/Vagrantfile || exit 1
fi
if [[ $(type -t "${run_function_name}" || true) == function ]]; then
eval "$(
cat <<- EOF
display_alert "Setting up run function for command" "${ARMBIAN_COMMAND}: ${run_function_name}" "debug"
function armbian_cli_run_command() {
# Set the variables defined in ARMBIAN_COMMAND_VARS
${set_vars_for_command}
display_alert "Calling run function for command" "${ARMBIAN_COMMAND}: ${run_function_name}" "debug"
${run_function_name}
}
EOF
)"
fi
}
function parse_each_cmdline_arg_as_command_param_or_config() {
local is_command="no" is_config="no" command_handler conf_path conf_sh_path config_file=""
local argument="${1}"
# lookup if it is a command.
if [[ -n "${ARMBIAN_COMMANDS_TO_HANDLERS_DICT[${argument}]}" ]]; then
is_command="yes"
command_handler="${ARMBIAN_COMMANDS_TO_HANDLERS_DICT[${argument}]}"
display_alert "Found command!" "${argument} is handled by '${command_handler}'" "debug"
fi
# see if we can find config file in userpatches. can be either config-${argument}.conf or config-${argument}.conf.sh
conf_path="${SRC}/userpatches/config-${argument}.conf"
conf_sh_path="${SRC}/userpatches/config-${argument}.conf.sh"
# early safety net: immediately bomb if we find both forms of config. it's too confusing. choose one.
if [[ -f ${conf_path} && -f ${conf_sh_path} ]]; then
exit_with_error "Found both config-${argument}.conf and config-${argument}.conf.sh in userpatches. Please remove one."
exit 1
elif [[ -f ${conf_sh_path} ]]; then
config_file="${conf_sh_path}"
is_config="yes"
elif [[ -f ${conf_path} ]]; then
config_file="${conf_path}"
is_config="yes"
fi
# Sanity check. If we have both a command and a config, bomb.
if [[ "${is_command}" == "yes" && "${is_config}" == "yes" ]]; then
exit_with_error "You cannot have a configuration file named '${config_file}'. '${argument}' is a command name and is reserved for internal Armbian usage. Sorry. Please rename your config file and pass its name it an argument, and I'll use it. PS: You don't need a config file for 'docker' anymore, Docker is all managed by Armbian now."
elif [[ "${is_config}" == "yes" ]]; then # we have a config only
display_alert "Adding config file to list" "${config_file}" "debug"
ARMBIAN_CONFIG_FILES+=("${config_file}") # full path to be sourced
ARMBIAN_CLI_RELAUNCH_CONFIGS+="${argument}" # name reference to be relaunched
elif [[ "${is_command}" == "yes" ]]; then # we have a command, only.
# sanity check. we can't have more than one command. decide!
if [[ -n "${ARMBIAN_COMMAND}" ]]; then
exit_with_error "You cannot specify more than one command. You have '${ARMBIAN_COMMAND}' and '${argument}'. Please decide which one you want to run and pass only that one."
exit 1
fi
ARMBIAN_COMMAND="${argument}" # too early for armbian_prepare_cli_command_to_run "${argument}"
else
# We've an unknown argument. Alert now, bomb later.
ARMBIAN_HAS_UNKNOWN_ARG="yes"
display_alert "Unknown argument" "${argument}" "err"
fi
}
# Produce relaunch parameters. Add the running configs, arguments, and command.
# Declare and use ARMBIAN_CLI_RELAUNCH_ARGS as "${ARMBIAN_CLI_RELAUNCH_ARGS[@]}"
function produce_relaunch_parameters() {
declare -g -a ARMBIAN_CLI_RELAUNCH_ARGS=()
# add the running parameters from ARMBIAN_CLI_RELAUNCH_PARAMS dict
for param in "${!ARMBIAN_CLI_RELAUNCH_PARAMS[@]}"; do
ARMBIAN_CLI_RELAUNCH_ARGS+=("${param}=${ARMBIAN_CLI_RELAUNCH_PARAMS[${param}]}")
done
# add the running configs
for config in ${ARMBIAN_CLI_RELAUNCH_CONFIGS}; do
ARMBIAN_CLI_RELAUNCH_ARGS+=("${config}")
done
display_alert "Produced relaunch args:" "ARMBIAN_CLI_RELAUNCH_ARGS: ${ARMBIAN_CLI_RELAUNCH_ARGS[*]}" "debug"
# @TODO: add the command. if we have one.
}

View File

@@ -24,9 +24,21 @@ function do_main_configuration() {
[[ -z $ROOTPWD ]] && ROOTPWD="1234" # Must be changed @first login
[[ -z $MAINTAINER ]] && MAINTAINER="Igor Pecovnik" # deb signature
[[ -z $MAINTAINERMAIL ]] && MAINTAINERMAIL="igor.pecovnik@****l.com" # deb signature
export SKIP_EXTERNAL_TOOLCHAINS="${SKIP_EXTERNAL_TOOLCHAINS:-yes}" # don't use any external toolchains, by default.
TZDATA=$(cat /etc/timezone) # Timezone for target is taken from host or defined here.
USEALLCORES=yes # Use all CPU cores for compiling
DEST_LANG="${DEST_LANG:-"en_US.UTF-8"}" # en_US.UTF-8 is default locale for target
display_alert "DEST_LANG..." "DEST_LANG: ${DEST_LANG}" "debug"
export SKIP_EXTERNAL_TOOLCHAINS="${SKIP_EXTERNAL_TOOLCHAINS:-yes}" # don't use any external toolchains, by default.
# Timezone
if [[ -f /etc/timezone ]]; then # Timezone for target is taken from host, if it exists.
TZDATA=$(cat /etc/timezone)
display_alert "Using host's /etc/timezone for" "TZDATA: ${TZDATA}" "debug"
else
display_alert "Host has no /etc/timezone" "Using Etc/UTC by default" "debug"
TZDATA="Etc/UTC" # If not /etc/timezone at host, default to UTC.
fi
USEALLCORES=yes # Use all CPU cores for compiling
HOSTRELEASE=$(cat /etc/os-release | grep VERSION_CODENAME | cut -d"=" -f2)
[[ -z $HOSTRELEASE ]] && HOSTRELEASE=$(cut -d'/' -f1 /etc/debian_version)
[[ -z $EXIT_PATCHING_ERROR ]] && EXIT_PATCHING_ERROR="" # exit patching if failed
@@ -34,8 +46,12 @@ function do_main_configuration() {
cd "${SRC}" || exit
[[ -z "${CHROOT_CACHE_VERSION}" ]] && CHROOT_CACHE_VERSION=7
BUILD_REPOSITORY_URL=$(git remote get-url "$(git remote | grep origin)")
BUILD_REPOSITORY_COMMIT=$(git describe --match=d_e_a_d_b_e_e_f --always --dirty)
if [[ -d "${SRC}/.git" ]]; then
BUILD_REPOSITORY_URL=$(git remote get-url "$(git remote | grep origin)")
BUILD_REPOSITORY_COMMIT=$(git describe --match=d_e_a_d_b_e_e_f --always --dirty)
fi
ROOTFS_CACHE_MAX=200 # max number of rootfs cache, older ones will be cleaned up
# .deb compression. xz is standard, but is slow, so if avoided by default if not running in CI. one day, zstd.
@@ -397,6 +413,7 @@ desktop/${RELEASE}/environments/${DESKTOP_ENVIRONMENT}/appgroups
PACKAGE_MAIN_LIST="$(cleanup_list PACKAGE_LIST)"
[[ $BUILD_DESKTOP == yes ]] && PACKAGE_LIST="$PACKAGE_LIST $PACKAGE_LIST_DESKTOP"
# @TODO: what is the use of changing PACKAGE_LIST after PACKAGE_MAIN_LIST was set?
PACKAGE_LIST="$(cleanup_list PACKAGE_LIST)"
# remove any packages defined in PACKAGE_LIST_RM in lib.config
@@ -455,7 +472,11 @@ function write_config_summary_output_file() {
local debug_dpkg_arch debug_uname debug_virt debug_src_mount debug_src_perms debug_src_temp_perms
debug_dpkg_arch="$(dpkg --print-architecture)"
debug_uname="$(uname -a)"
debug_virt="$(systemd-detect-virt || true)"
# We might not have systemd-detect-virt, specially inside docker. Docker images have no systemd...
debug_virt="unknown-nosystemd"
if [[ -n "$(command -v systemd-detect-virt)" ]]; then
debug_virt="$(systemd-detect-virt || true)"
fi
debug_src_mount="$(findmnt --output TARGET,SOURCE,FSTYPE,AVAIL --target "${SRC}" --uniq)"
debug_src_perms="$(getfacl -p "${SRC}")"
debug_src_temp_perms="$(getfacl -p "${SRC}"/.tmp 2> /dev/null)"

View File

@@ -15,14 +15,14 @@
fel_prepare_host() {
# Start rpcbind for NFS if inside docker container
[ "$(systemd-detect-virt)" == 'docker' ] && service rpcbind start
if armbian_is_running_in_container; then service rpcbind start; fi
# remove and re-add NFS share
rm -f /etc/exports.d/armbian.exports
mkdir -p /etc/exports.d
echo "$FEL_ROOTFS *(rw,async,no_subtree_check,no_root_squash,fsid=root)" > /etc/exports.d/armbian.exports
# Start NFS server if inside docker container
[ "$(systemd-detect-virt)" == 'docker' ] && service nfs-kernel-server start
if armbian_is_running_in_container; then service nfs-kernel-server; fi
exportfs -ra
}

View File

@@ -30,9 +30,13 @@ function improved_git_fetch() {
# workaround new limitations imposed by CVE-2022-24765 fix in git, otherwise "fatal: unsafe repository"
function git_ensure_safe_directory() {
local git_dir="$1"
display_alert "git: Marking directory as safe" "$git_dir" "debug"
run_host_command_logged git config --global --add safe.directory "$git_dir"
if [[ -n "$(command -v git)" ]]; then
local git_dir="$1"
display_alert "git: Marking directory as safe" "$git_dir" "debug"
run_host_command_logged git config --global --add safe.directory "$git_dir"
else
display_alert "git not installed" "a true wonder how you got this far without git - it will be installed for you" "warn"
fi
}
# fetch_from_repo <url> <directory> <ref> <ref_subdir>

View File

@@ -1,6 +1,7 @@
# Management of apt-cacher-ng aka acng
function acng_configure_and_restart_acng() {
if ! armbian_is_host_running_systemd; then return 0; fi # do nothing if host is not running systemd
[[ $NO_APT_CACHER == yes ]] && return 0 # don't if told not to. NO_something=yes is very confusing, but kept for historical reasons
[[ "${APT_PROXY_ADDR:-localhost:3142}" != "localhost:3142" ]] && return 0 # also not if acng not local to builder machine

Some files were not shown because too many files have changed in this diff Show More