rmtree() concatenates the directory path and entry name into a
PATH_MAX buffer without checking for truncation, which -Wformat-
truncation flags on the Windows build where PATH_MAX is 260. A
truncated path would also make the cleanup delete the wrong file in
principle.
Check the snprintf() result and skip entries that do not fit; using
the return value also satisfies the warning. A skipped entry means
the parent rmdir() fails with ENOTEMPTY and the workdir lingers, so
report the skip on stderr instead of hiding it - the message names
the entry, making a leftover workdir traceable to its cause.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Extend the hardware-in-the-loop suite with self-contained steps that
fit the existing round-trip model:
- list devices: enumerate the attached EDL device with the list verb
and require a vid:pid line in the output, covering device discovery
over real USB.
- chipinfo via Sahara: read the chip identity (serial number, HW ID
and OEM PK hash) with the chipinfo verb. The step runs before the
programmer upload because chip identity is only available while the
device still speaks Sahara; the verb switches the device back to
image-transfer mode on exit, so the subsequent flash still gets its
Sahara HELLO. SKIPs when a programmer is already running.
- create-zip and flash: package a build from a contents.xml with the
create-zip verb, then flash the produced zip, covering the zip
writer and the zip flashing path end to end. Gated on
QDL_HIL_CONTENTS.
- flash sparse build: flash a bundle that uses Android sparse images,
covering the sparse decoding path against real storage. Gated on
QDL_HIL_SPARSE.
- NAK on bad partition: a read of a nonexistent physical partition
must fail, verifying that a device NAK is reported as an error
instead of silently succeeding.
- sha256 by partition name: digest a partition addressed by its GPT
name rather than by partition and sector numbers, covering GPT name
resolution against the device.
- write chunked: rewrite the partition image with an overridden
--out-chunk-size=16384, covering the chunked transfer path.
- verify write integrity: compare the device-computed partition
digest against a local sha256 of the previously read-back and
rewritten image, proving the read/write round trip is bit-exact.
- reset board: switch the final step to the new reset verb, covering
the explicit device-reset path instead of relying on the implicit
reset after a dummy 1-sector read.
The two gated steps SKIP when their environment variable is unset,
and every step SKIPs without hardware, so a plain meson test stays
green.
Example invocation against an attached EDL device:
D=<path_to_build>
QDL_HIL_BUILD=$D QDL_HIL_CONTENTS=$D/contents.xml \
QDL_HIL_STORAGE=ufs meson test -C build --suite=hil
1/18 hil - qdl:HIL list devices OK 0.05s
2/18 hil - qdl:HIL chipinfo via Sahara OK 0.4s
3/18 hil - qdl:HIL flash whole build OK 29.03s
4/18 hil - qdl:HIL create-zip and flash OK 79.62s
5/18 hil - qdl:HIL flash sparse build SKIP 0.02s
...
18/18 hil - qdl:HIL reset board OK 3.68s
Ok: 17
Fail: 0
Skipped: 1
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Flashmap packages can contain more than one layout, but qdl previously
rejected anything other than a single layout. Add flashmap-specific
specifier parsing so callers can select a layout by name while
preserving the existing storage-only selector form.
The supported forms are now:
qdl flash flashmap.json::layout-name
qdl flash flashmap.json::layout-name/storage1,storage2
qdl flash flashmap.json::storage1,storage2
When a multi-layout flashmap is used without a layout selector, report
the valid layout names instead of failing with the old single-layout
limitation. Extend the cmocka flashmap tests to cover required layout
selection, layout-only selection, and layout plus storage filtering.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
The flashmap coverage was implemented as a shell integration test that
generated a flat build tree, created a zip archive, and invoked the qdl
binary in dry-run mode. That made the test slower and tied it to the
CLI, filesystem layout, and generated fixture data.
Replace it with a cmocka unit test around flashmap_load(). The new test
provides mock flashmap JSON, XML files, zip/file access, programmer
loading, and Firehose operation collection in-process. It preserves the
existing coverage for loading flashmap.json, loading a zip containing
flashmap.json, and applying a storage selector.
Register the test with the existing cmocka test group and remove the
shell TAP test.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
test_contents_selectors.c pulls in contents.c via #include, and
contents.c calls firehose_alloc_op(). The test target links neither
firehose.c nor libqdl_common, so the only reason it linked at all is
that at -O2 the compiler drops the (unreachable in this test) call.
At -O0 the call survives and the link fails:
src/contents.c:899: undefined reference to `firehose_alloc_op'
Provide a firehose_alloc_op() stub, like the other contents tests
already do, so the test links regardless of optimization level.
Closes: https://github.com/linux-msm/qdl/issues/260
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Before tagging a release we want to exercise qdl against real
hardware - flashing a full build, reading partitions, skipblock,
sha256, erase and write - but there is no automated way to do that
today. Add a "hil" meson suite that drives the built qdl binary
against an attached EDL device, so a release can be validated with a
single command. It is a stop-gap that simplifies testing until we have
proper CI HIL coverage via LAVA.
Each step is its own test in the "hil" suite. The steps are stateful,
so they run serially and in order (is_parallel:false plus a descending
priority): flash the build, read the EFI partition / GPT / address
forms, re-flash with --skipblock=sha256, run the sha256 command, erase
and write the EFI partition back, and finally reboot. Every step runs
with --skip-reset except the reboot. The erase/write pair round-trips
the partition's own readback, so nothing is left wiped, and once one
step fails the rest of the suite stops.
The suite is opt-in: the build and storage type are passed via the
environment, and when unset every step exits 77 (SKIP), so a plain
"meson test" / CI run stays green without hardware.
Usage:
$ QDL_HIL_BUILD=/path/to/build QDL_HIL_STORAGE=ufs \
meson test -C build --suite hil -v
1/10 hil - qdl:HIL flash whole build OK 21.63s
2/10 hil - qdl:HIL read EFI partition OK 10.50s
3/10 hil - qdl:HIL read GPT table OK 2.47s
4/10 hil - qdl:HIL read address P/S+L OK 2.46s
5/10 hil - qdl:HIL read address P/name OK 11.17s
6/10 hil - qdl:HIL flash skipblock sha256 OK 22.12s
7/10 hil - qdl:HIL sha256 command OK 2.47s
8/10 hil - qdl:HIL erase EFI partition OK 4.97s
9/10 hil - qdl:HIL write EFI partition OK 10.18s
10/10 hil - qdl:HIL reboot board OK 3.64s
Ok: 10 Fail: 0
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a cmocka unit test for the header-only helpers in oscompat.h:
- path_is_absolute(), checking POSIX and (under _WIN32) drive-letter
and UNC paths;
- qdl_mkdir_p(), checking recursive creation, idempotency on an
existing directory, trailing slashes, the "." and empty-path cases,
the ENAMETOOLONG guard, and failure when a path component is not a
directory.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Every source change funnells through the single ~200-line
root meson.build that enumerated the sources for all binaries,
so touching one area means editing a file that mostly described
unrelated code.
Declare the source lists in src/meson.build and the tests in
tests/meson.build and pull them in with subdir(). The root meson.build
is then left to compose (dependencies, version header, targets) while
the list of files to build lives next to the files themselves, so
adding or removing a source is a one-line edit in one obvious place.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The contents.xml provides two different "root path" tags, one with Linux
and one with Windows path delimiters. Use the Windows tag to avoid
issues related to path traversal.
Extend the tests as necessary.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
Introduce two initial set of test cases for the newly introduced
contents.xml handling.
One is used to validate the storage and flavor selector logic, and the
other more broadly the loading of contents.xml files.
Extract the common operation of creating a temporary directory to a new
file that is shared across tests.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
The typical usage of QDL so far has been for the user to manually
specify a programmer, as well as one or more program and patch XML
files, a so called "flat build". Some targets are built around multiple
storage medias (e.g. spinor + nvme), and some targets have a plethora of
sahara programmer files.
This puts a burden on the user to invoke qdl with the right combination
of storage selector, programmer(s), program and patch XML files. It also
adds an extra step to the build, in form of flattening the Qualcomm
builds.
The Qualcomm builds are described by a toplevel contents.xml file, which
describes to the various tools (e.g. the flattening tool) what
programmer, rawprogram, and patch files to use, and it provides mapping
of each referred file into the build directory structure.
The format also allow multiple "flavors" of flashable content to be
described, across multiple storage types.
Introduce support for parsing contents.xml files to allow the user to
flash non-flattened builds, across one or more storage devices and
filtered by select flavors.
The "qdl flash" operation is overloaded, to create a coherent user
experience. The filter/specifier interpretation is extended to allow
each, comma-separated, entry to describe storage types and flavors.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
There is a growing number of path concatenation cases, each built on
some memory allocation scheme and some variant of sprintf.
Introduce a pathbuf type and a few primitive operations to reset the
pathbuf, clone a pathbuf, push a path/filename to the pathbuf, get the
path, and truncate the path by dirname().
Also provide a convenient qdl_path_exist() operation to test if the
pathbuf refers to an existing file.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
The parsing of rawprogram files as well as the fallback and precedence
rules, is complex and subject for upcoming changes as part of the
introduction of contents XML support.
Introduce cmocka and a set of test cases to validate the <program>
entries in the rawprogram files are handled as expected.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
test_flashmap.sh and test_vip_generation.sh both wrote fixtures to
${builddir}/tests/data, and meson runs the two tests in parallel.
Their generate_flat_build.sh invocations therefore raced on dd, cp
and the final `zip flashmap.zip ...`, while test_vip_generation.sh's
EXIT trap could rm -rf the directory out from under its peer.
The race was masked while zip took several seconds on the unshrunk
1 GB fixtures; once fixture generation dropped to ~30 ms it started
firing on the macos-14 arm64 runner as zip exit status 18 ("could
not open a specified file to read").
Point each test at its own directory (data-flashmap / data-vip) so
they no longer share mutable state.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The fixture generator was creating efi.bin (512 MB) and rootfs.img
(500 MB) -- sizes inherited from a real-world device layout but with
no bearing on what the integration tests actually exercise. Both the
flashmap dry-run and the VIP digest test only need qdl to parse the
XML/JSON descriptors, walk the program list, and (for VIP) hash the
backing files. Whether the backing files are 8 MB or 512 MB makes no
difference to the code paths under test.
The previous switch to sparse files removed the dd write cost, but
zip still has to scan and CRC32 every byte of input regardless of
whether the bytes come from a hole. On the slow macos-15-intel CI
runner that scan was the new dominant cost and continued to push
both tests past meson's default 30 s timeout.
Drop efi.bin and rootfs.img to 8 MB each. Adjust rawprogram0.xml so
that the efi partition descriptor matches the new file size
(size_in_KB="8192.0", num_partition_sectors="2048") and rootfs's
start_sector / start_byte_hex follow on at sector 2054 (0x806000).
The other partition descriptors and patch entries are unaffected.
Local test run drops from ~5 s to ~0.25 s (combined). The slow
Intel CI runner should now have plenty of headroom under the 30 s
default timeout.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The integration tests' fixture generator dd'd ~1 GB of zeros to disk
for every test run (efi.bin = 512 MB, rootfs.img = 500 MB, plus
smaller files), and the script is invoked twice per `meson test`
invocation (once per test, with cleanup in between). Each test is
then capped by meson's default 30 s timeout. On the macos-15-intel
GitHub runner that combination was enough to push both tests over
the limit:
1/2 integration - qdl:VIP table generation TIMEOUT 30.05s
2/2 integration - qdl:Flashmap dry-run TIMEOUT 30.53s
while macos-14 arm64 (10 s / 16 s) and the Windows runners (14-23 s)
were already trending uncomfortably close. The tests don't actually
care about file *contents*: qdl --dry-run and the SHA256 of the VIP
digest both consume the byte stream as zeros either way, and zip
deflates zero blocks trivially.
Switch the generator from `dd ... bs=1024 count=N` (which writes
N KB) to `dd ... bs=1024 count=0 seek=N` (which extends the file to
N KB via lseek and writes nothing). On filesystems that support
sparse files (APFS, ext4, NTFS) the resulting file has zero
allocated blocks. Verified with `stat`:
count=0 seek=N: size=1048576 blocks=0 (sparse)
Reads of unallocated regions return zeros from the kernel without
disk I/O, so SHA256 input bytes are identical and the existing
EXPECTED_DIGEST in test_vip_generation.sh stays valid. Local test
time on Apple Silicon dropped from ~5-7 s to ~3-5 s; the much
larger gain is on the slow Intel CI runner where the writes were
the dominant cost.
Fixes: 6c0fbcce08 ("tests: add initial test set")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The flashmap support saves the user from having to specify programmer,
program and patch XML files, it allows reading the flashmap.json either
among the flat files, or as part of a zip-container of all the parts.
Add dry-run tests for the two cases, by introducing a flashmap.json and
zipping up all the files, then run qdl on the two versions.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
Place generated test artifacts (binaries, images) under the build
directory instead of the source tree, keeping the source tree clean
and following Meson's out-of-tree build convention.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Drop the run_tests.sh wrapper script and register each test suite
directly with Meson's test() function. This leverages Meson's native
test infrastructure for parallel execution, filtering by suite, and
per-test timeout/reporting instead of reimplementing it in shell.
Each test script is now self-contained: it generates its own fixtures
and cleans up via an EXIT trap, so tests can run independently and
in any order without relying on a shared setup/teardown wrapper.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Replace the existing Makefile-based build system with Meson to improve
build performance, cross-platform support, and maintainability.
Key improvements:
* Build performance: Shared source files (sahara.c, util.c, oscompat.c,
ux.c, sim.c, io.c) are now compiled once and linked into multiple
targets instead of being compiled separately for each executable.
This reduces total compilation units from ~30 to ~20 and speeds up
build.
Full build benchmark results below
Before (Make): $ time make
...
real 0m1.668s
user 0m1.424s
sys 0m0.242s
After (Meson): $ time bash -c '
meson setup build
meson compile -C build
'
...
real 0m0.759s
user 0m0.877s
sys 0m0.237s
* Flexible build directory: Unlike Make which builds in-tree, Meson
supports out-of-tree builds with custom build directories.
Example workflows:
meson setup builddir # Default debug build
meson setup build-release --buildtype=release
meson setup build-asan -Db_sanitize=address
meson setup build-arm64 --cross-file=aarch64-linux.txt
This enables:
- Multiple build configurations simultaneously
- Clean source tree (no .o files polluting git status)
- Easy comparison between debug/release builds
- Separate builds for different architectures
* Cross-compilation: Native support for cross-compilation via Meson
cross-files enables building for multiple architectures without
requiring separate build runners for each target platform.
Before (Make): Requires native ARM64 runner in GitHub Actions
runs-on: ubuntu-24.04-arm # Dedicated ARM64 hardware
run: make
After (Meson): Cross-compile from x64 host
runs-on: ubuntu-24.04 # x64 host
run: |
meson setup build-arm64 --cross-file=ci/aarch64-linux.txt
meson compile -C build-arm64
* Automatic dependency tracking: Meson automatically tracks header
dependencies, eliminating the need for manual dependency declarations
and preventing missed recompilation on header changes.
Before (Make): Manual dependency tracking
util.o: version.h # Must be manually maintained
After (Meson): Automatic header scanning
# No manual declarations needed
# Changing qdl.h automatically rebuilds all files that include it
Result: Prevents subtle bugs where changing a header doesn't trigger
necessary recompilation.
* Platform handling: Windows-specific linking (ws2_32) and platform
detection are now handled declaratively without shell conditionals.
* Testing integration: Test suite is now integrated into the build
system with proper dependency tracking and timeout handling.
Before (Make):
make tests # Shells out to bash script
After (Meson):
meson test -C builddir
meson test -C builddir --verbose
meson test -C builddir --wrapper='valgrind --leak-check=full'
Result: Better test output formatting, timeout handling, and ability
to run tests under valgrind or sanitizers.
* Installation: Standard installation paths and packaging support
through Meson's built-in install infrastructure.
Before (Make):
make install prefix=/usr/local DESTDIR=/tmp/package
After (Meson):
meson setup builddir --prefix=/usr/local
DESTDIR=/tmp/package meson install -C builddir
# Correctly installs to:
# - /usr/local/bin/{qdl,qdl-ramdump,ks}
# - /usr/local/share/man/man1/{qdl.1,qdl-ramdump.1,ks.1}
Result: Better integration with distribution packaging (Debian .deb,
RPM) and standard FHS paths.
Migration maintains full feature parity with the existing build system:
- All three executables (qdl, qdl-ramdump, ks) build correctly
- Version string generation from git tags
- Optional man page generation
- Support for Linux, macOS, and Windows (MSYS2/MinGW)
- Recreated tests integration via Meson’s test() function,
including support for script arguments (e.g. --builddir)
Usage:
$ meson setup build
$ meson compile -C build
$ meson test -C build
$ meson compile check -C build
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The expected digest value changed after recent updates to the
generated image content. The hardcoded checksum no longer matched
the artifact produced by the current build, causing verification
to fail.
Update the expected digest so checksum verification succeeds with
the latest generated image.
Signed-off-by: Yvonne Kaire <ykaire@qti.qualcomm.com>
Fix the capitalization of the 'ZlpAwareHost' and 'Verbose'
options, since the mis-capitalized versions seem to have no
effect on programmer behavior.
Signed-off-by: Steve Moskovchenko <stevemo@skydio.com>
[bjorn: Updated VIP digest]
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
Igor Opaniuk
Dmitry Baryshkov
Bjorn Andersson
Yvonne Kaire
Steve Moskovchenko