The contents.xml provides two different "root path" tags, one with Linux
and one with Windows path delimiters. Use the Windows tag to avoid
issues related to path traversal.
Extend the tests as necessary.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
Introduce two initial set of test cases for the newly introduced
contents.xml handling.
One is used to validate the storage and flavor selector logic, and the
other more broadly the loading of contents.xml files.
Extract the common operation of creating a temporary directory to a new
file that is shared across tests.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
The typical usage of QDL so far has been for the user to manually
specify a programmer, as well as one or more program and patch XML
files, a so called "flat build". Some targets are built around multiple
storage medias (e.g. spinor + nvme), and some targets have a plethora of
sahara programmer files.
This puts a burden on the user to invoke qdl with the right combination
of storage selector, programmer(s), program and patch XML files. It also
adds an extra step to the build, in form of flattening the Qualcomm
builds.
The Qualcomm builds are described by a toplevel contents.xml file, which
describes to the various tools (e.g. the flattening tool) what
programmer, rawprogram, and patch files to use, and it provides mapping
of each referred file into the build directory structure.
The format also allow multiple "flavors" of flashable content to be
described, across multiple storage types.
Introduce support for parsing contents.xml files to allow the user to
flash non-flattened builds, across one or more storage devices and
filtered by select flavors.
The "qdl flash" operation is overloaded, to create a coherent user
experience. The filter/specifier interpretation is extended to allow
each, comma-separated, entry to describe storage types and flavors.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
There is a growing number of path concatenation cases, each built on
some memory allocation scheme and some variant of sprintf.
Introduce a pathbuf type and a few primitive operations to reset the
pathbuf, clone a pathbuf, push a path/filename to the pathbuf, get the
path, and truncate the path by dirname().
Also provide a convenient qdl_path_exist() operation to test if the
pathbuf refers to an existing file.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
The parsing of rawprogram files as well as the fallback and precedence
rules, is complex and subject for upcoming changes as part of the
introduction of contents XML support.
Introduce cmocka and a set of test cases to validate the <program>
entries in the rawprogram files are handled as expected.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
test_flashmap.sh and test_vip_generation.sh both wrote fixtures to
${builddir}/tests/data, and meson runs the two tests in parallel.
Their generate_flat_build.sh invocations therefore raced on dd, cp
and the final `zip flashmap.zip ...`, while test_vip_generation.sh's
EXIT trap could rm -rf the directory out from under its peer.
The race was masked while zip took several seconds on the unshrunk
1 GB fixtures; once fixture generation dropped to ~30 ms it started
firing on the macos-14 arm64 runner as zip exit status 18 ("could
not open a specified file to read").
Point each test at its own directory (data-flashmap / data-vip) so
they no longer share mutable state.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The fixture generator was creating efi.bin (512 MB) and rootfs.img
(500 MB) -- sizes inherited from a real-world device layout but with
no bearing on what the integration tests actually exercise. Both the
flashmap dry-run and the VIP digest test only need qdl to parse the
XML/JSON descriptors, walk the program list, and (for VIP) hash the
backing files. Whether the backing files are 8 MB or 512 MB makes no
difference to the code paths under test.
The previous switch to sparse files removed the dd write cost, but
zip still has to scan and CRC32 every byte of input regardless of
whether the bytes come from a hole. On the slow macos-15-intel CI
runner that scan was the new dominant cost and continued to push
both tests past meson's default 30 s timeout.
Drop efi.bin and rootfs.img to 8 MB each. Adjust rawprogram0.xml so
that the efi partition descriptor matches the new file size
(size_in_KB="8192.0", num_partition_sectors="2048") and rootfs's
start_sector / start_byte_hex follow on at sector 2054 (0x806000).
The other partition descriptors and patch entries are unaffected.
Local test run drops from ~5 s to ~0.25 s (combined). The slow
Intel CI runner should now have plenty of headroom under the 30 s
default timeout.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The integration tests' fixture generator dd'd ~1 GB of zeros to disk
for every test run (efi.bin = 512 MB, rootfs.img = 500 MB, plus
smaller files), and the script is invoked twice per `meson test`
invocation (once per test, with cleanup in between). Each test is
then capped by meson's default 30 s timeout. On the macos-15-intel
GitHub runner that combination was enough to push both tests over
the limit:
1/2 integration - qdl:VIP table generation TIMEOUT 30.05s
2/2 integration - qdl:Flashmap dry-run TIMEOUT 30.53s
while macos-14 arm64 (10 s / 16 s) and the Windows runners (14-23 s)
were already trending uncomfortably close. The tests don't actually
care about file *contents*: qdl --dry-run and the SHA256 of the VIP
digest both consume the byte stream as zeros either way, and zip
deflates zero blocks trivially.
Switch the generator from `dd ... bs=1024 count=N` (which writes
N KB) to `dd ... bs=1024 count=0 seek=N` (which extends the file to
N KB via lseek and writes nothing). On filesystems that support
sparse files (APFS, ext4, NTFS) the resulting file has zero
allocated blocks. Verified with `stat`:
count=0 seek=N: size=1048576 blocks=0 (sparse)
Reads of unallocated regions return zeros from the kernel without
disk I/O, so SHA256 input bytes are identical and the existing
EXPECTED_DIGEST in test_vip_generation.sh stays valid. Local test
time on Apple Silicon dropped from ~5-7 s to ~3-5 s; the much
larger gain is on the slow Intel CI runner where the writes were
the dominant cost.
Fixes: 6c0fbcce08 ("tests: add initial test set")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The flashmap support saves the user from having to specify programmer,
program and patch XML files, it allows reading the flashmap.json either
among the flat files, or as part of a zip-container of all the parts.
Add dry-run tests for the two cases, by introducing a flashmap.json and
zipping up all the files, then run qdl on the two versions.
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
Place generated test artifacts (binaries, images) under the build
directory instead of the source tree, keeping the source tree clean
and following Meson's out-of-tree build convention.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Drop the run_tests.sh wrapper script and register each test suite
directly with Meson's test() function. This leverages Meson's native
test infrastructure for parallel execution, filtering by suite, and
per-test timeout/reporting instead of reimplementing it in shell.
Each test script is now self-contained: it generates its own fixtures
and cleans up via an EXIT trap, so tests can run independently and
in any order without relying on a shared setup/teardown wrapper.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Replace the existing Makefile-based build system with Meson to improve
build performance, cross-platform support, and maintainability.
Key improvements:
* Build performance: Shared source files (sahara.c, util.c, oscompat.c,
ux.c, sim.c, io.c) are now compiled once and linked into multiple
targets instead of being compiled separately for each executable.
This reduces total compilation units from ~30 to ~20 and speeds up
build.
Full build benchmark results below
Before (Make): $ time make
...
real 0m1.668s
user 0m1.424s
sys 0m0.242s
After (Meson): $ time bash -c '
meson setup build
meson compile -C build
'
...
real 0m0.759s
user 0m0.877s
sys 0m0.237s
* Flexible build directory: Unlike Make which builds in-tree, Meson
supports out-of-tree builds with custom build directories.
Example workflows:
meson setup builddir # Default debug build
meson setup build-release --buildtype=release
meson setup build-asan -Db_sanitize=address
meson setup build-arm64 --cross-file=aarch64-linux.txt
This enables:
- Multiple build configurations simultaneously
- Clean source tree (no .o files polluting git status)
- Easy comparison between debug/release builds
- Separate builds for different architectures
* Cross-compilation: Native support for cross-compilation via Meson
cross-files enables building for multiple architectures without
requiring separate build runners for each target platform.
Before (Make): Requires native ARM64 runner in GitHub Actions
runs-on: ubuntu-24.04-arm # Dedicated ARM64 hardware
run: make
After (Meson): Cross-compile from x64 host
runs-on: ubuntu-24.04 # x64 host
run: |
meson setup build-arm64 --cross-file=ci/aarch64-linux.txt
meson compile -C build-arm64
* Automatic dependency tracking: Meson automatically tracks header
dependencies, eliminating the need for manual dependency declarations
and preventing missed recompilation on header changes.
Before (Make): Manual dependency tracking
util.o: version.h # Must be manually maintained
After (Meson): Automatic header scanning
# No manual declarations needed
# Changing qdl.h automatically rebuilds all files that include it
Result: Prevents subtle bugs where changing a header doesn't trigger
necessary recompilation.
* Platform handling: Windows-specific linking (ws2_32) and platform
detection are now handled declaratively without shell conditionals.
* Testing integration: Test suite is now integrated into the build
system with proper dependency tracking and timeout handling.
Before (Make):
make tests # Shells out to bash script
After (Meson):
meson test -C builddir
meson test -C builddir --verbose
meson test -C builddir --wrapper='valgrind --leak-check=full'
Result: Better test output formatting, timeout handling, and ability
to run tests under valgrind or sanitizers.
* Installation: Standard installation paths and packaging support
through Meson's built-in install infrastructure.
Before (Make):
make install prefix=/usr/local DESTDIR=/tmp/package
After (Meson):
meson setup builddir --prefix=/usr/local
DESTDIR=/tmp/package meson install -C builddir
# Correctly installs to:
# - /usr/local/bin/{qdl,qdl-ramdump,ks}
# - /usr/local/share/man/man1/{qdl.1,qdl-ramdump.1,ks.1}
Result: Better integration with distribution packaging (Debian .deb,
RPM) and standard FHS paths.
Migration maintains full feature parity with the existing build system:
- All three executables (qdl, qdl-ramdump, ks) build correctly
- Version string generation from git tags
- Optional man page generation
- Support for Linux, macOS, and Windows (MSYS2/MinGW)
- Recreated tests integration via Meson’s test() function,
including support for script arguments (e.g. --builddir)
Usage:
$ meson setup build
$ meson compile -C build
$ meson test -C build
$ meson compile check -C build
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The expected digest value changed after recent updates to the
generated image content. The hardcoded checksum no longer matched
the artifact produced by the current build, causing verification
to fail.
Update the expected digest so checksum verification succeeds with
the latest generated image.
Signed-off-by: Yvonne Kaire <ykaire@qti.qualcomm.com>
Fix the capitalization of the 'ZlpAwareHost' and 'Verbose'
options, since the mis-capitalized versions seem to have no
effect on programmer behavior.
Signed-off-by: Steve Moskovchenko <stevemo@skydio.com>
[bjorn: Updated VIP digest]
Signed-off-by: Bjorn Andersson <bjorn.andersson@oss.qualcomm.com>
There is a regression in the current implementation of simulation mode,
which leads to sending two configure packets:
FIREHOSE WRITE: <?xml version="1.0"?>
<data><configure MemoryName="ufs" MaxPayloadSizeToTargetInBytes="1048576"
verbose="0" ZLPAwareHost="1" SkipStorageInit="0"/></data>
FIREHOSE WRITE: <?xml version="1.0"?>
<data><configure MemoryName="ufs" MaxPayloadSizeToTargetInBytes="0"
verbose="0" ZLPAwareHost="1" SkipStorageInit="0"/></data>
In simulated mode "remote" target can't propose different size, so
we just don't re-send configure packet.
Move the check for simulation mode to prevent sending additional
configure packets with the attribute MaxPayloadSizeToTargetInBytes=0.
Also change DIGEST_TABLE.bin sha256 hash to
a05e1124edbe34dc504a327544fb66572591353dc3fa25e6e7eafbe4803e63e0,
as amount of packets supposed to be sent to the target changed.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add tests make target and a simple test that executes qdl in dry-run mode
for a synthetic reference FLAT build with VIP table generation.
The FLAT build contains:
- patch0.xml
- patch1.xml
- rawprogram0.xml
- rawprogram1.xml
All binaries that these XML files point to are filled with zeros, generated
during github action execution.
Tests ensures that the table is generated correctly by comparing
calculated and expected SHA256 hashes of DigestToSign.bin file.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Bjorn Andersson
Igor Opaniuk
Yvonne Kaire
Steve Moskovchenko