Populate the changelog for the upcoming v2.8 release, per step 1 of
the release process. Beyond the mirrored list of merged pull requests,
the entry introduces two curated sections: Highlights, summarizing the
release from a user's perspective, and Breaking changes, calling out
the dropped qdl-ks/qdl-ramdump standalone binaries and the rejection
of mixed XML-and-verb invocations that scripts and packaging may rely
on.
Tested-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com> (RB3Gen2 secure, Linux)
Tested-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com> (Uno Q, Linux)
Tested-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com> (Uno Q, Win)
Tested-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com> (RB3Gen2, MacOS)
Tested-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com> (Radxa Q8B, SC8280XP, Linux)
Tested-by: liuth <lth@t-chip.com.cn> (IQ-9075)
Tested-by: Christopher Obbard <chris.obbard@oss.qualcomm.com> (RB3Gen2, Debian Unstable amd64)
Tested-by: Shawn Guo <shengchao.guo@oss.qualcomm.com> (IQ10 RRD, Windows)
Tested-by: Pankaj Patil <pankaj.patil@oss.qualcomm.com> (Glymur CRD, Linux - Ubuntu 24.04.4 LTS)
Tested-by: Ananthu C V <ananthu.cv@oss.qualcomm.com> (Hamoa CRD, Ubuntu 24.04.4 LTS)
Tested-by: Shoudi Li <shoudil@qti.qualcomm.com> (Hamoa/Purwa IoT EVK, Ubuntu 24.04.4 LTS)
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
firehose_try_configure() discovers the storage sector size by reading
sector 1 of partition 0 with each candidate size. The provisioning
path also runs this probe, and on a device with unprovisioned UFS the
reads fail inside the programmer with "OPEN handle NULL and no error,
weird": there is no LU 0 to open. The programmer does not recover
from that failed open - the subsequent provisioning commit fails with
"ufs_open(0, WLUN_DEV) failed for configure" and the device is left
unprovisioned, while the same programmer, device and XML provision
fine through QFIL, which sends no reads before the ufs tags.
Gate the probe on skip_storage_init: the probe reads force the
programmer to initialize the very storage the caller declared
uninitialized, and provisioning - the only path passing
skip_storage_init - expresses UFS geometry in kilobytes and never
needs a sector size. Reported in the github issue [1].
[1] https://github.com/linux-msm/qdl/issues/294
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
rmtree() concatenates the directory path and entry name into a
PATH_MAX buffer without checking for truncation, which -Wformat-
truncation flags on the Windows build where PATH_MAX is 260. A
truncated path would also make the cleanup delete the wrong file in
principle.
Check the snprintf() result and skip entries that do not fit; using
the return value also satisfies the warning. A skipped entry means
the parent rmdir() fails with ENOTEMPTY and the workdir lingers, so
report the skip on stderr instead of hiding it - the message names
the entry, making a leftover workdir traceable to its cause.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The WSL2 section carries a typographic apostrophe and em dashes;
project documentation is plain ASCII (see AGENTS.md). Replace them
with their ASCII equivalents.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Describe the kernel-style stable branches (qdl-X.Y.y) agreed for
shipping urgent fixes independently of master: cut the branch from
the release tag immediately after tagging, backport fixes that landed
on master with cherry-pick -x through stable-targeted pull requests
(CI runs on those unchanged), and cut point releases (vX.Y.Z) from
the branch with the same flow as a normal release.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The markdown lint wrapper names README.md and docs/ explicitly, so
documents like AGENTS.md, CONTRIBUTING.md and RELEASING.md have never
been linted and accumulated findings silently.
Derive the file list from git ls-files instead, excluding
CHANGELOG.md - generated by the release process, where long
release-note URLs and per-release repeated section headers are part
of the format - and CLAUDE.md, a symlink to the already-linted
AGENTS.md. New documents are picked up automatically.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Preparation for linting every markdown file in the tree: surround the
sign-off example in AGENTS.md with the blank lines the linter wants,
and flatten the RELEASING.md table of contents - its single top-level
entry only repeated the document title, and the nested indentation
tripped the linter's list-indent rule under kramdown.
While at it, replace the literal Signed-off-by example with the
'git commit -s' command that produces it. checkpatch validates
anything shaped like a signature tag on any patch line - added,
removed and context lines alike - so any patch touching these lines
trips 'Unrecognized email address' on the $(git config ...)
placeholders. Documenting the command instead keeps trailer-shaped
text out of the file entirely, with no checkpatch exceptions needed.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The Windows branch reads path[1] guarded only by short-circuit
evaluation, which is correct at runtime but trips -Warray-bounds when
the function is inlined with a string literal shorter than two bytes,
as the oscompat unit tests do with "". It also passes a plain char
to isalpha(), which is undefined behavior for negative values.
Return early for the empty string, so the path[1] access is provably
in bounds, and feed isalpha() an unsigned char.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The signed VIP table (DigestsToSign.bin.mbn) and the chained digest
tables are opened in vip_transfer_init() with a plain O_RDONLY. On
Windows/MSYS2 a descriptor opened without O_BINARY defaults to text
mode, which performs CRLF translation and treats a 0x1A (Ctrl-Z) byte
as end-of-file.
These tables are arbitrary binary data (e.g. the 8192-byte signed
table). As soon as read() hits a 0x1A byte or collapses a CRLF pair it
returns fewer bytes than the file size, so vip_transfer_send_raw() fails
its n == sb.st_size length check and aborts with "failed to read
binary". The VIP configure request then fails. On Linux there is no
text/binary distinction (O_BINARY is defined as 0) so the same code
works fine.
Add O_BINARY to both open() calls in vip_transfer_init(), matching what
write_digests_to_table() already does, so the tables are read verbatim
on Windows as well.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
EDL, crash-mode and ramdump devices enumerate with a growing set of
product ids, and every newly discovered variant needed another
allowlist entry - devices with ids not yet on the list were silently
invisible until someone updated it. Both backends now identify the
EDL function by the vendor-specific interface signature instead: the
libusb backend via the config descriptors, the QUD backend via the
Windows compatible ids derived from them.
Trust the vendor id alone in qdl_is_edl_device() and drop the
now-redundant pid gate in the QUD enumeration. The pid parameter
stays in the signature so call sites are unchanged; the interface
gates are what reject non-EDL Qualcomm devices from here on.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The QUD backend accepts any Qualcomm COM port whose pid is on the
shared EDL allowlist, while the libusb backend identifies the EDL
function by its interface signature - vendor-specific class and
subclass with a known Sahara protocol. Windows exposes exactly that
triple without opening the device: PnP derives compatible ids of the
form "USB\Class_xx&SubClass_xx&Prot_xx" for every USB devnode,
including each interface of a composite device.
Require the same signature set the libusb backend matches (0x10, 0x11
and 0x13 on modern devices, 0xff on older targets), bringing the two
backends' notion of an EDL device in line and filtering Qualcomm COM
ports that expose no vendor-specific function at all.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
decode_sahara_config() was defined in qdl.c - the command-line front-end -
but it is also called from contents.c, which is part of the shared qdl
library. The qdl binary links both, so it built and ran fine; but any
program that links the library without qdl.c ends up with an undefined
reference to decode_sahara_config().
That is exactly what happens to the nbdkit plugin, which is built from
the shared sources plus its own entry point and no qdl.c: loading the
resulting shared object aborts with "undefined symbol:
decode_sahara_config".
Move the function into a dedicated src/sahara_config.c that is part of
the shared source set, so the library is self-contained. It is kept in
its own translation unit rather than folded into contents.c so that the
contents unit tests, which #include contents.c and stub this function,
keep working.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Kernel-style commit bodies occasionally wrap a sentence or an
example command a few characters past 72, which tripped
commitlint's body-max-line-length even though the messages are
otherwise fine. Raise the body limit from 72 to 75 to match the
wrapping already used in the tree.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The usage synopsis documents input XML files and the command verbs
(read/write/erase/sha256/flash/reset) as separate invocation forms,
but the argument loop accepted any mixture of them. Combining a bare
program XML with the reset verb, for example, appended the reset op
during parsing - before qdl_determine_bootable() adds the
setbootablestoragedrive op after the loop - so the device could be
told to reboot before its bootable storage drive was set. The same
mixture on the provisioning path silently performed no reset at all.
Rather than order the two against each other, reject the combination:
track whether a file-type and a verb-type argument have both been
seen and error out when they have. This keeps each documented form
self-consistent and makes the reset verb usable only on its own or
alongside other verbs, where ordering is well defined.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Several long-standing robustness gaps in the Firehose layer:
- firehose_write() keyed its timeout retry on errno, but the qdl_write()
backends report the timeout through the return value and never touch
errno, so the retry could misfire and a stale errno leaked into the
return code. Key both the retry and the error return on the return
value, and free the serialized document on the VIP send-table error
path.
- firehose_configure_response_parser() leaked the response value on the
ACK path when MaxPayloadSizeToTargetInBytesSupported was absent; free
it.
- firehose_program() and firehose_issue_read() divided by a sector size
that could be zero; guard against it and fail cleanly. Route the
VIP send-table failure through the existing cleanup path instead of
returning with the document and buffer still allocated.
- firehose_detect_and_configure() ignored its skip_storage_init argument
and always passed false; honor the caller's request.
- Normalize a few error returns to -1 so callers see a consistent
failure status.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
firehose_read() read up to sizeof(buf) bytes and then wrote a NUL at
buf[n], overflowing the stack buffer by one byte whenever a read filled
it - routine on stream transports that concatenate responses. Read at
most sizeof(buf) - 1 bytes.
Fixes: da68b4add8 ("qdl: Support for program and patch")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
firehose_program() logged a failed or NAK'd final programming response
but then returned 0, so firehose_execute_ops() carried on and qdl
exited successfully with the partition left unwritten. Return an error
on a non-ACK final response.
Fixes: a9ec681705 ("program: Refactor the global "programs" list")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
qdl ramdump passes a NULL images array, but the READ_DATA handlers and
the done-pending quirk indexed it unconditionally and crashed when the
device requested an image. Treat NULL images as "none". Also make the
offset+len range check overflow-safe, use a signed type for write()'s
result, propagate I/O errors from the ramdump helper, bound the region
loop by the bytes actually read, and reject region filenames that
contain a path separator.
Fixes: ff1a3bb1d3 ("ramdump: Make ramdump a qdl subcommand")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
GPT LBAs are 64-bit but were narrowed to unsigned int, so a partition
beyond 2 TB wrapped and a name-addressed write/erase hit the wrong
sectors. Carry start_sector/num_sectors as uint64_t and range-check
before storing num_sectors. Also reject a bad part_entry_size, fix an
off-by-one in the UTF-16 surrogate handling that read past the name
buffer, and initialise the sim-mode outputs.
Fixes: 60db50966d ("qdl: Extend read/write support to accept GPT partition names")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The image-id checks used "id == 0", but id comes from strtoul(), so an
id above LONG_MAX parsed negative, passed the check and indexed the
sahara_image array out of bounds.
Reject id <= 0 and also require the cpio entry name to be non-empty
and NUL-terminated before strtok_r()/strcmp(), and treat an empty
image_path as invalid rather than calling strlen(NULL).
Fixes: 4bdcaaec1a ("qdl: Support for "programmer archives"")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
A program tagged sparse="true" with no sparse header but a matching
size was appended to the op list twice - once in program_load_sparse()
and again by the caller - self-linking the node (infinite loop /
use-after-free) and dropping its filename so it is never flashed.
Return a distinct code so the caller appends it once. Also guard the
sparse path against a zero SECTOR_SIZE_IN_BYTES (SIGFPE) and empty
start_sector/label attributes (NULL deref).
Fixes: 6a67b6bd64 ("sparse: Parse sparse files at load time")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
flashmap_load() left "ret" holding the 0 from a successful open, so a
failed load or JSON parse returned success and qdl continued with an
empty op list instead of aborting.
Set ret to -1 on both paths, and NULL-check the configure-op allocation.
Fixes: 81b68cbdb2 ("flashmap: Introduce flashmap support")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
zipper_write() reused "ret" as a loop scratch variable, so several
error paths jumped to the discard label without resetting it to -1:
create-zip then exited 0 with no output file.
Set ret before those gotos, also hand sparse fill buffers straight to
libzip instead of copying them a second time, which could double peak
memory on large fills.
Fixes: cc6d9c3596 ("zipper: Introduce support for creating a flashmap zip file")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Extend the hardware-in-the-loop suite with self-contained steps that
fit the existing round-trip model:
- list devices: enumerate the attached EDL device with the list verb
and require a vid:pid line in the output, covering device discovery
over real USB.
- chipinfo via Sahara: read the chip identity (serial number, HW ID
and OEM PK hash) with the chipinfo verb. The step runs before the
programmer upload because chip identity is only available while the
device still speaks Sahara; the verb switches the device back to
image-transfer mode on exit, so the subsequent flash still gets its
Sahara HELLO. SKIPs when a programmer is already running.
- create-zip and flash: package a build from a contents.xml with the
create-zip verb, then flash the produced zip, covering the zip
writer and the zip flashing path end to end. Gated on
QDL_HIL_CONTENTS.
- flash sparse build: flash a bundle that uses Android sparse images,
covering the sparse decoding path against real storage. Gated on
QDL_HIL_SPARSE.
- NAK on bad partition: a read of a nonexistent physical partition
must fail, verifying that a device NAK is reported as an error
instead of silently succeeding.
- sha256 by partition name: digest a partition addressed by its GPT
name rather than by partition and sector numbers, covering GPT name
resolution against the device.
- write chunked: rewrite the partition image with an overridden
--out-chunk-size=16384, covering the chunked transfer path.
- verify write integrity: compare the device-computed partition
digest against a local sha256 of the previously read-back and
rewritten image, proving the read/write round trip is bit-exact.
- reset board: switch the final step to the new reset verb, covering
the explicit device-reset path instead of relying on the implicit
reset after a dummy 1-sector read.
The two gated steps SKIP when their environment variable is unset,
and every step SKIPs without hardware, so a plain meson test stays
green.
Example invocation against an attached EDL device:
D=<path_to_build>
QDL_HIL_BUILD=$D QDL_HIL_CONTENTS=$D/contents.xml \
QDL_HIL_STORAGE=ufs meson test -C build --suite=hil
1/18 hil - qdl:HIL list devices OK 0.05s
2/18 hil - qdl:HIL chipinfo via Sahara OK 0.4s
3/18 hil - qdl:HIL flash whole build OK 29.03s
4/18 hil - qdl:HIL create-zip and flash OK 79.62s
5/18 hil - qdl:HIL flash sparse build SKIP 0.02s
...
18/18 hil - qdl:HIL reset board OK 3.68s
Ok: 17
Fail: 0
Skipped: 1
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
win_enumerate_qcom() accepted any present COM port whose hardware id
starts with USB\VID_05C6&PID_, i.e. any Qualcomm serial port the
QDLoader driver (or another vendor serial driver) has bound. That
over-matches: the diag or NMEA port of a phone in normal composite
mode also enumerates this way, and would be counted by
qud_probe_present(), listed by 'qdl list' and even opened and driven
by qud_open() as if it were an EDL device.
Gate the enumeration with qdl_is_edl_device(), the same identity
policy the libusb backend applies. The backends can no longer drift
apart on what counts as an EDL device: extending the policy (for
example with a ramdump-specific pid) is a one-line change in qdl.h
that both transports pick up.
Devices whose hardware id carries no parsable pid were previously
listed with pid 0 and are now skipped.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
usb_open_device() interleaved matching and acting: the interface
signature check, the device open, the serial filter and the claim all
lived in one loop, with the device-level serial check nested inside
the per-interface iteration. A serial mismatch was therefore
discovered only after opening the device, and rediscovered - open,
read, close - for every matching interface.
Introduce usb_match_device(), which answers the complete device-level
question on an opened candidate: does it carry the requested serial,
and does it expose an EDL interface? usb_open_device() becomes a
plain sequence: open, match, detach and claim, fill in the endpoint
pair. The match helpers now form one family: usb_match_device() is
usb_match_serial() (renamed from usb_match_usb_serial() to drop the
stutter) plus usb_match_edl_interface() over the active config.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
usb_open_once() and usb_list() each carried their own copy of the
same enumeration loop: walk the bus, fetch descriptors, apply the EDL
identity gate. On top of that, usb_open_once() applied the gate
before handing a device to usb_open_device(), which then fetched the
same descriptor and applied the same gate a second time; the
duplicated check let the two layers disagree about what counts as an
EDL candidate.
Extract the loop into usb_enumerate_edl_devices(), which hands every
candidate to a callback and reports how many it saw, and pass the
already-fetched descriptor down to usb_open_device() so candidate
selection happens exactly once, in exactly one place. Only what
happens to a matched candidate now differs between the paths: the
open path claims the first candidate that opens, while the list verb
reads the descriptors of all of them without claiming anything
(listing must never detach a kernel driver or claim an interface, as
that could disturb a concurrently running flash). usb_list() no
longer sizes its result array from the raw device count but grows it
per matched candidate.
The libusb session stays with the callers, since their lifetimes
differ: usb_open_once() must keep the session alive for the handle it
returns, while usb_list() tears it down before returning.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
usb_list() open-codes the iProduct "_SN:" parsing that
usb_read_serial() already implements for the open path. Reuse the
helper so the serial-number format knowledge exists once.
Two edge cases change deliberately: a device whose iProduct string
cannot be read is now listed with a "(none)" serial instead of
being skipped, and a serial longer than the 16-byte descriptor field
is truncated (as the open path already does) instead of hiding the
device. Suppressing a present device was the more surprising
behavior for a listing verb.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
usb_open_device() mixes three concerns in one hundred-line function:
deciding whether a USB device is an EDL candidate, locating the
vendor-specific EDL interface with its bulk endpoint pair, and
actually opening and claiming it. usb_open_once() and usb_list()
additionally duplicate the vid/pid gate inline, so the answer to
"what does qdl consider an EDL device" is currently scattered over
three call sites that can (and in the past did) drift apart.
Centralize the policy:
- qdl_is_edl_pid()/qdl_is_edl_device() in qdl.h hold the shared,
transport-agnostic identity check (Qualcomm vid, known EDL pids).
Living in the common header, the same policy is available to the
QUD backend, which parses pids from Windows hardware ids.
- usb_is_edl_device() wraps it for libusb descriptor types and is
now the single gate used by the open, scan and list paths.
- usb_match_edl_interface() checks one interface descriptor for the
ff/ff class/subclass signature and a known protocol, and extracts
the bulk endpoint pair.
usb_open_device() shrinks to the actual open/claim sequence. Future
changes to device acceptance (for example crash-mode devices exposing
a different interface protocol or pid) become a one-line change in
exactly one place, applied consistently everywhere.
No functional change.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
try_usb_open() and usb_try_open() differ only in word order, which
makes the libusb open path needlessly hard to follow: one is a full
enumeration pass over the bus, the other opens a single candidate
device.
Rename them so the names encode the layering instead of hiding it:
usb_open() - backend .open op, wait loop (unchanged)
usb_open_once() - one enumeration pass (was try_usb_open())
usb_open_device() - open a single candidate (was usb_try_open())
Document the resulting call chain, including the auto backend that
sits on top of usb_open_once() on Windows, at the top of usb.c.
No functional change.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
usb_list() initializes a libusb session but never calls
libusb_exit() on any path, and the empty-bus early return also leaks
the device list itself. This is harmless for the one-shot 'qdl list'
verb, but it is inconsistent with usb_open_once(), which sets up and
tears down its session on every pass, and it trips leak checkers.
Free the device list and close the session on all exit paths.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
QDL only ever drives Sahara in its image-transfer and memory-debug
modes. Sahara also defines a command mode that lets the primary
bootloader be queried for the device's identity before any programmer
is uploaded. Expose that as a new "chipinfo" subcommand.
It enters command mode by requesting SAHARA_MODE_COMMAND in the HELLO
response and echoing the device's advertised protocol version, then
drives EXECUTE transactions to read the chip serial number, the HW ID
(broken out into MSM_ID / OEM_ID / MODEL_ID) and the OEM PK hash.
Version 3 targets no longer answer MSM_HW_ID_READ, so the same fields
are recovered through READ_CHIP_ID_V3 on those devices.
Command mode is left by switching back to image-transfer mode, which
returns the device to its power-on HELLO state and keeps it usable for
a subsequent query or flash. A Sahara reset is deliberately avoided
here: on some targets it leaves the device enumerated on PID 0x9008 but
no longer answering Sahara until it is power-cycled back into EDL.
This is handy for identifying an attached target and, in particular,
for troubleshooting secure-boot provisioning: the fused OEM PK hash and
OEM ID can be read straight from the PBL without loading a programmer.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The success line for a matched EDL device is emitted at open time,
before any data is transferred, and it is shared by every operation
that opens a device - including read-only ones like the new chipinfo
query. Labelling that moment "Flashing" is inaccurate: nothing is being
flashed yet, and for a pure query nothing will be.
Use the operation-neutral verb "Talking to", which is accurate at
connect time regardless of what follows. The crash-dump wording for the
0x900e product ID is left untouched, as it remains specific and correct.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Claude Code looks for its project instructions in CLAUDE.md and
ignores AGENTS.md.
Add a CLAUDE.md symlink pointing to AGENTS.md so Claude Code reads
the same instructions without duplicating the content.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Automation agents contributing to QDL need a single, machine-oriented
reference for how to build, test, and format their changes. Scattering
this information across the README and CONTRIBUTING.md makes it easy for
agents to miss the project conventions.
Add AGENTS.md consolidating the build prerequisites, the pre-commit
build and check targets, the Linux kernel coding style requirement, the
subsystem-tagged commit convention with a mandatory rationale body, the
ASCII-only rule for code, docs, and commit messages, and the mandatory
S-o-b trailer. It points back to CONTRIBUTING.md so the two stay
in sync.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The README advertised only the license and CI build status, leaving
the project's availability invisible: qdl ships a tagged GitHub
release and is packaged across many distributions (Debian, Arch,
Fedora, Ubuntu, Nix, ...).
Add three badges near the title:
- latest GitHub release, so the current version is visible at a glance;
- Debian package version, linking to the Debian package tracker;
- Repology packaging status, summarizing distro coverage in one badge.
Also fix the existing build badge link, which pointed at the badge image
instead of the workflow's runs page.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The release artifacts were not self-contained. qdl links libxml2 and
libzip (and, on Linux, their transitive closure), but packaging copied
only a subset of those libraries, so qdl failed to start on hosts
without them installed. Inspecting the uploaded artifacts:
Artifact Missing Status
--------------- ---------------------------- ------
ubuntu-22-x64 libxml2.so.2, libzip.so.4 broken
ubuntu-22-arm64 libxml2.so.2, libzip.so.4 broken
ubuntu-24-x64 libxml2.so.2, libzip.so.4 broken
ubuntu-24-arm64 libxml2.so.2, libzip.so.4 broken
macos-arm64 libzip.5.dylib (Homebrew) broken
macos-intel libzip.5.dylib (Homebrew) broken
windows-x64 - ok
windows-arm64 - ok
On Linux only libusb was copied; libxml2 and libzip (plus their closure:
libz, liblzma, libzstd, libbz2, libcrypto, libudev) were left to the
host. On macOS libzip was never handled, so the binary kept an absolute
Homebrew path to libzip.5.dylib. Windows already bundled the full set.
Bundle the complete dependency closure on Linux (every ldd entry except
the C runtime and dynamic loader) and set an $ORIGIN RUNPATH on the
binary and each bundled library, since RUNPATH is not inherited by
transitively-loaded libraries.
On macOS, use dylibbundler to recursively copy the non-system dylibs and
rewrite the load commands to @executable_path, replacing the
hand-maintained install_name_tool calls that missed libzip.
Both jobs now fail if any non-system library remains unresolved after
packaging.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Cutting a release has so far been an informal, tribal-knowledge effort:
the exact steps, the tag conventions, which hardware/driver combinations
must be validated, and how the GitHub release is assembled from CI
artifacts.
Add RELEASING.md to formalize and standardize that process so any
maintainer can cut a release the same way. It documents, end to end:
- how to populate CHANGELOG.md from the merged PRs and start testing;
- opening the changelog PR and tagging release candidates;
- the validation matrix (HIL suite across Windows+QUD, Windows+WinUSB,
Linux and macOS) plus manual VIP validation, and rc-tag iteration;
- applying the final immutable release tag (with a note on the
annotated vs lightweight tag history);
- creating the GitHub release from the existing Buildtest artifacts,
with release notes extracted straight from CHANGELOG.md.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Provide an in-tree changelog so the release history is available without
visiting GitHub. Each entry corresponds to a published GitHub release
(v2.4 onwards), listing the merged pull requests and new contributors,
with PR references rendered as #NNN links.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The README has grown to cover build instructions, a wide range of
usage workflows (EDL, WSL2, VIP, multi-programmer targets, crash
dumps, kickstart, ...) and the contribution policy in a single file,
which makes it long and harder to navigate.
Move the contribution documentation - PR workflow, Linux kernel coding
style, the checkpatch (check-cached/check-range/check) and markdown-lint
targets - into a dedicated CONTRIBUTING.md. GitHub surfaces that file
automatically on the new-PR and issue pages, so contributors find the
guidance without scrolling through the README. The README keeps a short
Contributing section linking to it.
No content was changed in the move; only its location.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
mdl's MD004 rule defaults to "consistent" style, taking the first
unordered list's marker as the expected one for the whole file. The
first list used "*" while every other list used "-", so the dash lists
were flagged. Switch the three "*" bullets to "-" to match the dominant
style and satisfy the linter.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
test_contents_selectors.c pulls in contents.c via #include, and
contents.c calls firehose_alloc_op(). The test target links neither
firehose.c nor libqdl_common, so the only reason it linked at all is
that at -O2 the compiler drops the (unreachable in this test) call.
At -O0 the call survives and the link fails:
src/contents.c:899: undefined reference to `firehose_alloc_op'
Provide a firehose_alloc_op() stub, like the other contents tests
already do, so the test links regardless of optimization level.
Closes: https://github.com/linux-msm/qdl/issues/260
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Before tagging a release we want to exercise qdl against real
hardware - flashing a full build, reading partitions, skipblock,
sha256, erase and write - but there is no automated way to do that
today. Add a "hil" meson suite that drives the built qdl binary
against an attached EDL device, so a release can be validated with a
single command. It is a stop-gap that simplifies testing until we have
proper CI HIL coverage via LAVA.
Each step is its own test in the "hil" suite. The steps are stateful,
so they run serially and in order (is_parallel:false plus a descending
priority): flash the build, read the EFI partition / GPT / address
forms, re-flash with --skipblock=sha256, run the sha256 command, erase
and write the EFI partition back, and finally reboot. Every step runs
with --skip-reset except the reboot. The erase/write pair round-trips
the partition's own readback, so nothing is left wiped, and once one
step fails the rest of the suite stops.
The suite is opt-in: the build and storage type are passed via the
environment, and when unset every step exits 77 (SKIP), so a plain
"meson test" / CI run stays green without hardware.
Usage:
$ QDL_HIL_BUILD=/path/to/build QDL_HIL_STORAGE=ufs \
meson test -C build --suite hil -v
1/10 hil - qdl:HIL flash whole build OK 21.63s
2/10 hil - qdl:HIL read EFI partition OK 10.50s
3/10 hil - qdl:HIL read GPT table OK 2.47s
4/10 hil - qdl:HIL read address P/S+L OK 2.46s
5/10 hil - qdl:HIL read address P/name OK 11.17s
6/10 hil - qdl:HIL flash skipblock sha256 OK 22.12s
7/10 hil - qdl:HIL sha256 command OK 2.47s
8/10 hil - qdl:HIL erase EFI partition OK 4.97s
9/10 hil - qdl:HIL write EFI partition OK 10.18s
10/10 hil - qdl:HIL reboot board OK 3.64s
Ok: 10 Fail: 0
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
firehose_issue_read() treated a 0-byte qdl_read() return as a fatal
"unexpected EOF" and aborted the transfer. On a USB bulk-in pipe a
0-byte completion is not EOF, it is a zero-length packet (ZLP) - a
legitimate USB-level event that the transport layer can surface to
user space.
In practice this trips on Windows with the libusb (WinUSB) backend
when reading a large partition: after the device ACKs a <read> with
rawmode="true" and transitions from XML responses to the raw binary
phase, it emits a ZLP before the first sector chunk. WinUSB returns
that as libusb_bulk_transfer() success with actual=0, usb_read()
forwards 0, and the inner accumulation loop in firehose_issue_read()
treated it as a terminated stream and bailed out. The same workload
succeeds on Linux because the usbfs path does not surface the
transitional ZLP on the rawmode boundary, so the first qdl_read()
already returns the full payload.
Small reads (the per-sector GPT loads at the start of every session
go through the same loop) did not hit this: their request size is a
single sector, the device sends the bytes immediately, and there is
no preceding boundary ZLP to consume. The failure only showed up on
the first big payload, e.g.:
$ qdl.exe --debug --storage=emmc prog_firehose_ddr.elf \
read 0/rootfs rootfs.image
...
FIREHOSE READ: <response value="ACK" rawmode="true" />
unexpected EOF while reading sector data
Treat n == 0 as "no payload yet, keep reading" instead of EOF and
continue the inner loop. A genuine stall is still caught: qdl_read()
returns -ETIMEDOUT once the 30s timeout passed to libusb expires,
and that path already errors out via the n < 0 arm just above.
Fixes: 0512c0b983 ("firehose: parse multi-document and rawmode-trailed reads")
Closes: https://github.com/linux-msm/qdl/issues/132
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a cmocka unit test for the header-only helpers in oscompat.h:
- path_is_absolute(), checking POSIX and (under _WIN32) drive-letter
and UNC paths;
- qdl_mkdir_p(), checking recursive creation, idempotency on an
existing directory, trailing slashes, the "." and empty-path cases,
the ENAMETOOLONG guard, and failure when a path component is not a
directory.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
"qdl ramdump -o <dir>" writes each downloaded region to <dir>/<file>,
but it never created <dir> itself. When the directory did not exist
every open() failed with ENOENT, and the user only saw a confusing
per-file error with no hint that the missing output directory was the
actual problem:
$ qdl ramdump -o ./ramdump
Waiting for EDL device
Collecting crash dump from device (PID 0x900e, serial: 8AE002FE)
qdl: failed to open "OCIMEM.BIN": No such file or directory
Create the output directory, and any missing parents like "mkdir -p",
before the download starts via a new portable qdl_mkdir_p() helper, so
the ramdump path is guaranteed to exist. Failing to create it is now
reported explicitly instead of surfacing as an opaque open() error.
Closes: https://github.com/linux-msm/qdl/issues/224
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The --skipblock=sha256 fast-path issued a single <getsha256digest> over
the entire <program> region before deciding whether to write it. On a
multi-GB partition such as rootfs the device has to read and hash the
whole region synchronously, which easily exceeds
firehose_getsha256digest()'s 30 s read timeout. Worse, while the device
is busy hashing it does not service the bulk-OUT endpoint, so qdl's next
write stalls and fails - turning the intended optimization into a hard
flashing failure.
Walk the region in 512 MiB chunks instead. Each chunk is hashed locally,
compared against the device's digest for the matching flash sub-region,
and reflashed on its own (via an independent <program>) only when it
differs; matching chunks are left untouched. No single <getsha256digest>
now covers more than 512 MiB, keeping every digest well within the read
timeout and never blocking the device long enough to wedge the following
write.
Non-sparse, non-NAND, VIP-disabled programs are still the only ones
eligible, and a region smaller than one chunk behaves exactly as the
previous whole-region compare did.
Closes: https://github.com/linux-msm/qdl/issues/261
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
With kickstart now available as `qdl ks`, the standalone qdl-ks binary
is the same redundancy qdl-ramdump was: a second entry point, meson
target and manpage for code the main binary already carries. Removing
it completes the consolidation onto a single qdl binary.
Drop the binary and its build/doc scaffolding, and rewrite the README
kickstart section to use `qdl ks`. The description is kept, not
removed -- the feature lives on, only its entry point changed.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Kickstart (loading images to a flashless-boot device such as the
Qualcomm Cloud AI 100 over Sahara) only exists as the standalone
qdl-ks binary.
Fold it into the single qdl binary the way ramdump already is.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
qdl-ramdump exposed crash-dump collection as its own binary, but the
exact same code is already reachable as `qdl ramdump`. Keeping a second
entry point for it costs a meson target, a manpage and README real
estate, and gives users a second name to learn, all for functionality
the main binary already provides. Consolidating on a single qdl binary
(as suggested in review) removes that duplication.
Drop the standalone binary and everything that only existed to build
and document it; the `qdl ramdump` subcommand is unchanged.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Every source change funnells through the single ~200-line
root meson.build that enumerated the sources for all binaries,
so touching one area means editing a file that mostly described
unrelated code.
Declare the source lists in src/meson.build and the tests in
tests/meson.build and pull them in with subdir(). The root meson.build
is then left to compose (dependencies, version header, targets) while
the list of files to build lives next to the files themselves, so
adding or removing a source is a one-line edit in one obvious place.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The repository root had accumulated ~45 .c/.h files sitting directly
next to the project files (meson.build, scripts/, tests/, docs).
Move all sources under src/ and the public header to include/ so the
top level reflects the shape of the project rather than its source
file count.
This commit is a pure git mv with no content changes, so git log
--follow keeps working across the move.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a CODEOWNERS file so every pull request automatically requests a
review from the @linux-msm/qdl-maintainers team, instead of relying on
contributors to add reviewers by hand. The wildcard pattern covers the
whole tree; ownership can later be narrowed per path if needed.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The --skipblock=sha256 fast-path could appear to hang before a large
<program> such as rootfs: qdl_should_skipblock() reads the whole file to
build a local SHA-256, then waits on the device's <getsha256digest> over
the same flash region, all without printing anything. On a multi-hundred
MiB partition both steps take a noticeable while and look like a stall.
Narrate each phase so the wait is explained:
- "hashing ... locally" before the local read/hash,
- "requesting flash digest" before the device digest query,
- "sha256 mismatch ..., flashing partition" when the digests differ
and the partition falls back to the normal program path.
The messages are emitted only once the skipblock guards have passed, so
runs with skipblock disabled or not applicable (sparse/NAND/VIP) stay
quiet as before. The existing "skipped ... (sha256 match)" message
covers the match case.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
qdl_should_skipblock() builds a local SHA-256 over the bytes that would
be written and compares it against the device's <getsha256digest> of the
same flash region, skipping the <program> on a match.
For the comparison to be valid the local digest must cover exactly what
the program path puts on flash. That path always sends a full
num_sectors-sized region and zero-pads any residue past EOF (see the
memset() in firehose_program()). This happens whenever the file size is
not a sector multiple, or a non-zero file_sector_offset is used while
num_sectors still spans the whole file. The skipblock path instead
treated that short read as "cannot build a matching digest" and returned
0 before ever issuing <getsha256digest>, so those partitions were always
reprogrammed and never compared.
Mirror the write path: hash the file bytes within the region, then hash
zeros for the remainder up to num_sectors * sector_size. Files that fill
the region exactly are unchanged; short or offset files now produce a
digest that can match flash, letting xbl/tz/rpm/hyp-style partitions be
skipped like boot/dtb already were.
Reported-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
Closes: https://github.com/linux-msm/qdl/issues/258
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The .gitreview file configured git-review to push changes for code
review to a Gerrit instance at review.linaro.org. The project's review
workflow has since moved to GitHub pull requests against
github.com/linux-msm/qdl, and the Linaro Gerrit project is no longer
used. Remove the stale config so contributors are not misled into
sending patches to an abandoned review server.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The recently added free() calls in load_erase_tag() pass
program->start_sector directly to free(), but the field is declared
as const char * in struct firehose_op. GCC emits a
-Wdiscarded-qualifiers warning for both call sites.
Cast through (void *) to match the pattern already used at the other
free sites for the same field (program.c, patch.c, firehose.c).
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Both directions of host/device VIP misconfiguration currently surface
as opaque failures that operators in the field have to diagnose from
raw byte-stream errors, when the right answer is fully derivable from
the programmer's own startup logs. Do the detection here so customers
don't have to debug this manually on every new platform.
The Firehose programmer announces VIP activation by emitting an
"INFO: VIP is enabled, receiving the signed table of size N" log line
during startup. Watch for that marker in firehose_read() and use it in
firehose_detect_and_configure() to handle the two mismatch cases:
- --vip-table-path missing, programmer requires VIP: previously qdl
would send configure, the programmer would wait silently for a
signed table that never comes, and the user would see only a
generic 5s configure timeout. Now bail out immediately with an
explicit error pointing at the missing option.
- --vip-table-path given, programmer not in VIP mode: previously qdl
would push the signed digest table at the programmer, which tried
to parse those binary bytes as XML and bailed out with "Failed to
parse xml at offset 4090, 4096". Now detect the absence of the
marker after the startup-log drain, print a warning, demote
vip_data.state to VIP_DISABLED so the table is never sent, and
continue flashing normally.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Automated flashing pipelines often re-program identical content on every
run, which dominates total flash time on slow storage like spinor. With
the <getsha256digest> primitive now exposed, qdl can ask the device for
the SHA-256 of the target region before each <program> and skip the
write when it already matches what would have been sent.
Add --skipblock=<none|sha256>, defaulting to none. When sha256 is
selected, firehose_program() hashes the bytes it would otherwise stream
to the device - reading num_partition_sectors * SECTOR_SIZE_IN_BYTES
from the file with zero-padding for the trailing partial sector, exactly
matching what the existing chunked write path puts on the wire - then
asks the device for the digest of the same region via
firehose_getsha256digest(), reusing the op->digest field that the
sha256 subcommand path also reads. On a match, the <program> is skipped
entirely and "skipped \"<label>\" (sha256 match)" is logged in place of
the usual flashed-at-rate line.
The optimization is gated on three conditions, all enforced silently:
- !program->sparse: each sparse chunk would need its own digest;
keeping sparse images on the normal program path avoids per-chunk
round trips and the bookkeeping that would come with them.
- !program->is_nand: NAND spare/OOB byte semantics differ across
programmers, so the digest the device returns is not guaranteed to
correspond to the bytes qdl would compute locally.
- VIP disabled: <getsha256digest> is not part of pre-built VIP
digest tables, so issuing it under VIP would cause a hash mismatch
on the device.
Any failure on the optimization path - file read error, query failure,
digest absent, or digest mismatch - falls through to the normal program
flow. The file position is restored unconditionally by the existing
qdl_file_seek() at the top of the write loop, so the fall-through path
is byte-identical to --skipblock=none.
[1] https://github.com/linux-msm/qdl/issues/237
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
firehose.c is the firehose protocol layer; deciding what to do with a
digest (print it for the user, compare it to a locally-computed one)
is qdl policy. Move the result back onto struct firehose_op so the
parser can fill it directly, drop the local intermediate result struct,
and let firehose_run() return without emitting anything to stdout.
For the `sha256` subcommand, format and print each filled digest in
qdl_flash() once firehose_run() succeeds. The visible output for the
subcommand is unchanged.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Expose the FIREHOSE_OP_GET_SHA256_DIGEST primitive as a user-facing
verb. The syntax mirrors the existing read / write / erase verbs and
reuses parse_storage_address(), so addresses can be expressed as
<P>/<S>+<L>, by GPT partition name, or any of the other supported
forms.
Each invocation prints one lowercase 64-character hex digest per region
to stdout, so the output can be diffed against a sha256sum(1) line or
piped into other tooling without further parsing.
Typical use is verifying that what is on the device matches a known
local image - useful on shared bench machines where multiple people
flash the same board:
$ qdl prog_firehose_ddr.elf sha256 xbl_a
...
c288e50a94687a130a64db608c83678777ce5e7f9229248696c6f0dc41406cc9
$ qdl prog_firehose_ddr.elf read xbl_a xbl_a.bin
...
$ sha256sum xbl_a.bin
c288e50a94687a130a64db608c83678777ce5e7f9229248696c6f0dc41406cc9
Multiple sha256 verbs can be chained in a single invocation, with one
digest line emitted per region in order.
[1] https://github.com/linux-msm/qdl/issues/237
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Firehose programmers can compute a SHA-256 over a region of flash on the
target side via the <getsha256digest> command, but qdl currently has no
way to issue it. Bench workflows that want to answer "what's actually
flashed on this board?" today have to read the partition back over USB
and hash it locally - slow on every storage type and particularly bad on
spinor.
Introduce a new firehose op type, FIREHOSE_OP_GET_SHA256_DIGEST, and a
matching firehose_getsha256digest() that builds the XML request with the
same address attributes used by <read> / <program> / <erase>
(SECTOR_SIZE_IN_BYTES, num_partition_sectors, physical_partition_number,
start_sector, and slot when applicable).
The digest response format is not consistent across Firehose variants.
Most implementations emit the digest inside a <log value="..."/> stream
prior to the ACK, but some attach it as an attribute on the <response>
itself. Add firehose_sha256_parser() which scans every incoming log
payload for a 64-hex-character run bounded by non-hex characters and,
as a fallback, probes the response node for Digest / SHA256 / sha256
attributes. The parser still defers ACK / NAK / rawmode handling to
firehose_generic_parser() so behaviour is identical to other commands
on the control path.
On success the digest is printed to stdout as a single lowercase
64-character hex line, separate from the ux_info / ux_log channels, so
it can be cleanly captured or piped without stripping prefixes.
gpt_resolve_deferrals() learns about the new op type so callers can
address regions by GPT partition name, matching the ergonomics of
<read>.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The libusb_open() call in usb_list() fails routinely on Windows
because the QDLoader 9008 driver claims matched devices. Printing a
warning per failing match drowns the actual rows that qdl list emits.
The open-attempt path in usb_open_attempt() still warns, so the
diagnostic survives where it is actionable.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a meta-backend QDL_DEVICE_AUTO that defers the transport choice
to qdl_open(): instead of picking libusb or QUD upfront, its open()
runs a unified wait loop and binds whichever transport first reaches
an EDL device. The "auto" --backend token (and the default) maps to
it.
Each 250 ms tick the loop does:
1. usb_open_attempt() - one libusb scan-and-open. On success the
libusb device is bound as the inner transport.
2. On Windows, if libusb saw the device but could not open it
(-EBUSY: the QDLoader 9008 driver has claimed the interface)
or SetupAPI sees a Qualcomm COM port libusb missed, calls
qud_open(). On success the QUD device is bound as the inner.
3. Otherwise sleeps and retries. No timeout: the loop waits
indefinitely, same cadence as the prior libusb-only loop.
qdl_read/write/close on the outer auto wrapper forward to the inner.
Pre-open out_chunk_size is stashed on the wrapper and applied at
bind time; vip_data and slot live on the outer where firehose
accesses them, so no forwarding is needed for those.
usb_open_attempt() (factored out of usb_open()) returns
0 / -ENODEV / -EBUSY / -EIO. The --backend usb path loops over it
with visibility-transition messaging. The auto loop owns its own
messaging and never emits the -EBUSY diagnostic because that
condition triggers the QUD attempt in the same tick.
qdl-ramdump now uses QDL_DEVICE_AUTO so Windows users with the
official driver can collect crash dumps without re-binding the USB
driver to WinUSB.
QDL_DEVICE_AUTO is explicitly a meta-backend, not a peer of the USB
and QUD backends: its job is to defer the transport choice to the
moment a probe reaches a device. The libusb and QUD backends remain
standalone and selectable via --backend usb / --backend qud.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
firehose_read() previously assumed each qdl_read() returns a single
Firehose XML document. That holds on libusb (each USB bulk-in
transfer surfaces as its own read return), but not on stream-oriented
transports. The Windows COM port driver behind the QUD backend just
streams bytes into its RX buffer, and a single ReadFile() can deliver
several back-to-back Firehose responses concatenated; on top of that
the binary payload of a rawmode response (e.g. the sector data that
follows a <read> ACK) can be spliced onto the same buffer right after
the closing </data> tag.
Walk the read buffer using "<?xml" as the document start and
"</data>" as the envelope end, then dispatch each XML chunk through
the existing firehose_response_parse() + response_parser path.
<log/> and <response/> elements are still honoured in arrival order,
and rawmode is observed on whichever chunk carries it. When that
chunk is followed by trailing bytes, those bytes are the first
sectors of the binary payload: hand them to qdl_push_back() so the
next qdl_read() picks them up before the transport is touched again.
firehose_issue_read() now accumulates each sector-chunk across
qdl_read() calls instead of requiring a single full-sized return,
since on a stream transport the rawmode tail and the remaining
sector bytes generally arrive in several reads. libusb keeps
delivering an entire bulk transfer in one call, so the inner loop
runs once there.
The change is a no-op on libusb where each read already contains
exactly one document and the rawmode payload arrives in its own
bulk transfer; it unblocks all current and future stream-oriented
transports without touching the libusb path's behaviour.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Wire up an actual transport for the QUD backend on Windows. The
official QDLoader 9008 driver exposes a Qualcomm EDL device as a
serial port (\\.\COMx). The Sahara/Firehose framing on top is
identical to the libusb backend; this patch only deals with how
user space reaches the underlying bulk pipes.
Non-Windows hosts keep the stub from the previous patch.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Introduce QDL_DEVICE_QUD as a third device-transport flavour alongside
the libusb-based USB backend and the SIM backend used by --dry-run.
After this patch --backend=qud parses successfully but errors out at
init time on every host. The actual transport is added in a follow-up.
The intent is to support kernel-mode drivers that expose the EDL
device as a character file (e.g. the official QDLoader 9008 driver on
Windows, where the device shows up as \\.\COMx), so libusb is not
required on hosts where claiming the USB interface from user space is
inconvenient.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
When chaining flash invocations - typically with --skip-reset to leave
the programmer alive between calls - re-uploading the programmer via
Sahara is unnecessary, and re-entering Sahara on a device that is
already serving Firehose simply hangs until timeout.
Detect this on the very first read of sahara_run(): if the device emits
a Firehose XML banner ("<?xml") instead of a Sahara HELLO, or if the
read times out, assume the programmer is already running and return
success so the caller proceeds straight to the Firehose phase.
The detection is gated on the call site: ramdump mode, where Sahara is
the only valid protocol, retains the previous strict behaviour.
Discarding the early Firehose banner is harmless - firehose_run() drives
its own request/response exchange and does not depend on prior data.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Some workflows require the device to remain in the programmer state
after flashing - for example, when chaining multiple flash operations,
running post-flash verification, or debugging with the programmer still
responsive. The unconditional reset at the end of firehose_run() and
firehose_provision() makes this impossible without patching the binary.
Model the reset as a regular firehose op (FIREHOSE_OP_RESET) that
firehose_execute_ops() dispatches like any other operation. qdl.c then
appends a reset op at the end of the ops list, gated on the new -R /
--skip-reset CLI flag, instead of teaching firehose_run() a special
case. firehose_run() no longer issues a reset on its own.
The UFS provisioning path does not consume a firehose ops list, so it
keeps a dedicated reset call; firehose_provision() now takes the
skip_reset boolean as a parameter rather than reading it back off the
qdl_device, keeping all CLI-level state out of struct qdl_device.
Default behaviour is unchanged.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Both the qdl_common static library and each of the
qdl/qdl-ramdump/qdl-ks executables were declared with the same
`dependencies: common_dep` (libusb-1.0, libxml-2.0, libzip,
plus ws2_32 on Windows). Meson propagates a static library's
dependencies to its consumers via `link_with`, so listing those
dependencies again on each executable caused every common library to
appear twice on the final link line. Apple's ld surfaced this as:
ld: warning: ignoring duplicate libraries: '-lxml2'
The other dependencies are passed by absolute dylib path on macOS
(libusb-1.0, libzip), so the linker silently deduplicated those while
still warning about libxml2, which is referenced as `-lxml2` from the
system path.
Fix it by introducing a compile-only view of the common dependencies
using `partial_dependency(compile_args: true, includes: true)` and
attaching that to the executables instead of the full deps. The
executables still see the headers needed to compile their own sources
(e.g. usb.c -> libusb.h, file.c -> zip.h), while link arguments now
flow to them exactly once via the static library's automatic
propagation.
Fixes: ff189b1a12 ("build: migrate from GNU Make to Meson build system")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
libxml2 prints warnings and errors directly to stderr by default. This
leaks raw parser diagnostics into both test output and user-facing
output when an XML file is missing or malformed, bypassing the tool's
ux_err() output layer.
Install a structured error handler via xmlSetStructuredErrorFunc() in
ux_init() that formats each diagnostic as
libxml2 <level>: <file>:<line>: <message>
and routes it through ux_err(). This preserves the file, line and
reason detail libxml2 produces for malformed XML while keeping output
flow consistent with the rest of the tool, and applies uniformly to
every xmlReadFile()/xmlReadMemory() call without per-site flag
changes.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The config relaxes footer-max-line-length to 200 specifically to
accommodate long Signed-off-by addresses, but commitlint's default
parser only treats `BREAKING CHANGE` as a note keyword. Other
kernel-style trailers (Fixes, Reviewed-by, Tested-by, ...) get
classified as body lines, so a trailer that exceeds 72 characters --
typically a `Fixes:` quoting a long commit subject -- trips
body-max-line-length even though the relaxed footer limit was meant
to cover exactly this case.
Configure parserPreset.parserOpts.noteKeywords with the trailer
tokens this project uses, so those lines move into the footer
section and inherit the existing footer-max-line-length=200 rule.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
test_flashmap.sh and test_vip_generation.sh both wrote fixtures to
${builddir}/tests/data, and meson runs the two tests in parallel.
Their generate_flat_build.sh invocations therefore raced on dd, cp
and the final `zip flashmap.zip ...`, while test_vip_generation.sh's
EXIT trap could rm -rf the directory out from under its peer.
The race was masked while zip took several seconds on the unshrunk
1 GB fixtures; once fixture generation dropped to ~30 ms it started
firing on the macos-14 arm64 runner as zip exit status 18 ("could
not open a specified file to read").
Point each test at its own directory (data-flashmap / data-vip) so
they no longer share mutable state.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The fixture generator was creating efi.bin (512 MB) and rootfs.img
(500 MB) -- sizes inherited from a real-world device layout but with
no bearing on what the integration tests actually exercise. Both the
flashmap dry-run and the VIP digest test only need qdl to parse the
XML/JSON descriptors, walk the program list, and (for VIP) hash the
backing files. Whether the backing files are 8 MB or 512 MB makes no
difference to the code paths under test.
The previous switch to sparse files removed the dd write cost, but
zip still has to scan and CRC32 every byte of input regardless of
whether the bytes come from a hole. On the slow macos-15-intel CI
runner that scan was the new dominant cost and continued to push
both tests past meson's default 30 s timeout.
Drop efi.bin and rootfs.img to 8 MB each. Adjust rawprogram0.xml so
that the efi partition descriptor matches the new file size
(size_in_KB="8192.0", num_partition_sectors="2048") and rootfs's
start_sector / start_byte_hex follow on at sector 2054 (0x806000).
The other partition descriptors and patch entries are unaffected.
Local test run drops from ~5 s to ~0.25 s (combined). The slow
Intel CI runner should now have plenty of headroom under the 30 s
default timeout.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The integration tests' fixture generator dd'd ~1 GB of zeros to disk
for every test run (efi.bin = 512 MB, rootfs.img = 500 MB, plus
smaller files), and the script is invoked twice per `meson test`
invocation (once per test, with cleanup in between). Each test is
then capped by meson's default 30 s timeout. On the macos-15-intel
GitHub runner that combination was enough to push both tests over
the limit:
1/2 integration - qdl:VIP table generation TIMEOUT 30.05s
2/2 integration - qdl:Flashmap dry-run TIMEOUT 30.53s
while macos-14 arm64 (10 s / 16 s) and the Windows runners (14-23 s)
were already trending uncomfortably close. The tests don't actually
care about file *contents*: qdl --dry-run and the SHA256 of the VIP
digest both consume the byte stream as zeros either way, and zip
deflates zero blocks trivially.
Switch the generator from `dd ... bs=1024 count=N` (which writes
N KB) to `dd ... bs=1024 count=0 seek=N` (which extends the file to
N KB via lseek and writes nothing). On filesystems that support
sparse files (APFS, ext4, NTFS) the resulting file has zero
allocated blocks. Verified with `stat`:
count=0 seek=N: size=1048576 blocks=0 (sparse)
Reads of unallocated regions return zeros from the kernel without
disk I/O, so SHA256 input bytes are identical and the existing
EXPECTED_DIGEST in test_vip_generation.sh stays valid. Local test
time on Apple Silicon dropped from ~5-7 s to ~3-5 s; the much
larger gain is on the slow Intel CI runner where the writes were
the dominant cost.
Fixes: 6c0fbcce08 ("tests: add initial test set")
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Until now checkpatch had two divergent code paths: the local meson
'check' target ran 'checkpatch.pl --no-tree -f' against every tracked
source file, while the CI workflow used the webispy/checkpatch-action
action, which runs checkpatch in patch mode against each commit in the
PR. The two modes report different defects -- in particular, file mode
flags style issues in long-untouched code and skips commit-log checks,
while patch mode validates the commit message (Signed-off-by, subject
length, ...) and only inspects added/changed lines. Contributors could
end up with a green local 'check' but a red CI run, and vice versa.
Introduce a 'check-range' mode in scripts/checkpatch_wrapper.sh that
mirrors what the GitHub Action does: iterate over every commit in
CHECKPATCH_BASE..CHECKPATCH_HEAD (defaulting to origin/master..HEAD),
feed each commit through 'git format-patch -1 --stdout' and pipe the
result into checkpatch.pl. All commits are processed even on failure
so the user sees every offending commit in one go.
Expose the new mode as a meson run_target ('meson compile check-range
-C build') and switch the CI workflow to invoke the wrapper directly
with the PR base/head SHAs, so local and CI invocations exercise the
exact same code path. Document the new target in README under the
Contributing section.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The markdown lint check previously only ran in CI via the
markdownlint-cli2 GitHub Action, which meant developers had no
straightforward way to reproduce the check locally before pushing. As a
result, contributors only discovered lint errors after opening a PR.
Introduce scripts/markdownlint_wrapper.sh that drives mdl on the same
set of files the CI workflow lints. mdl is shipped as a Debian/Ubuntu
package, so contributors can install it with 'sudo apt install mdl'
without pulling in a Node.js toolchain. Wire the wrapper up as a meson
run_target so it can be invoked with:
meson compile markdown-lint -C build
Switch the markdown-lint workflow to install mdl from apt and invoke
the wrapper directly, making the script the single source of truth for
both local and CI runs. Document the usage in README under the
Contributing section.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The alias_target('manpages', manpage_targets) call lived outside the
'if help2man.found()' / 'can_run_exe' block, but manpage_targets is
only populated inside it. When help2man is absent (or this is a cross
build), manpage_targets is undefined and meson setup fails. The same
call would also fail if reached with an empty list, since alias_target
requires at least one dependency.
Move the alias_target declaration inside the same guard so it is only
emitted when there are actual manpage targets to alias.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
The crash dump section only mentioned the 'qdl ramdump' subcommand,
but the build also produces a standalone qdl-ramdump executable that
exposes the same functionality. Mention it so users who only need
crash-dump collection know they can use it directly.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a section describing qdl-ks, the "kickstart" Sahara helper used to
load images from the host to flashless-boot devices such as the
Qualcomm Cloud AI 100, which fetch their runtime firmware from the
host on every boot rather than storing it on-device.
Explain why qdl-ks is a separate tool (no USB, no Firehose; talks to a
kernel-provided device node via open/read/write), describe the two
required arguments, and show a sample invocation. Note that mapped
files do not need to exist at invocation time -- if qdl-ks cannot open
a requested file the device decides the next action, which is useful
for wiring qdl-ks into a single udev rule covering multiple device
configurations (e.g. an optional DDR training image).
The description is based on the commit bdcf93b4a3 ("Add ks utility")
message.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Place generated test artifacts (binaries, images) under the build
directory instead of the source tree, keeping the source tree clean
and following Meson's out-of-tree build convention.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Drop the run_tests.sh wrapper script and register each test suite
directly with Meson's test() function. This leverages Meson's native
test infrastructure for parallel execution, filtering by suite, and
per-test timeout/reporting instead of reimplementing it in shell.
Each test script is now self-contained: it generates its own fixtures
and cleans up via an EXIT trap, so tests can run independently and
in any order without relying on a shared setup/teardown wrapper.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Replace the existing Makefile-based build system with Meson to improve
build performance, cross-platform support, and maintainability.
Key improvements:
* Build performance: Shared source files (sahara.c, util.c, oscompat.c,
ux.c, sim.c, io.c) are now compiled once and linked into multiple
targets instead of being compiled separately for each executable.
This reduces total compilation units from ~30 to ~20 and speeds up
build.
Full build benchmark results below
Before (Make): $ time make
...
real 0m1.668s
user 0m1.424s
sys 0m0.242s
After (Meson): $ time bash -c '
meson setup build
meson compile -C build
'
...
real 0m0.759s
user 0m0.877s
sys 0m0.237s
* Flexible build directory: Unlike Make which builds in-tree, Meson
supports out-of-tree builds with custom build directories.
Example workflows:
meson setup builddir # Default debug build
meson setup build-release --buildtype=release
meson setup build-asan -Db_sanitize=address
meson setup build-arm64 --cross-file=aarch64-linux.txt
This enables:
- Multiple build configurations simultaneously
- Clean source tree (no .o files polluting git status)
- Easy comparison between debug/release builds
- Separate builds for different architectures
* Cross-compilation: Native support for cross-compilation via Meson
cross-files enables building for multiple architectures without
requiring separate build runners for each target platform.
Before (Make): Requires native ARM64 runner in GitHub Actions
runs-on: ubuntu-24.04-arm # Dedicated ARM64 hardware
run: make
After (Meson): Cross-compile from x64 host
runs-on: ubuntu-24.04 # x64 host
run: |
meson setup build-arm64 --cross-file=ci/aarch64-linux.txt
meson compile -C build-arm64
* Automatic dependency tracking: Meson automatically tracks header
dependencies, eliminating the need for manual dependency declarations
and preventing missed recompilation on header changes.
Before (Make): Manual dependency tracking
util.o: version.h # Must be manually maintained
After (Meson): Automatic header scanning
# No manual declarations needed
# Changing qdl.h automatically rebuilds all files that include it
Result: Prevents subtle bugs where changing a header doesn't trigger
necessary recompilation.
* Platform handling: Windows-specific linking (ws2_32) and platform
detection are now handled declaratively without shell conditionals.
* Testing integration: Test suite is now integrated into the build
system with proper dependency tracking and timeout handling.
Before (Make):
make tests # Shells out to bash script
After (Meson):
meson test -C builddir
meson test -C builddir --verbose
meson test -C builddir --wrapper='valgrind --leak-check=full'
Result: Better test output formatting, timeout handling, and ability
to run tests under valgrind or sanitizers.
* Installation: Standard installation paths and packaging support
through Meson's built-in install infrastructure.
Before (Make):
make install prefix=/usr/local DESTDIR=/tmp/package
After (Meson):
meson setup builddir --prefix=/usr/local
DESTDIR=/tmp/package meson install -C builddir
# Correctly installs to:
# - /usr/local/bin/{qdl,qdl-ramdump,ks}
# - /usr/local/share/man/man1/{qdl.1,qdl-ramdump.1,ks.1}
Result: Better integration with distribution packaging (Debian .deb,
RPM) and standard FHS paths.
Migration maintains full feature parity with the existing build system:
- All three executables (qdl, qdl-ramdump, ks) build correctly
- Version string generation from git tags
- Optional man page generation
- Support for Linux, macOS, and Windows (MSYS2/MinGW)
- Recreated tests integration via Meson’s test() function,
including support for script arguments (e.g. --builddir)
Usage:
$ meson setup build
$ meson compile -C build
$ meson test -C build
$ meson compile check -C build
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Add a subsection under the VIP section that explains the workflow,
what QDL verifies, and why running a dry-run validation before touching
hardware saves time and avoids leaving devices in a broken state.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Without any verification on the host side, a dry-run with pre-built VIP
tables (-n -D <path>) would silently succeed even when the digest tables
did not match the data being flashed. The only way to catch a mismatch
was to attempt a real flash and wait for the device to reject it, which
wastes time and risks leaving a device in a broken state.
Add hash validation to the simulator so mismatches are caught at dry-run
time, before any hardware is involved.
The signed initial table (DigestsToSign.bin.mbn) and all chained tables
(ChainedTableOfDigests<n>.bin) are parsed as they arrive. Chain hashes
linking each table to the next are verified to ensure table integrity.
Per-chunk SHA256 hashes are checked in order against every XML command
write and raw payload write; any mismatch is reported via ux_err()
without aborting so all mismatches remain visible at the end of a run.
Validation is active in exactly the intended modes:
-n plain dry-run, no tables sent -- no-op
-n -D <path> dry-run with VIP tables -- full hash validation
-n -t <dir> digest generation -- validation disabled
MAX_DIGESTS_PER_SIGNED_FILE and MAX_DIGESTS_PER_CHAINED_FILE are moved
from vip.c to vip.h so sim.c can reference them. sha2.c is added to
RAMDUMP_SRCS because qdl-ramdump links sim.o which now calls SHA256.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Replace the stub sim_read()/sim_write() with a simulator that
parses incoming XML commands and enqueues matching XML responses,
mirroring the real Firehose device protocol. This decreases divergence
in behavior between the real HW and the dry-run mode, giving better
validation of the flashing process in dry-run mode.
sim_write() now dispatches based on the command tag:
- configure: enqueue log + ACK echoing the same max payload size so
firehose_try_configure() does not trigger a redundant exchange
- program/read: enqueue log + rawmode=true ACK and transition to
SIM_STATE_RAW_IN/RAW_OUT to track the raw binary transfer
- erase/patch/setbootablestoragedrive/ufs: enqueue log + ACK
- power: enqueue log + ACK and mark the simulator as closed
sim_read() first drains the XML response queue (ensuring protocol
responses are delivered before any raw data), then serves zero-filled
sector data in SIM_STATE_RAW_OUT, and returns -EIO once closed so
the trailing drain in firehose_reset() completes immediately.
Since the simulator now produces correct protocol responses, remove the
QDL_DEVICE_SIM early-return guards in firehose_read() and
firehose_try_configure() that previously bypassed response parsing.
Pre-initialize qdl->sector_size = 4096 in sim_init() and only probe
sector size in firehose_try_configure() if unset. Previously, the
probe <read> commands were executed during --createdigest dry-runs and
included in the VIP digest table, but not issued during real VIP
flashing (where probing is gated on VIP_DISABLED). This caused all
subsequent digest entries to shift, resulting in hash mismatches on
the device.
Signed-off-by: Igor Opaniuk <igor.opaniuk@oss.qualcomm.com>
Igor Opaniuk