When adf_dev_up() partially completes and then fails, the IRQ
handlers registered during adf_isr_resource_alloc() are not detached
before the MSI-X vectors are released.
Since the device is enabled with pcim_enable_device(), calling
pci_alloc_irq_vectors() internally registers pcim_msi_release() as a
devres action. On probe failure, devres runs pcim_msi_release() which
calls pci_free_irq_vectors(), tearing down the MSI-X vectors while IRQ
handlers (for example 'qat0-bundle0') are still attached. This causes
remove_proc_entry() warnings:
[ 22.163964] remove_proc_entry: removing non-empty directory 'irq/143', leaking at least 'qat0-bundle0'
Moving the devm_add_action_or_reset() before adf_dev_up() does not solve
the problem since devres runs in LIFO order and pcim_msi_release(),
registered later inside adf_dev_up(), would still fire before
adf_device_down().
Fix by calling adf_dev_down() explicitly when adf_dev_up() fails, to
properly free IRQ handlers before devres releases the MSI-X vectors.
Fixes: 17fd7514ae ("crypto: qat - add qat_6xxx driver")
Cc: stable@vger.kernel.org
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Laurent M Coquerel <laurent.m.coquerel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add support for the ZSTD algorithm for QAT GEN4, GEN5 and GEN6 via the
acomp API.
For GEN4 and GEN5, compression is performed in hardware using LZ4s, a
QAT-specific variant of LZ4. The compressed output is post-processed to
generate ZSTD sequences, and the ZSTD library is then used to produce
the final ZSTD stream via zstd_compress_sequences_and_literals(). Only
inputs between 8 KB and 512 KB are offloaded to the device. The minimum
size restriction will be relaxed once polling support is added. The
maximum size is limited by the use of pre-allocated per-CPU scratch
buffers. On these generations, only compression is offloaded to hardware;
decompression always falls back to software.
For GEN6, both compression and decompression are offloaded to the
accelerator, which natively supports the ZSTD algorithm. There is no
limit on the input buffer size supported. However, since GEN6 is limited
to a history size of 64 KB, decompression of frames compressed with a
larger history falls back to software.
Since GEN2 devices do not support ZSTD or LZ4s, add a mechanism that
prevents selecting GEN2 compression instances for ZSTD or LZ4s when a
GEN2 plug-in card is present on a system with an embedded GEN4, GEN5 or
GEN6 device.
In addition, modify the algorithm registration logic to allow
registering the correct implementation, i.e. LZ4s based for GEN4 and
GEN5 or native ZSTD for GEN6.
Co-developed-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Laurent M Coquerel <laurent.m.coquerel@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace __builtin_bswap32() with swab32 in icp_qat_hw_20_comp.h to fix
the following build errors on architectures without native byte-swap
support:
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.o: in function `adf_gen4_build_decomp_block':
drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:141:(.text+0xeec): undefined reference to `__bswapsi2'
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:141:(.text+0xef8): undefined reference to `__bswapsi2'
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/adf_gen4_hw_data.o: in function `adf_gen4_build_comp_block':
drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:57:(.text+0xf64): undefined reference to `__bswapsi2'
alpha-linux-ld: drivers/crypto/intel/qat/qat_common/icp_qat_hw_20_comp.h:57:(.text+0xf7c): undefined reference to `__bswapsi2'
Fixes: 5b14b2b307 ("crypto: qat - enable deflate for QAT GEN4")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202603290259.Ig9kDOmI-lkp@intel.com/
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Enabling any template selects CRYPTO_MANAGER, which causes
CRYPTO_MANAGER2 to enable itself, which selects every algorithm type
option. However, pulling in all algorithm types is needed only when the
self-tests are enabled. So condition the selections accordingly.
To make this possible, also add the missing selections to various
symbols that were relying on transitive selections via CRYPTO_MANAGER.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace scnprintf() with sysfs_emit() in the three RAS error counter
sysfs show callbacks. sysfs_emit() is the recommended API for sysfs show
functions as per Documentation/filesystems/sysfs.rst; it enforces the
PAGE_SIZE limit implicitly, removing the need to pass it explicitly.
Signed-off-by: Atharv Dubey <atharvd440@gmail.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ADF_RAS_ERR_CTR_READ() expands to atomic_read(), which returns int.
The local variable 'counter' was declared as 'unsigned long', causing
a type mismatch on the assignment. The format specifier '%ld' was
consequently wrong in two ways: wrong length modifier and wrong
signedness.
Use int to match the return type of atomic_read() and update the
format specifier to '%d' accordingly.
Fixes: 532d7f6bc4 ("crypto: qat - add error counters")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
qat_comp_alg_init_tfm() acquires a compression instance via
qat_compression_get_instance_node() before calling qat_comp_build_ctx()
to initialize the compression context. If qat_comp_build_ctx() fails, the
function returns an error without releasing the compression instance,
causing a resource leak.
When qat_comp_build_ctx() fails, release the compression instance with
qat_compression_put_instance() and clear the context to avoid leaving a
stale reference to the released instance.
The issue was introduced when build_deflate_ctx() (which always returned
void) was replaced by qat_comp_build_ctx() (which can return an error)
without adding error handling for the failure path.
Fixes: cd0e7160f8 ("crypto: qat - refactor compression template logic")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Laurent M Coquerel <laurent.m.coquerel@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Wojciech Drewek <wojciech.drewek@linux.intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The get_ae_mask() function only disables individual engines based on
the fuse register, but engines are organized in clusters of 4. If the
lead engine of a cluster is fused off, the entire cluster must be
disabled.
Replace the single bitmask inversion with explicit test_bit() checks
on the lead engine of each group, disabling the full ADF_AE_GROUP
when the lead bit is set.
Signed-off-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Fixes: fcf60f4bcf ("crypto: qat - add support for 420xx devices")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The get_ae_mask() function only disables individual engines based on
the fuse register, but engines are organized in clusters of 4. If the
lead engine of a cluster is fused off, the entire cluster must be
disabled.
Replace the single bitmask inversion with explicit test_bit() checks
on the lead engine of each group, disabling the full ADF_AE_GROUP
when the lead bit is set.
Signed-off-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Fixes: 8c8268166e ("crypto: qat - add qat_4xxx driver")
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Anti-Rollback (ARB) is a QAT GEN6 hardware feature that prevents loading
firmware with a Security Version Number (SVN) lower than an authorized
minimum. This protects against downgrade attacks by ensuring that only
firmware at or above a committed SVN can run on the acceleration device.
During firmware loading, the driver checks the SVN validation status via
a hardware CSR. If the check reports a failure, firmware authentication
is aborted. If it reports a retry status, the driver reissues the
authentication command up to a maximum number of retries.
Extend the firmware admin interface with two new messages,
ICP_QAT_FW_SVN_READ and ICP_QAT_FW_SVN_COMMIT, to query and commit the
SVN, respectively. Integrate the SVN check into the firmware
authentication path in qat_uclo.c so the driver can react to
anti-rollback status during device bring-up.
Expose SVN information to userspace via a new sysfs attribute group,
qat_svn, under the PCI device directory. The group provides read-only
attributes for the active, enforced minimum, and permanent minimum SVN
values, as well as a write-only commit attribute that allows a system
administrator to commit the currently active SVN as the new authorized
minimum.
This is based on earlier work by Ciunas Bennett.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add wireless mode support for QAT GEN6 devices.
When the WCP_WAT fuse bit is clear, the device operates in wireless
cipher mode (wcy_mode). In this mode all accelerator engines load the
wireless firmware and service configuration via 'cfg_services' sysfs
attribute is restricted to 'sym' only.
The get_accel_cap() function is extended to report wireless-specific
capabilities (ZUC, ZUC-256, 5G, extended algorithm chaining) gated by
their respective slice-disable fuse bits. The set_ssm_wdtimer() function
is updated to configure WCP (wireless cipher) and WAT (wireless
authentication) watchdog timers. The adf_gen6_cfg_dev_init() function is
updated to use adf_6xxx_is_wcy() to enforce sym-only service selection
for WCY devices during initialization.
Co-developed-by: Aviraj Cj <aviraj.cj@intel.com>
Signed-off-by: Aviraj Cj <aviraj.cj@intel.com>
Signed-off-by: George Abraham P <george.abraham.p@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
QAT GEN6 hardware requires a minimum 3 us delay during the acceleration
engine reset sequence to ensure the hardware fully settles.
Without this delay, the firmware load may fail intermittently.
Add a delay after placing the AE into reset and before clearing the reset,
matching the hardware requirements and ensuring stable firmware loading.
Earlier generations remain unaffected.
Fixes: 17fd7514ae ("crypto: qat - add qat_6xxx driver")
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Cc: stable@vger.kernel.org
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The macros in qat_hal.c were using a mixture of tabs and spaces.
Update all macro indentation to use tabs consistently, matching the
predominant style.
This does not introduce any functional change.
Signed-off-by: Suman Kumar Chakraborty <suman.kumar.chakraborty@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Return sysfs_emit() directly and drop 'ret' in cap_rem_show().
In cap_rem_store(), use 'ret' when calling set_param_u() instead of
assigning it to 'val' first, and remove 'val'.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Acked-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This converts some of the visually simpler cases that have been split
over multiple lines. I only did the ones that are easy to verify the
resulting diff by having just that final GFP_KERNEL argument on the next
line.
Somebody should probably do a proper coccinelle script for this, but for
me the trivial script actually resulted in an assertion failure in the
middle of the script. I probably had made it a bit _too_ trivial.
So after fighting that far a while I decided to just do some of the
syntactically simpler cases with variations of the previous 'sed'
scripts.
The more syntactically complex multi-line cases would mostly really want
whitespace cleanup anyway.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
Pull crypto update from Herbert Xu:
"API:
- Fix race condition in hwrng core by using RCU
Algorithms:
- Allow authenc(sha224,rfc3686) in fips mode
- Add test vectors for authenc(hmac(sha384),cbc(aes))
- Add test vectors for authenc(hmac(sha224),cbc(aes))
- Add test vectors for authenc(hmac(md5),cbc(des3_ede))
- Add lz4 support in hisi_zip
- Only allow clear key use during self-test in s390/{phmac,paes}
Drivers:
- Set rng quality to 900 in airoha
- Add gcm(aes) support for AMD/Xilinx Versal device
- Allow tfms to share device in hisilicon/trng"
* tag 'v7.0-p1' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (100 commits)
crypto: img-hash - Use unregister_ahashes in img_{un}register_algs
crypto: testmgr - Add test vectors for authenc(hmac(md5),cbc(des3_ede))
crypto: cesa - Simplify return statement in mv_cesa_dequeue_req_locked
crypto: testmgr - Add test vectors for authenc(hmac(sha224),cbc(aes))
crypto: testmgr - Add test vectors for authenc(hmac(sha384),cbc(aes))
hwrng: core - use RCU and work_struct to fix race condition
crypto: starfive - Fix memory leak in starfive_aes_aead_do_one_req()
crypto: xilinx - Fix inconsistant indentation
crypto: rng - Use unregister_rngs in register_rngs
crypto: atmel - Use unregister_{aeads,ahashes,skciphers}
hwrng: optee - simplify OP-TEE context match
crypto: ccp - Add sysfs attribute for boot integrity
dt-bindings: crypto: atmel,at91sam9g46-sha: add microchip,lan9691-sha
dt-bindings: crypto: atmel,at91sam9g46-aes: add microchip,lan9691-aes
dt-bindings: crypto: qcom,inline-crypto-engine: document the Milos ICE
crypto: caam - fix netdev memory leak in dpaa2_caam_probe
crypto: hisilicon/qm - increase wait time for mailbox
crypto: hisilicon/qm - obtain the mailbox configuration at one time
crypto: hisilicon/qm - remove unnecessary code in qm_mb_write()
crypto: hisilicon/qm - move the barrier before writing to the mailbox register
...
The restarting message from PF to VF is sent twice during AER error
handling: once from adf_error_detected() and again from
adf_disable_sriov().
This causes userspace subservices to shutdown unexpectedly when they
receive a duplicate restarting message after already being restarted.
Avoid calling adf_pf2vf_notify_restarting() and
adf_pf2vf_wait_for_restarting_complete() from adf_error_detected() so
that the restarting msg is sent only once from PF to VF.
Fixes: 9567d3dc76 ("crypto: qat - improve aer error reset handling")
Signed-off-by: Harshita Bhilwaria <harshita.bhilwaria@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Ravikumar PM <ravikumar.pm@intel.com>
Reviewed-by: Srikanth Thokala <srikanth.thokala@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Building the QAT driver with -Wmaybe-uninitialized triggers warnings in
qat_common/adf_pfvf_pf_proto.c. Specifically, the variables blk_type,
blk_byte, and byte_max may be used uninitialized in handle_blkmsg_req():
make M=drivers/crypto/intel/qat W=1 C=2 "KCFLAGS=-Werror" \
KBUILD_CFLAGS_KERNEL=-Wmaybe-uninitialized \
CFLAGS_MODULE=-Wmaybe-uninitialized
...
warning: ‘byte_max’ may be used uninitialized [-Wmaybe-uninitialized]
warning: ‘blk_type’ may be used uninitialized [-Wmaybe-uninitialized]
warning: ‘blk_byte’ may be used uninitialized [-Wmaybe-uninitialized]
Although the caller of handle_blkmsg_req() always provides a req.type
that is handled by the switch, the compiler cannot guarantee this.
Add a default case to the switch statement to handle an invalid req.type.
Fixes: 673184a2a5 ("crypto: qat - introduce support for PFVF block messages")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Register the functions required to save and restore the state of a ring
bank on the qat_420xx device. Since this logic is shared across QAT
GEN4 devices, reuse the existing GEN4 implementation.
This functionality enables saving and restoring the state of a Virtual
Function (VF), which is required for supporting VM Live Migration.
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The macro ICP_QAT_FW_COMN_FLAGS_BUILD sets flags in the firmware
descriptor to indicate:
* Whether the content descriptor is a pointer or contains embedded
data.
* Whether the source and destination buffers are scatter-gather lists
or flat buffers.
The correct parameter order is:
* First: content descriptor type
* Second: source/destination pointer type
In the asymmetric crypto code, the macro was used with the parameters
swapped. Although this does not cause functional issues, since both
macros currently evaluate to 0, it is incorrect.
Fix the parameter order in the Diffie-Hellman and RSA code paths.
Fixes: a990532023 ("crypto: qat - Add support for RSA algorithm")
Fixes: c9839143eb ("crypto: qat - Add DH support")
Reported-by: Qihua Dai <qihua.dai@intel.com> # off-list
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Pull bitmap updates from Yury Norov:
- Runtime field_{get,prep}() (Geert)
- Rust ID pool updates (Alice)
- min_t() simplification (David)
- __sw_hweightN kernel-doc fixes (Andy)
- cpumask.h headers cleanup (Andy)
* tag 'bitmap-for-6.19' of github.com:/norov/linux: (32 commits)
rust_binder: use bitmap for allocation of handles
rust: id_pool: do not immediately acquire new ids
rust: id_pool: do not supply starting capacity
rust: id_pool: rename IdPool::new() to with_capacity()
rust: bitmap: add BitmapVec::new_inline()
rust: bitmap: add MAX_LEN and MAX_INLINE_LEN constants
cpumask: Don't use "proxy" headers
soc: renesas: Use bitfield helpers
clk: renesas: Use bitfield helpers
ALSA: usb-audio: Convert to common field_{get,prep}() helpers
soc: renesas: rz-sysc: Convert to common field_get() helper
pinctrl: ma35: Convert to common field_{get,prep}() helpers
iio: mlx90614: Convert to common field_{get,prep}() helpers
iio: dac: Convert to common field_prep() helper
gpio: aspeed: Convert to common field_{get,prep}() helpers
EDAC/ie31200: Convert to common field_get() helper
crypto: qat - convert to common field_get() helper
clk: at91: Convert to common field_{get,prep}() helpers
bitfield: Add non-constant field_{prep,get}() helpers
bitfield: Add less-checking __FIELD_{GET,PREP}()
...
