Enabling any template selects CRYPTO_MANAGER, which causes
CRYPTO_MANAGER2 to enable itself, which selects every algorithm type
option. However, pulling in all algorithm types is needed only when the
self-tests are enabled. So condition the selections accordingly.
To make this possible, also add the missing selections to various
symbols that were relying on transitive selections via CRYPTO_MANAGER.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
-Wflex-array-member-not-at-end was introduced in GCC-14, and we are
getting ready to enable it, globally.
Move the conflicting declaration to the end of the corresponding
structure. Notice that `struct ahash_request` is a flexible structure,
this is a structure that contains a flexible-array member.
With these changes fix the following warning:
drivers/crypto/allwinner/sun8i-ss/sun8i-ss.h:251:30: warning: structure containing a flexible array member is not at the end of another structure [-Wflex-array-member-not-at-end]
Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
With gcc-14 I get
../drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c: In function ‘sun8i_ss_hash_run’:
../drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c:631:21: warning: ‘j’ may be used uninitialized [-Wmaybe-uninitialized]
631 | j = hash_pad(bf, 4096, j, byte_count, true, bs);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c:493:13: note: ‘j’ was declared here
493 | int j, i, k, todo;
| ^
Fix this false positive by moving the initialisation of j earlier.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In order to make the ahash code more clear and modular, split the
monolithic sun8i_ce_hash_run() callback into two parts, prepare and
unprepare (therefore aligning it with the sun8i-ce skcipher code).
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Similar to sun8i-ce skcipher code, move all request-specific data to
request context. This simplifies sun8i_ce_hash_run() and it eliminates
the remaining kmalloc() calls from the digest path.
Since the 'result' buffer in the request ctx struct is used for from-device
DMA, it needs to be properly aligned to CRYPTO_DMA_ALIGN. Therefore:
- increase reqsize by CRYPTO_DMA_PADDING
- add __aligned(CRYPTO_DMA_ALIGN) attribute for the 'result' buffer
- convert all ahash_request_ctx_dma() calls to ahash_request_ctx_dma()
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
To remove some duplicated code, directly pass 'struct skcipher_request' and
'struct ce_task' pointers to sun8i_ce_cipher_{prepare,unprepare}.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fold sun8i_ce_cipher_run() into it's only caller, sun8i_ce_cipher_do_one(),
to eliminate a bit of boilerplate.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently, the iv buffers are allocated per flow during driver probe,
which means that the buffers are shared by all requests. This works
because the driver is not yet truly asynchronous, since it waits inside
do_one_request() for the completion irq.
However, the iv data is request-specific, so it should be part of the
request context. Move iv buffers to request context.
The bounce_iv buffer is aligned to sizeof(u32) to match the 'word address'
requirement for the task descriptor's iv field.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In order to remove code duplication, factor out task descriptor dumping to
a new function sun8i_ce_dump_task_descriptors().
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There are 3 instances of '__maybe_unused' annotations that are not needed,
as the variables are always used, so remove them.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Using the number of bytes in the request as DMA timeout is really
inconsistent, as large requests could possibly set a timeout of
hundreds of seconds.
Remove the per-channel timeout field and use a single, static DMA
timeout of 3 seconds for all requests.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Tested-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Reviewed-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use crypto_ahash_blocksize() and crypto_ahash_digestsize() helpers instead
of directly accessing 'struct ahash_alg' members.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove `ivlen` field of `sun8i_cipher_req_ctx`, as it is not really useful.
The iv length returned by crypto_skcipher_ivsize() is already available
everywhere and can be used instead.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In sun8i_ce_cipher_unprepare(), dma_unmap_sg() is incorrectly called with
the number of entries returned by dma_map_sg(), rather than using the
original number of entries passed when mapping the scatterlist.
To fix this, stash the original number of entries passed to dma_map_sg()
in the request context.
Fixes: 0605fa0f78 ("crypto: sun8i-ce - split into prepare/run/unprepare")
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Acked-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Tested-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
'struct ahash_request' has a flexible array at the end, so it must be the
last member in a struct, to avoid overwriting other struct members.
Therefore, move 'fallback_req' to the end of the 'sun8i_ce_hash_reqctx'
struct.
Fixes: 56f6d5aee8 ("crypto: sun8i-ce - support hash algorithms")
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace pm_runtime_get_sync() usage with pm_runtime_resume_and_get() to
simplify error handling.
This is recommended in the documentation of pm_runtime_get_sync().
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The pm_runtime_use_autosuspend() call must be undone with
pm_runtime_dont_use_autosuspend() at driver exit, but this is not
currently handled in the driver.
To fix this issue and at the same time simplify error handling, switch
to devm_pm_runtime_enable(). It will call both pm_runtime_disable() and
pm_runtime_dont_use_autosuspend() during driver removal.
Fixes: 06f751b613 ("crypto: allwinner - Add sun8i-ce Crypto Engine")
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When testing sun8i-ss with multi_v7_defconfig, all CBC algorithm fail crypto
selftests.
This is strange since on sunxi_defconfig, everything was ok.
The problem was in the IV setup loop which never run because sg_dma_len
was 0.
Fixes: 359e893e8a ("crypto: sun8i-ss - rework handling of IV")
Signed-off-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace pm_runtime_get_sync() usage with pm_runtime_resume_and_get() to
simplify error handling.
This is recommended in the documentation of pm_runtime_get_sync():
/**
* pm_runtime_get_sync - Bump up usage counter of a device and resume it.
...
* Consider using pm_runtime_resume_and_get() instead of it, especially
* if its return value is checked by the caller, as this is likely to result
* in cleaner code.
...
*/
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Tested-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Acked-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add IS_ENABLED(CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG) checks before the
fallback counter updates to make sure the code is not included when
debugfs statistics support is not enabled.
Also, drop the existing ifdef guards, since 'struct sun8i_ce_alg_template'
is always defined, even with CONFIG_CRYPTO_DEV_SUN8I_CE_DEBUG disabled.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Tested-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Acked-by: Corentin LABBE <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fix two DMA cleanup issues on the error path in sun8i_ce_cipher_prepare():
1] If dma_map_sg() fails for areq->dst, the device driver would try to free
DMA memory it has not allocated in the first place. To fix this, on the
"theend_sgs" error path, call dma unmap only if the corresponding dma
map was successful.
2] If the dma_map_single() call for the IV fails, the device driver would
try to free an invalid DMA memory address on the "theend_iv" path:
------------[ cut here ]------------
DMA-API: sun8i-ce 1904000.crypto: device driver tries to free an invalid DMA memory address
WARNING: CPU: 2 PID: 69 at kernel/dma/debug.c:968 check_unmap+0x123c/0x1b90
Modules linked in: skcipher_example(O+)
CPU: 2 UID: 0 PID: 69 Comm: 1904000.crypto- Tainted: G O 6.15.0-rc3+ #24 PREEMPT
Tainted: [O]=OOT_MODULE
Hardware name: OrangePi Zero2 (DT)
pc : check_unmap+0x123c/0x1b90
lr : check_unmap+0x123c/0x1b90
...
Call trace:
check_unmap+0x123c/0x1b90 (P)
debug_dma_unmap_page+0xac/0xc0
dma_unmap_page_attrs+0x1f4/0x5fc
sun8i_ce_cipher_do_one+0x1bd4/0x1f40
crypto_pump_work+0x334/0x6e0
kthread_worker_fn+0x21c/0x438
kthread+0x374/0x664
ret_from_fork+0x10/0x20
---[ end trace 0000000000000000 ]---
To fix this, check for !dma_mapping_error() before calling
dma_unmap_single() on the "theend_iv" path.
Fixes: 06f751b613 ("crypto: allwinner - Add sun8i-ce Crypto Engine")
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rather than setting up the fallback request by hand, use
ahash_request_set_callback() and ahash_request_set_crypt() API helpers
to properly setup the new request.
This also ensures that the completion callback is properly passed down
to the fallback algorithm, which avoids a crash with async fallbacks.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Tested-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Acked-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Rather than setting up the fallback request by hand, use
ahash_request_set_callback() and ahash_request_set_crypt() API helpers
to properly setup the new request.
Signed-off-by: Ovidiu Panait <ovidiu.panait.oss@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
