mirror of
https://github.com/ukui/kernel.git
synced 2026-03-09 10:07:04 -07:00
fd04899208d2057b2de808e8447cfd806fd0a607
3159 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Linus Torvalds
|
dabd40ecaf |
Merge tag 'tpmdd-next-v5.17-fixed' of git://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd
Pull TPM updates from Jarkko Sakkinen: "Other than bug fixes for TPM, this includes a patch for asymmetric keys to allow to look up and verify with self-signed certificates (keys without so called AKID - Authority Key Identifier) using a new "dn:" prefix in the query" * tag 'tpmdd-next-v5.17-fixed' of git://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd: lib: remove redundant assignment to variable ret tpm: fix NPE on probe for missing device tpm: fix potential NULL pointer access in tpm_del_char_device tpm: Add Upgrade/Reduced mode support for TPM2 modules char: tpm: cr50: Set TPM_FIRMWARE_POWER_MANAGED based on device property keys: X.509 public key issuer lookup without AKID tpm_tis: Fix an error handling path in 'tpm_tis_core_init()' tpm: tpm_tis_spi_cr50: Add default RNG quality tpm/st33zp24: drop unneeded over-commenting tpm: add request_locality before write TPM_INT_ENABLE |
||
|
Linus Torvalds
|
5c947d0dba |
Merge branch 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
Pull crypto updates from Herbert Xu: "Algorithms: - Drop alignment requirement for data in aesni - Use synchronous seeding from the /dev/random in DRBG - Reseed nopr DRBGs every 5 minutes from /dev/random - Add KDF algorithms currently used by security/DH - Fix lack of entropy on some AMD CPUs with jitter RNG Drivers: - Add support for the D1 variant in sun8i-ce - Add SEV_INIT_EX support in ccp - PFVF support for GEN4 host driver in qat - Compression support for GEN4 devices in qat - Add cn10k random number generator support" * 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (145 commits) crypto: af_alg - rewrite NULL pointer check lib/mpi: Add the return value check of kcalloc() crypto: qat - fix definition of ring reset results crypto: hisilicon - cleanup warning in qm_get_qos_value() crypto: kdf - select SHA-256 required for self-test crypto: x86/aesni - don't require alignment of data crypto: ccp - remove unneeded semicolon crypto: stm32/crc32 - Fix kernel BUG triggered in probe() crypto: s390/sha512 - Use macros instead of direct IV numbers crypto: sparc/sha - remove duplicate hash init function crypto: powerpc/sha - remove duplicate hash init function crypto: mips/sha - remove duplicate hash init function crypto: sha256 - remove duplicate generic hash init function crypto: jitter - add oversampling of noise source MAINTAINERS: update SEC2 driver maintainers list crypto: ux500 - Use platform_get_irq() to get the interrupt crypto: hisilicon/qm - disable qm clock-gating crypto: omap-aes - Fix broken pm_runtime_and_get() usage MAINTAINERS: update caam crypto driver maintainers list crypto: octeontx2 - prevent underflow in get_cores_bmap() ... |
||
 Andrew Zaborowski
|
7d30198ee2 |
keys: X.509 public key issuer lookup without AKID
There are non-root X.509 v3 certificates in use out there that contain no Authority Key Identifier extension (RFC5280 section 4.2.1.1). For trust verification purposes the kernel asymmetric key type keeps two struct asymmetric_key_id instances that the key can be looked up by, and another two to look up the key's issuer. The x509 public key type and the PKCS7 type generate them from the SKID and AKID extensions in the certificate. In effect current code has no way to look up the issuer certificate for verification without the AKID. To remedy this, add a third asymmetric_key_id blob to the arrays in both asymmetric_key_id's (for certficate subject) and in the public_keys_signature's auth_ids (for issuer lookup), using just raw subject and issuer DNs from the certificate. Adapt asymmetric_key_ids() and its callers to use the third ID for lookups when none of the other two are available. Attempt to keep the logic intact when they are, to minimise behaviour changes. Adapt the restrict functions' NULL-checks to include that ID too. Do not modify the lookup logic in pkcs7_verify.c, the AKID extensions are still required there. Internally use a new "dn:" prefix to the search specifier string generated for the key lookup in find_asymmetric_key(). This tells asymmetric_key_match_preparse to only match the data against the raw DN in the third ID and shouldn't conflict with search specifiers already in use. In effect implement what (2) in the struct asymmetric_key_id comment (include/keys/asymmetric-type.h) is probably talking about already, so do not modify that comment. It is also how "openssl verify" looks up issuer certificates without the AKID available. Lookups by the raw DN are unambiguous only provided that the CAs respect the condition in RFC5280 4.2.1.1 that the AKID may only be omitted if the CA uses a single signing key. The following is an example of two things that this change enables. A self-signed ceritficate is generated following the example from https://letsencrypt.org/docs/certificates-for-localhost/, and can be looked up by an identifier and verified against itself by linking to a restricted keyring -- both things not possible before due to the missing AKID extension: $ openssl req -x509 -out localhost.crt -outform DER -keyout localhost.key \ -newkey rsa:2048 -nodes -sha256 \ -subj '/CN=localhost' -extensions EXT -config <( \ echo -e "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\n" \ "subjectAltName=DNS:localhost\nkeyUsage=digitalSignature\n" \ "extendedKeyUsage=serverAuth") $ keyring=`keyctl newring test @u` $ trusted=`keyctl padd asymmetric trusted $keyring < localhost.crt`; \ echo $trusted 39726322 $ keyctl search $keyring asymmetric dn:3112301006035504030c096c6f63616c686f7374 39726322 $ keyctl restrict_keyring $keyring asymmetric key_or_keyring:$trusted $ keyctl padd asymmetric verified $keyring < localhost.crt Signed-off-by: Andrew Zaborowski <andrew.zaborowski@intel.com> Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org> Acked-by: Jarkko Sakkinen <jarkko@kernel.org> Acked-by: David Howells <dhowells@redhat.com> Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org> |
||
 Jiasheng Jiang
|
5f21d7d283 |
crypto: af_alg - rewrite NULL pointer check
Because of the possible alloc failure of the alloc_page(), it could return NULL pointer. And there is a check below the sg_assign_page(). But it will be more logical to move the NULL check before the sg_assign_page(). Signed-off-by: Jiasheng Jiang <jiasheng@iscas.ac.cn> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
 Jason A. Donenfeld
|
6048fdcc5f |
lib/crypto: blake2s: include as built-in
In preparation for using blake2s in the RNG, we change the way that it is wired-in to the build system. Instead of using ifdefs to select the right symbol, we use weak symbols. And because ARM doesn't need the generic implementation, we make the generic one default only if an arch library doesn't need it already, and then have arch libraries that do need it opt-in. So that the arch libraries can remain tristate rather than bool, we then split the shash part from the glue code. Acked-by: Herbert Xu <herbert@gondor.apana.org.au> Acked-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: linux-kbuild@vger.kernel.org Cc: linux-crypto@vger.kernel.org Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com> |
||
 Stephan Müller
|
304b4acee2 |
crypto: kdf - select SHA-256 required for self-test
The self test of the KDF is based on SHA-256. Thus, this algorithm must be present as otherwise a warning is issued. Reported-by: kernel test robot <oliver.sang@intel.com> Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
 Tianjia Zhang
|
96ede30f4b |
crypto: sha256 - remove duplicate generic hash init function
crypto_sha256_init() and sha256_base_init() are the same repeated implementations, remove the crypto_sha256_init() in generic implementation, sha224 is the same process. Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Stephan Müller
|
908dffaf88 |
crypto: jitter - add oversampling of noise source
The output n bits can receive more than n bits of min entropy, of course, but the fixed output of the conditioning function can only asymptotically approach the output size bits of min entropy, not attain that bound. Random maps will tend to have output collisions, which reduces the creditable output entropy (that is what SP 800-90B Section 3.1.5.1.2 attempts to bound). The value "64" is justified in Appendix A.4 of the current 90C draft, and aligns with NIST's in "epsilon" definition in this document, which is that a string can be considered "full entropy" if you can bound the min entropy in each bit of output to at least 1-epsilon, where epsilon is required to be <= 2^(-32). Note, this patch causes the Jitter RNG to cut its performance in half in FIPS mode because the conditioning function of the LFSR produces 64 bits of entropy in one block. The oversampling requires that additionally 64 bits of entropy are sampled from the noise source. If the conditioner is changed, such as using SHA-256, the impact of the oversampling is only one fourth, because for the 256 bit block of the conditioner, only 64 additional bits from the noise source must be sampled. This patch is derived from the user space jitterentropy-library. Signed-off-by: Stephan Mueller <smueller@chronox.de> Reviewed-by: Simo Sorce <simo@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
 Nicolai Stange
|
710ce4b88f |
crypto: jitter - quit sample collection loop upon RCT failure
The jitterentropy collection loop in jent_gen_entropy() can in principle run indefinitely without making any progress if it only receives stuck measurements as determined by jent_stuck(). After 31 consecutive stuck samples, the Repetition Count Test (RCT) would fail anyway and the jitterentropy RNG instances moved into ->health_failure == 1 state. jent_gen_entropy()'s caller, jent_read_entropy() would then check for this ->health_failure condition and return an error if found set. It follows that there's absolutely no point in continuing the collection loop in jent_gen_entropy() once the RCT has failed. Make the jitterentropy collection loop more robust by terminating it upon jent_health_failure() so that it won't continue to run indefinitely without making any progress. Signed-off-by: Nicolai Stange <nstange@suse.de> Reviewed-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Nicolai Stange
|
b454fb7025 |
crypto: jitter - don't limit ->health_failure check to FIPS mode
The jitterentropy's Repetition Count Test (RCT) as well as the Adaptive Proportion Test (APT) are run unconditionally on any collected samples. However, their result, i.e. ->health_failure, will only get checked if fips_enabled is set, c.f. the jent_health_failure() wrapper. I would argue that a RCT or APT failure indicates that something's seriously off and that this should always be reported as an error, independently of whether FIPS mode is enabled or not: it should be up to callers whether or not and how to handle jitterentropy failures. Make jent_health_failure() to unconditionally return ->health_failure, independent of whether fips_enabled is set. Note that fips_enabled isn't accessed from the jitterentropy code anymore now. Remove the linux/fips.h include as well as the jent_fips_enabled() wrapper. Signed-off-by: Nicolai Stange <nstange@suse.de> Reviewed-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Nicolai Stange
|
8f79772843 |
crypto: drbg - ignore jitterentropy errors if not in FIPS mode
A subsequent patch will make the jitterentropy RNG to unconditionally report health test errors back to callers, independent of whether fips_enabled is set or not. The DRBG needs access to a functional jitterentropy instance only in FIPS mode (because it's the only SP800-90B compliant entropy source as it currently stands). Thus, it is perfectly fine for the DRBGs to obtain entropy from the jitterentropy source only on a best effort basis if fips_enabled is off. Make the DRBGs to ignore jitterentropy failures if fips_enabled is not set. Signed-off-by: Nicolai Stange <nstange@suse.de> Reviewed-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
 Guo Zhengkui
|
3219c2b1bd |
crypto: dh - remove duplicate includes
Remove a duplicate #include <linux/fips.h>. Signed-off-by: Guo Zhengkui <guozhengkui@vivo.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Stephan Müller
|
330507fbc9 |
crypto: des - disallow des3 in FIPS mode
On Dec 31 2023 NIST sunsets TDES for FIPS use. To prevent FIPS validations to be completed in the future to be affected by the TDES sunsetting, disallow TDES already now. Otherwise a FIPS validation would need to be "touched again" end 2023 to handle TDES accordingly. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Stephan Müller
|
1e146c393b |
crypto: dh - limit key size to 2048 in FIPS mode
FIPS disallows DH with keys < 2048 bits. Thus, the kernel should consider the enforcement of this limit. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Stephan Müller
|
1ce1bacc48 |
crypto: rsa - limit key size to 2048 in FIPS mode
FIPS disallows RSA with keys < 2048 bits. Thus, the kernel should consider the enforcement of this limit. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Stephan Müller
|
552d03a223 |
crypto: jitter - consider 32 LSB for APT
The APT compares the current time stamp with a pre-set value. The current code only considered the 4 LSB only. Yet, after reviews by mathematicians of the user space Jitter RNG version >= 3.1.0, it was concluded that the APT can be calculated on the 32 LSB of the time delta. Thi change is applied to the kernel. This fixes a bug where an AMD EPYC fails this test as its RDTSC value contains zeros in the LSB. The most appropriate fix would have been to apply a GCD calculation and divide the time stamp by the GCD. Yet, this is a significant code change that will be considered for a future update. Note, tests showed that constantly the GCD always was 32 on these systems, i.e. the 5 LSB were always zero (thus failing the APT since it only considered the 4 LSB for its calculation). Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Stephan Müller
|
026a733e66 |
crypto: kdf - add SP800-108 counter key derivation function
SP800-108 defines three KDFs - this patch provides the counter KDF implementation. The KDF is implemented as a service function where the caller has to maintain the hash / HMAC state. Apart from this hash/HMAC state, no additional state is required to be maintained by either the caller or the KDF implementation. The key for the KDF is set with the crypto_kdf108_setkey function which is intended to be invoked before the caller requests a key derivation operation via crypto_kdf108_ctr_generate. SP800-108 allows the use of either a HMAC or a hash as crypto primitive for the KDF. When a HMAC primtive is intended to be used, crypto_kdf108_setkey must be used to set the HMAC key. Otherwise, for a hash crypto primitve crypto_kdf108_ctr_generate can be used immediately after allocating the hash handle. Signed-off-by: Stephan Mueller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Nicolai Stange
|
8ea5ee00be |
crypto: drbg - reseed 'nopr' drbgs periodically from get_random_bytes()
In contrast to the fully prediction resistant 'pr' DRBGs, the 'nopr'
variants get seeded once at boot and reseeded only rarely thereafter,
namely only after 2^20 requests have been served each. AFAICT, this
reseeding based on the number of requests served is primarily motivated
by information theoretic considerations, c.f. NIST SP800-90Ar1,
sec. 8.6.8 ("Reseeding").
However, given the relatively large seed lifetime of 2^20 requests, the
'nopr' DRBGs can hardly be considered to provide any prediction resistance
whatsoever, i.e. to protect against threats like side channel leaks of the
internal DRBG state (think e.g. leaked VM snapshots). This is expected and
completely in line with the 'nopr' naming, but as e.g. the
"drbg_nopr_hmac_sha512" implementation is potentially being used for
providing the "stdrng" and thus, the crypto_default_rng serving the
in-kernel crypto, it would certainly be desirable to achieve at least the
same level of prediction resistance as get_random_bytes() does.
Note that the chacha20 rngs underlying get_random_bytes() get reseeded
every CRNG_RESEED_INTERVAL == 5min: the secondary, per-NUMA node rngs from
the primary one and the primary rng in turn from the entropy pool, provided
sufficient entropy is available.
The 'nopr' DRBGs do draw randomness from get_random_bytes() for their
initial seed already, so making them to reseed themselves periodically from
get_random_bytes() in order to let them benefit from the latter's
prediction resistance is not such a big change conceptually.
In principle, it would have been also possible to make the 'nopr' DRBGs to
periodically invoke a full reseeding operation, i.e. to also consider the
jitterentropy source (if enabled) in addition to get_random_bytes() for the
seed value. However, get_random_bytes() is relatively lightweight as
compared to the jitterentropy generation process and thus, even though the
'nopr' reseeding is supposed to get invoked infrequently, it's IMO still
worthwhile to avoid occasional latency spikes for drbg_generate() and
stick to get_random_bytes() only. As an additional remark, note that
drawing randomness from the non-SP800-90B-conforming get_random_bytes()
only won't adversely affect SP800-90A conformance either: the very same is
being done during boot via drbg_seed_from_random() already once
rng_is_initialized() flips to true and it follows that if the DRBG
implementation does conform to SP800-90A now, it will continue to do so.
Make the 'nopr' DRBGs to reseed themselves periodically from
get_random_bytes() every CRNG_RESEED_INTERVAL == 5min.
More specifically, introduce a new member ->last_seed_time to struct
drbg_state for recording in units of jiffies when the last seeding
operation had taken place. Make __drbg_seed() maintain it and let
drbg_generate() invoke a reseed from get_random_bytes() via
drbg_seed_from_random() if more than 5min have passed by since the last
seeding operation. Be careful to not to reseed if in testing mode though,
or otherwise the drbg related tests in crypto/testmgr.c would fail to
reproduce the expected output.
In order to keep the formatting clean in drbg_generate() wrap the logic
for deciding whether or not a reseed is due in a new helper,
drbg_nopr_reseed_interval_elapsed().
Signed-off-by: Nicolai Stange <nstange@suse.de>
Reviewed-by: Stephan Müller <smueller@chronox.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
||
|
Nicolai Stange
|
559edd47cc |
crypto: drbg - make drbg_prepare_hrng() handle jent instantiation errors
Now that drbg_prepare_hrng() doesn't do anything but to instantiate a jitterentropy crypto_rng instance, it looks a little odd to have the related error handling at its only caller, drbg_instantiate(). Move the handling of jitterentropy allocation failures from drbg_instantiate() close to the allocation itself in drbg_prepare_hrng(). There is no change in behaviour. Signed-off-by: Nicolai Stange <nstange@suse.de> Reviewed-by: Stephan Müller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Nicolai Stange
|
074bcd4000 |
crypto: drbg - make reseeding from get_random_bytes() synchronous
get_random_bytes() usually hasn't full entropy available by the time DRBG instances are first getting seeded from it during boot. Thus, the DRBG implementation registers random_ready_callbacks which would in turn schedule some work for reseeding the DRBGs once get_random_bytes() has sufficient entropy available. For reference, the relevant history around handling DRBG (re)seeding in the context of a not yet fully seeded get_random_bytes() is: commit |
||
|
Nicolai Stange
|
262d83a429 |
crypto: drbg - move dynamic ->reseed_threshold adjustments to __drbg_seed()
Since commit
|
||
|
Nicolai Stange
|
2bcd254438 |
crypto: drbg - track whether DRBG was seeded with !rng_is_initialized()
Currently, the DRBG implementation schedules asynchronous works from random_ready_callbacks for reseeding the DRBG instances with output from get_random_bytes() once the latter has sufficient entropy available. However, as the get_random_bytes() initialization state can get queried by means of rng_is_initialized() now, there is no real need for this asynchronous reseeding logic anymore and it's better to keep things simple by doing it synchronously when needed instead, i.e. from drbg_generate() once rng_is_initialized() has flipped to true. Of course, for this to work, drbg_generate() would need some means by which it can tell whether or not rng_is_initialized() has flipped to true since the last seeding from get_random_bytes(). Or equivalently, whether or not the last seed from get_random_bytes() has happened when rng_is_initialized() was still evaluating to false. As it currently stands, enum drbg_seed_state allows for the representation of two different DRBG seeding states: DRBG_SEED_STATE_UNSEEDED and DRBG_SEED_STATE_FULL. The former makes drbg_generate() to invoke a full reseeding operation involving both, the rather expensive jitterentropy as well as the get_random_bytes() randomness sources. The DRBG_SEED_STATE_FULL state on the other hand implies that no reseeding at all is required for a !->pr DRBG variant. Introduce the new DRBG_SEED_STATE_PARTIAL state to enum drbg_seed_state for representing the condition that a DRBG was being seeded when rng_is_initialized() had still been false. In particular, this new state implies that - the given DRBG instance has been fully seeded from the jitterentropy source (if enabled) - and drbg_generate() is supposed to reseed from get_random_bytes() *only* once rng_is_initialized() turns to true. Up to now, the __drbg_seed() helper used to set the given DRBG instance's ->seeded state to constant DRBG_SEED_STATE_FULL. Introduce a new argument allowing for the specification of the to be written ->seeded value instead. Make the first of its two callers, drbg_seed(), determine the appropriate value based on rng_is_initialized(). The remaining caller, drbg_async_seed(), is known to get invoked only once rng_is_initialized() is true, hence let it pass constant DRBG_SEED_STATE_FULL for the new argument to __drbg_seed(). There is no change in behaviour, except for that the pr_devel() in drbg_generate() would now report "unseeded" for ->pr DRBG instances which had last been seeded when rng_is_initialized() was still evaluating to false. Signed-off-by: Nicolai Stange <nstange@suse.de> Reviewed-by: Stephan Müller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Nicolai Stange
|
ce8ce31b2c |
crypto: drbg - prepare for more fine-grained tracking of seeding state
There are two different randomness sources the DRBGs are getting seeded from, namely the jitterentropy source (if enabled) and get_random_bytes(). At initial DRBG seeding time during boot, the latter might not have collected sufficient entropy for seeding itself yet and thus, the DRBG implementation schedules a reseed work from a random_ready_callback once that has happened. This is particularly important for the !->pr DRBG instances, for which (almost) no further reseeds are getting triggered during their lifetime. Because collecting data from the jitterentropy source is a rather expensive operation, the aforementioned asynchronously scheduled reseed work restricts itself to get_random_bytes() only. That is, it in some sense amends the initial DRBG seed derived from jitterentropy output at full (estimated) entropy with fresh randomness obtained from get_random_bytes() once that has been seeded with sufficient entropy itself. With the advent of rng_is_initialized(), there is no real need for doing the reseed operation from an asynchronously scheduled work anymore and a subsequent patch will make it synchronous by moving it next to related logic already present in drbg_generate(). However, for tracking whether a full reseed including the jitterentropy source is required or a "partial" reseed involving only get_random_bytes() would be sufficient already, the boolean struct drbg_state's ->seeded member must become a tristate value. Prepare for this by introducing the new enum drbg_seed_state and change struct drbg_state's ->seeded member's type from bool to that type. For facilitating review, enum drbg_seed_state is made to only contain two members corresponding to the former ->seeded values of false and true resp. at this point: DRBG_SEED_STATE_UNSEEDED and DRBG_SEED_STATE_FULL. A third one for tracking the intermediate state of "seeded from jitterentropy only" will be introduced with a subsequent patch. There is no change in behaviour at this point. Signed-off-by: Nicolai Stange <nstange@suse.de> Reviewed-by: Stephan Müller <smueller@chronox.de> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
 Lei He
|
a9887010ed |
crypto: testmgr - Fix wrong test case of RSA
According to the BER encoding rules, integer value should be encoded as two's complement, and if the highest bit of a positive integer is 1, should add a leading zero-octet. The kernel's built-in RSA algorithm cannot recognize negative numbers when parsing keys, so it can pass this test case. Export the key to file and run the following command to verify the fix result: openssl asn1parse -inform DER -in /path/to/key/file Signed-off-by: Lei He <helei.sig11@bytedance.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> |
||
|
Linus Torvalds
|
c8c109546a |
Merge tag 'zstd-for-linus-v5.16' of git://github.com/terrelln/linux
Pull zstd update from Nick Terrell:
"Update to zstd-1.4.10.
Add myself as the maintainer of zstd and update the zstd version in
the kernel, which is now 4 years out of date, to a much more recent
zstd release. This includes bug fixes, much more extensive fuzzing,
and performance improvements. And generates the kernel zstd
automatically from upstream zstd, so it is easier to keep the zstd
verison up to date, and we don't fall so far out of date again.
This includes 5 commits that update the zstd library version:
- Adds a new kernel-style wrapper around zstd.
This wrapper API is functionally equivalent to the subset of the
current zstd API that is currently used. The wrapper API changes to
be kernel style so that the symbols don't collide with zstd's
symbols. The update to zstd-1.4.10 maintains the same API and
preserves the semantics, so that none of the callers need to be
updated. All callers are updated in the commit, because there are
zero functional changes.
- Adds an indirection for `lib/decompress_unzstd.c` so it doesn't
depend on the layout of `lib/zstd/` to include every source file.
This allows the next patch to be automatically generated.
- Imports the zstd-1.4.10 source code. This commit is automatically
generated from upstream zstd (https://github.com/facebook/zstd).
- Adds me (terrelln@fb.com) as the maintainer of `lib/zstd`.
- Fixes a newly added build warning for clang.
The discussion around this patchset has been pretty long, so I've
included a FAQ-style summary of the history of the patchset, and why
we are taking this approach.
Why do we need to update?
-------------------------
The zstd version in the kernel is based off of zstd-1.3.1, which is
was released August 20, 2017. Since then zstd has seen many bug fixes
and performance improvements. And, importantly, upstream zstd is
continuously fuzzed by OSS-Fuzz, and bug fixes aren't backported to
older versions. So the only way to sanely get these fixes is to keep
up to date with upstream zstd.
There are no known security issues that affect the kernel, but we need
to be able to update in case there are. And while there are no known
security issues, there are relevant bug fixes. For example the problem
with large kernel decompression has been fixed upstream for over 2
years [1]
Additionally the performance improvements for kernel use cases are
significant. Measured for x86_64 on my Intel i9-9900k @ 3.6 GHz:
- BtrFS zstd compression at levels 1 and 3 is 5% faster
- BtrFS zstd decompression+read is 15% faster
- SquashFS zstd decompression+read is 15% faster
- F2FS zstd compression+write at level 3 is 8% faster
- F2FS zstd decompression+read is 20% faster
- ZRAM decompression+read is 30% faster
- Kernel zstd decompression is 35% faster
- Initramfs zstd decompression+build is 5% faster
On top of this, there are significant performance improvements coming
down the line in the next zstd release, and the new automated update
patch generation will allow us to pull them easily.
How is the update patch generated?
----------------------------------
The first two patches are preparation for updating the zstd version.
Then the 3rd patch in the series imports upstream zstd into the
kernel. This patch is automatically generated from upstream. A script
makes the necessary changes and imports it into the kernel. The
changes are:
- Replace all libc dependencies with kernel replacements and rewrite
includes.
- Remove unncessary portability macros like: #if defined(_MSC_VER).
- Use the kernel xxhash instead of bundling it.
This automation gets tested every commit by upstream's continuous
integration. When we cut a new zstd release, we will submit a patch to
the kernel to update the zstd version in the kernel.
The automated process makes it easy to keep the kernel version of zstd
up to date. The current zstd in the kernel shares the guts of the
code, but has a lot of API and minor changes to work in the kernel.
This is because at the time upstream zstd was not ready to be used in
the kernel envrionment as-is. But, since then upstream zstd has
evolved to support being used in the kernel as-is.
Why are we updating in one big patch?
-------------------------------------
The 3rd patch in the series is very large. This is because it is
restructuring the code, so it both deletes the existing zstd, and
re-adds the new structure. Future updates will be directly
proportional to the changes in upstream zstd since the last import.
They will admittidly be large, as zstd is an actively developed
project, and has hundreds of commits between every release. However,
there is no other great alternative.
One option ruled out is to replay every upstream zstd commit. This is
not feasible for several reasons:
- There are over 3500 upstream commits since the zstd version in the
kernel.
- The automation to automatically generate the kernel update was only
added recently, so older commits cannot easily be imported.
- Not every upstream zstd commit builds.
- Only zstd releases are "supported", and individual commits may have
bugs that were fixed before a release.
Another option to reduce the patch size would be to first reorganize
to the new file structure, and then apply the patch. However, the
current kernel zstd is formatted with clang-format to be more
"kernel-like". But, the new method imports zstd as-is, without
additional formatting, to allow for closer correlation with upstream,
and easier debugging. So the patch wouldn't be any smaller.
It also doesn't make sense to import upstream zstd commit by commit
going forward. Upstream zstd doesn't support production use cases
running of the development branch. We have a lot of post-commit
fuzzing that catches many bugs, so indiviudal commits may be buggy,
but fixed before a release. So going forward, I intend to import every
(important) zstd release into the Kernel.
So, while it isn't ideal, updating in one big patch is the only patch
I see forward.
Who is responsible for this code?
---------------------------------
I am. This patchset adds me as the maintainer for zstd. Previously,
there was no tree for zstd patches. Because of that, there were
several patches that either got ignored, or took a long time to merge,
since it wasn't clear which tree should pick them up. I'm officially
stepping up as maintainer, and setting up my tree as the path through
which zstd patches get merged. I'll make sure that patches to the
kernel zstd get ported upstream, so they aren't erased when the next
version update happens.
How is this code tested?
------------------------
I tested every caller of zstd on x86_64 (BtrFS, ZRAM, SquashFS, F2FS,
Kernel, InitRAMFS). I also tested Kernel & InitRAMFS on i386 and
aarch64. I checked both performance and correctness.
Also, thanks to many people in the community who have tested these
patches locally.
Lastly, this code will bake in linux-next before being merged into
v5.16.
Why update to zstd-1.4.10 when zstd-1.5.0 has been released?
------------------------------------------------------------
This patchset has been outstanding since 2020, and zstd-1.4.10 was the
latest release when it was created. Since the update patch is
automatically generated from upstream, I could generate it from
zstd-1.5.0.
However, there were some large stack usage regressions in zstd-1.5.0,
and are only fixed in the latest development branch. And the latest
development branch contains some new code that needs to bake in the
fuzzer before I would feel comfortable releasing to the kernel.
Once this patchset has been merged, and we've released zstd-1.5.1, we
can update the kernel to zstd-1.5.1, and exercise the update process.
You may notice that zstd-1.4.10 doesn't exist upstream. This release
is an artifical release based off of zstd-1.4.9, with some fixes for
the kernel backported from the development branch. I will tag the
zstd-1.4.10 release after this patchset is merged, so the Linux Kernel
is running a known version of zstd that can be debugged upstream.
Why was a wrapper API added?
----------------------------
The first versions of this patchset migrated the kernel to the
upstream zstd API. It first added a shim API that supported the new
upstream API with the old code, then updated callers to use the new
shim API, then transitioned to the new code and deleted the shim API.
However, Cristoph Hellwig suggested that we transition to a kernel
style API, and hide zstd's upstream API behind that. This is because
zstd's upstream API is supports many other use cases, and does not
follow the kernel style guide, while the kernel API is focused on the
kernel's use cases, and follows the kernel style guide.
Where is the previous discussion?
---------------------------------
Links for the discussions of the previous versions of the patch set
below. The largest changes in the design of the patchset are driven by
the discussions in v11, v5, and v1. Sorry for the mix of links, I
couldn't find most of the the threads on lkml.org"
Link: https://lkml.org/lkml/2020/9/29/27 [1]
Link: https://www.spinics.net/lists/linux-crypto/msg58189.html [v12]
Link: https://lore.kernel.org/linux-btrfs/20210430013157.747152-1-nickrterrell@gmail.com/ [v11]
Link: https://lore.kernel.org/lkml/20210426234621.870684-2-nickrterrell@gmail.com/ [v10]
Link: https://lore.kernel.org/linux-btrfs/20210330225112.496213-1-nickrterrell@gmail.com/ [v9]
Link: https://lore.kernel.org/linux-f2fs-devel/20210326191859.1542272-1-nickrterrell@gmail.com/ [v8]
Link: https://lkml.org/lkml/2020/12/3/1195 [v7]
Link: https://lkml.org/lkml/2020/12/2/1245 [v6]
Link: https://lore.kernel.org/linux-btrfs/20200916034307.2092020-1-nickrterrell@gmail.com/ [v5]
Link: https://www.spinics.net/lists/linux-btrfs/msg105783.html [v4]
Link: https://lkml.org/lkml/2020/9/23/1074 [v3]
Link: https://www.spinics.net/lists/linux-btrfs/msg105505.html [v2]
Link: https://lore.kernel.org/linux-btrfs/20200916034307.2092020-1-nickrterrell@gmail.com/ [v1]
Signed-off-by: Nick Terrell <terrelln@fb.com>
Tested By: Paul Jones <paul@pauljones.id.au>
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM/Clang v13.0.0 on x86-64
Tested-by: Jean-Denis Girard <jd.girard@sysnux.pf>
* tag 'zstd-for-linus-v5.16' of git://github.com/terrelln/linux:
lib: zstd: Add cast to silence clang's -Wbitwise-instead-of-logical
MAINTAINERS: Add maintainer entry for zstd
lib: zstd: Upgrade to latest upstream zstd version 1.4.10
lib: zstd: Add decompress_sources.h for decompress_unzstd
lib: zstd: Add kernel-specific API
|