On 9/25/22 17:02, Simon Glass wrote:
With gcc 12 we ge the following warning:
%s/ge/get/
In file included from tools/lib/rsa/rsa-verify.c:1: lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized 275 | db[0] &= 0xff >> leftmost_bits;
Check the value of db_len to ensure this cannot happen.
Signed-off-by: Simon Glass sjg@chromium.org
lib/rsa/rsa-verify.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/lib/rsa/rsa-verify.c b/lib/rsa/rsa-verify.c index 1d95cfbdee0..81c39251e59 100644 --- a/lib/rsa/rsa-verify.c +++ b/lib/rsa/rsa-verify.c @@ -234,6 +234,11 @@ int padding_pss_verify(struct image_sign_info *info, uint8_t leftmost_mask; struct checksum_algo *checksum = info->checksum;
- if (db_len <= 0) {
Why are length fields msg_len, hash_len, db_len signed? I cannot imagine an array with negative length. Any of the parameters msg_len and hash_len being negative would be an error.
The check here should be
if (msg_len >= hash_len)
to allow db_len moving to unsigned types.
struct padding_algo() should be corrected to use size_t.
Best regards
Heinrich
ret = -EINVAL;goto out;- }
- /* first, allocate everything */ db_mask = malloc(db_len); db = malloc(db_len);