On 25.05.20 22:43, Philipp Tomsich wrote:
On 22.05.2020, at 16:19, Heiko Stuebner heiko@sntech.de wrote:
From: Heiko Stuebner heiko.stuebner@theobroma-systems.com
When calculating rrtmp/rr rsa_gen_key_prop() tries to make (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[] with rlen being num_bits * 2
On a 4096bit key this comes down to to 257 uint32_t elements in rr and 256 elements in rrtmp but with the current allocation rr and rrtmp only have 129 uint32_t elements.
On 2048bit keys this works by chance as the defined max_rsa_size=4096 allocates a suitable number of elements, but with an actual 4096bit key this results in other memory parts getting overwritten.
so double the number of elements in rr and rrtmp so that it matches the needed number and should increase nicely if max_rsa_size gets increased in the future.
Signed-off-by: Heiko Stuebner heiko.stuebner@theobroma-systems.com
Reviewed-by: Philipp Tomsich philipp.tomsich@theobroma-systems.com
Why would we allocate memory according to max_rsa_size and not according to the actual size (*prop)->num_bits = (rsa_key.n_sz - i) * 8;
Who stops a user from using 8192 or 16384 bits? We should avoid any constant here if we do not validate that it is not exceeded.
openssl genrsa -out mykey.pem 8192 works fine for me.
Best regards
Heinrich