[PATCH 0/2] spl, rk3399: fix FIT image loadingg out-of-range
The patches in this series address an issue I met when trying to enable FIT signature verification by SPL on a RockPi4B board.
- The first patch avoids a buffer overflow when writing to INTMEM1 SRAM - The second one addresses reliability issues I had with back-to-back MMC reads from the microSD card. By allowing DMA operation, the issue is gone and generally speaking DMA is preferred over FIFO mode anyways.
Jerome Forissier (2): spl: fit: add config option for temporary buffer when loading image rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed
arch/arm/dts/rk3399-u-boot.dtsi | 2 + common/spl/Kconfig | 45 +++++++++++++++++++ common/spl/spl_fit.c | 79 ++++++++++++++++++++++++++++++--- 3 files changed, 120 insertions(+), 6 deletions(-)
When the load address of a FIT image isn't properly aligned, spl_load_fit_image() may write past the end of the destination buffer. It is not an issue in many cases because the memory happens to be writeable and nothing important is present in the overflow. On RockPi4 however there is a configuration where a TF-A image (bl31_0xff3b0000.bin) has to be loaded into a 8K range of SRAM memory, between 0xff3b0000 and 0xff3b2000. The end address is a hard limit, because due to the way the hardware is wired, the addresses wrap and any overflow gets written back to 0xff3b0000 thus overwriting previous data.
To address this problem, introduce a helper function which loads data using a temporary buffer allocated on the stack. The size of the buffer is defined by SPL_LOAD_FIT_IMAGE_BUFFER_SIZE (default 0x0 or disabled).
Co-developed-by: Xavier Drudis Ferran xdrudis@tinet.cat Signed-off-by: Xavier Drudis Ferran xdrudis@tinet.cat Signed-off-by: Jerome Forissier jerome.forissier@linaro.org --- common/spl/Kconfig | 45 +++++++++++++++++++++++++ common/spl/spl_fit.c | 79 ++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 118 insertions(+), 6 deletions(-)
diff --git a/common/spl/Kconfig b/common/spl/Kconfig index 50ff113cab..d95141a6f6 100644 --- a/common/spl/Kconfig +++ b/common/spl/Kconfig @@ -1339,6 +1339,51 @@ config SPL_OPENSBI_LOAD_ADDR help Load address of the OpenSBI binary.
+config SPL_LOAD_FIT_IMAGE_BUFFER_SIZE + hex "Read unaligned external FIT images to a temporary buffer in SPL" + default 0x0 + depends on SPL_LOAD_FIT + help + An aligned FIT image is such that it starts at the beginning + of a block in media and has a load_addr in its FIT header + that is DMA aligned in RAM. These aligned images can be read + directly from media to RAM. Unaligned external FIT images + are those that need reading some extra data before and/or after + the image because they don't occupy fully the blocks they're + in in media, or their destination is not DMA aligned and must + be read somewhere aligned before copying them to load_addr. + + With this option set to 0x0 full blocks will just be read in + the closest DMA aligned address, and the unaligned image + inside those read blocks will later be copied to + load_addr. Meanwhile memory outside [load_addr, + load_addr+length) will have been written. That's no problem + when the block length, image size and load_addr have been + taken into account when laying out memory. + + But in some cases not all memory is writable, or undesired + effects arise when writing outside [load_addr, + load_addr+length). For instance, in RK3399, one of the + images is ATF2, of size 8KiB which should be loaded into + INTMEM1, at address 0xff3b0000. This address is DMA aligned + but tight. It maps to a 8KiB SRAM area. If the image on + media is not block aligned some extra bytes get read before + and after, and everything extra written in + 0xff3b2000-0xff3bffff corrupts SRAM at + 0xff3b0000-0xff3b1fff before the image is memcpyied in place. + + With this option set to a nonzero value a DMA aligned buffer + will be allocated on the stack and the image will be read + in chuncks of blocks to this buffer, with each chunk being + memcpyied to [load_addr,load_addr+length), so never writing + outside the destination area. + + The advantage of enabling this option is safety, and the + disadvantage is more stack use and slower image load (one read + per chunk instead of just one). + + The default is 0x0 to replicate previous behaviour. + config TPL bool depends on SUPPORT_TPL diff --git a/common/spl/spl_fit.c b/common/spl/spl_fit.c index 1bbf824684..56775fd744 100644 --- a/common/spl/spl_fit.c +++ b/common/spl/spl_fit.c @@ -12,6 +12,7 @@ #include <log.h> #include <malloc.h> #include <mapmem.h> +#include <memalign.h> #include <spl.h> #include <sysinfo.h> #include <asm/cache.h> @@ -218,6 +219,64 @@ static int get_aligned_image_size(struct spl_load_info *info, int data_size, return (data_size + info->bl_len - 1) / info->bl_len; }
+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE != 0x0) +static int load_with_tmpbuf(struct spl_load_info *info, ulong load_addr, + ulong sector, int offs, size_t len) +{ + ALLOC_CACHE_ALIGN_BUFFER(u8, buf, + CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE); + void *dst = (void *)load_addr; + int nsect = (len + offs + info->bl_len - 1) / info->bl_len; + int bufsect = (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE) / info->bl_len; + size_t sz, tail = 0; + + if (offs) { + sz = info->bl_len - offs; + if (sz > len) + sz = len; + if (info->read(info, sector, 1, buf) != 1) + return -EIO; + memcpy(dst, buf + offs, sz); + dst += sz; + sector++; + nsect--; + } + + if (nsect) { + tail = (len + offs) % info->bl_len; + nsect--; + } + + while (nsect) { + int n = nsect; + + if (n > bufsect) + n = bufsect; + if (info->read(info, sector, n, buf) != n) + return -EIO; + sz = n * info->bl_len; + memcpy(dst, buf, sz); + dst += sz; + sector += n; + nsect -= n; + } + + if (tail) { + if (info->read(info, sector, 1, buf) != 1) + return -EIO; + memcpy(dst, buf, tail); + } + + return 0; +} +#else +static int load_with_tmpbuf(struct spl_load_info *info, ulong load_addr, + ulong sector, int offs, size_t len) +{ + return -ENOENT; +} +#endif + /** * spl_load_fit_image(): load the image described in a certain FIT node * @info: points to information about the device to load data from @@ -236,6 +295,7 @@ static int spl_load_fit_image(struct spl_load_info *info, ulong sector, const struct spl_fit_info *ctx, int node, struct spl_image_info *image_info) { + int ret; int offset; size_t length; int len; @@ -298,15 +358,22 @@ static int spl_load_fit_image(struct spl_load_info *info, ulong sector,
overhead = get_aligned_image_overhead(info, offset); nr_sectors = get_aligned_image_size(info, length, offset); - - if (info->read(info, - sector + get_aligned_image_offset(info, offset), - nr_sectors, src_ptr) != nr_sectors) - return -EIO; + sector += get_aligned_image_offset(info, offset); + if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE) { + ret = load_with_tmpbuf(info, load_addr, sector, + overhead, len); + if (ret < 0) + return ret; + src = (void *)load_addr; + } else { + if (info->read(info, sector, nr_sectors, + src_ptr) != nr_sectors) + return -EIO; + src = src_ptr + overhead; + }
debug("External data: dst=%p, offset=%x, size=%lx\n", src_ptr, offset, (unsigned long)length); - src = src_ptr + overhead; } else { /* Embedded data */ if (fit_image_get_data(fit, node, &data, &length)) {
Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.
Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.
Link: [1] https://www.rockchip.fr/Rockchip%20RK3399%20TRM%20V1.4%20Part1.pdf CC: Deepak Das deepakdas.linux@gmail.com Signed-off-by: Jerome Forissier jerome.forissier@linaro.org --- arch/arm/dts/rk3399-u-boot.dtsi | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/arm/dts/rk3399-u-boot.dtsi b/arch/arm/dts/rk3399-u-boot.dtsi index 716b9a433a..a1b6d6f007 100644 --- a/arch/arm/dts/rk3399-u-boot.dtsi +++ b/arch/arm/dts/rk3399-u-boot.dtsi @@ -124,8 +124,10 @@ &sdmmc { u-boot,dm-pre-reloc;
+#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE /* mmc to sram can't do dma, prevent aborts transferring TF-A parts */ u-boot,spl-fifo-mode; +#endif };
&spi1 {
On 6/9/22 08:23, Jerome Forissier wrote:
Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.
Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.
Link: [1] https://www.rockchip.fr/Rockchip%20RK3399%20TRM%20V1.4%20Part1.pdf CC: Deepak Das deepakdas.linux@gmail.com Signed-off-by: Jerome Forissier jerome.forissier@linaro.org
arch/arm/dts/rk3399-u-boot.dtsi | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/arm/dts/rk3399-u-boot.dtsi b/arch/arm/dts/rk3399-u-boot.dtsi index 716b9a433a..a1b6d6f007 100644 --- a/arch/arm/dts/rk3399-u-boot.dtsi +++ b/arch/arm/dts/rk3399-u-boot.dtsi @@ -124,8 +124,10 @@ &sdmmc { u-boot,dm-pre-reloc;
+#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE
Oops, that should rather be:
+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE == 0)
/* mmc to sram can't do dma, prevent aborts transferring TF-A parts */ u-boot,spl-fifo-mode; +#endif };
&spi1 {
El Tue, Jun 14, 2022 at 11:16:42AM -0700, Jerome Forissier deia:
Oops, that should rather be:
+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE == 0)
I tested with this change, not that my opinion counts much, but anyway:
Reviewed-by: Xavier Drudis Ferran xdrudis@tinet.cat Tested-by: Xavier Drudis Ferran xdrudis@tinet.cat
This changes reduce some 0.2 s the boot time of my Rock Pi 4B.
Before this series, just with u-boot/master from today c18e5fb055 dtoc: Update test_src_scan.py for new tegra compatibles plus the patches I sent to this list (I can't boot from MMC without them) https://lists.denx.de/pipermail/u-boot/2022-June/485497.html and bootstage configured, I get :
Timer summary in microseconds (10 records): Mark Elapsed Stage 1,903,436 1,903,436 board_init_f 1,903,436 0 board_init_f 2,900,331 996,895 board_init_r 4,091,657 1,191,326 id=64 4,930,650 838,993 id=65 4,930,827 177 main_loop 7,946,715 3,015,888 bootm_start 9,010,637 1,063,922 id=15 9,010,639 2 start_kernel
Accumulated time: 22,700 dm_r 479,397 dm_f
With that plus 1/2 in this series and CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE=0x3000 I get something similar (slightly slower because of >=3 correct reads instead of 1 overwriting read per image):
Timer summary in microseconds (10 records): Mark Elapsed Stage 1,979,414 1,979,414 board_init_f 1,979,414 0 board_init_f 2,976,429 997,015 board_init_r 4,166,623 1,190,194 id=64 5,005,623 839,000 id=65 5,005,800 177 main_loop 8,020,791 3,014,991 bootm_start 9,084,116 1,063,325 id=15 9,084,118 2 start_kernel
Accumulated time: 22,699 dm_r 479,480 dm_f
With that plus this 2/2 it's faster (while safer) than initially.
Timer summary in microseconds (10 records): Mark Elapsed Stage 1,709,384 1,709,384 board_init_f 1,709,384 0 board_init_f 2,706,192 996,808 board_init_r 3,895,269 1,189,077 id=64 4,733,786 838,517 id=65 4,733,963 177 main_loop 7,751,063 3,017,100 bootm_start 8,814,449 1,063,386 id=15 8,814,451 2 start_kernel
Accumulated time: 22,703 dm_r 479,520 dm_f
With this change your 2/2 patch becomes
---
From: Jerome Forissier jerome.forissier@linaro.org Date: Thu, 9 Jun 2022 17:23:22 +0200 Subject: [PATCH] rockchip: rk3399: enable spl-fifo-mode for sdmmc only when needed
Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.
Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.
Link: [1] https://www.rockchip.fr/Rockchip%20RK3399%20TRM%20V1.4%20Part1.pdf CC: Deepak Das deepakdas.linux@gmail.com Signed-off-by: Jerome Forissier jerome.forissier@linaro.org --- arch/arm/dts/rk3399-u-boot.dtsi | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/arm/dts/rk3399-u-boot.dtsi b/arch/arm/dts/rk3399-u-boot.dtsi index 716b9a433a..e0bb230022 100644 --- a/arch/arm/dts/rk3399-u-boot.dtsi +++ b/arch/arm/dts/rk3399-u-boot.dtsi @@ -124,9 +124,10 @@ &sdmmc { u-boot,dm-pre-reloc;
+#if (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE == 0) /* mmc to sram can't do dma, prevent aborts transferring TF-A parts */ u-boot,spl-fifo-mode; +#endif };
&spi1 {
On 2022/6/9 23:23, Jerome Forissier wrote:
Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.
Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.
Link: [1] https://www.rockchip.fr/Rockchip%20RK3399%20TRM%20V1.4%20Part1.pdf CC: Deepak Das deepakdas.linux@gmail.com Signed-off-by: Jerome Forissier jerome.forissier@linaro.org
Reviewed-by: Kever Yang kever.yang@rock-chips.com
Thanks,
- Kever
arch/arm/dts/rk3399-u-boot.dtsi | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/arch/arm/dts/rk3399-u-boot.dtsi b/arch/arm/dts/rk3399-u-boot.dtsi index 716b9a433a..a1b6d6f007 100644 --- a/arch/arm/dts/rk3399-u-boot.dtsi +++ b/arch/arm/dts/rk3399-u-boot.dtsi @@ -124,8 +124,10 @@ &sdmmc { u-boot,dm-pre-reloc;
+#ifndef CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE /* mmc to sram can't do dma, prevent aborts transferring TF-A parts */ u-boot,spl-fifo-mode; +#endif };
&spi1 {
On 6/29/22 05:01, Kever Yang wrote:
On 2022/6/9 23:23, Jerome Forissier wrote:
Commit 5c606ca35c42 ("rockchip: rk3399: enable spl-fifo-mode for sdmmc") mentions that the RK3399 SoC can't do DMA between SDMMC and SRAM. According to the TRM "7.3.2 Embedded SRAM access path" [1], only the 8KB SRAM at 0xff3b0000 (INTMEM1) is in this situation. The 192KB SRAM can be accessed by both DMA controllers.
Assuming the only use case for writing from MMC to INTMEM1 is loading a FIT image, and with the introduction of a temporary buffer for that purpose (CONFIG_SPL_LOAD_FIT_IMAGE_BUFFER_SIZE, which is required anyways to ensure the destination boundaries are enforced), then spl-fifo-mode is not needed anymore and DMA can be enabled safely.
Link: [1] https://www.rockchip.fr/Rockchip%20RK3399%20TRM%20V1.4%20Part1.pdf CC: Deepak Das deepakdas.linux@gmail.com Signed-off-by: Jerome Forissier jerome.forissier@linaro.org
Reviewed-by: Kever Yang kever.yang@rock-chips.com
Thanks. I will post a v2 with the corrected conditional [#if (xxx == 0)].
participants (3)
-
Jerome Forissier -
Kever Yang -
Xavier Drudis Ferran