[U-Boot] [PATCH v2 0/12] bootm: Tidy up decompression and add tests

(Note: this builds on a single patch left over from a recent series, which is why it is v2. That patch refactored the code but did not add tests.)
bootm's decompression functions currently have tests in the 'test_compression' unit test command, but this calls the decompression routines directly and is more aimed at ensuring that they never overwrite their output buffer.
But bootm is critical code and it is important that test coverage be adequate for the task it performs. Existing tests in test/image provide some coverage, but not for decompression.
This series adds a new unit test specifically for bootm's decompression. It tests that bootm deals correctly with a normal decompress, a case where there is insufficient output buffer space, and a case where part of the data is corrupted.
Previously the uncompressed case had no error checking, so this is added. This is a change in behaviour so I hope it will not cause problems.
The current error message on failure is confusing, since it is unclear whether the image is too large or the data is corrupt. This is improved and patches added to make sure we can distinguish these two cases for each decompression method in most situations.
The existing 'Must RESET board to recover' message is retained, but this is likely to be unnecessary, since all decompressions routines are, I believe, careful not to overwrite their output buffer even in the case of corrupted data. It may be desirable to remove it later.
The output of the new sandbox 'ut_image_decomp' command is below:
Testing: gzip compressed Uncompressing Kernel Image ... OK Uncompressing Kernel Image ... Error: inflate() returned -5 Image too large: increase CONFIG_SYS_BOOTM_LEN Must RESET board to recover Uncompressing Kernel Image ... Error: inflate() returned -5 Image too large: increase CONFIG_SYS_BOOTM_LEN Must RESET board to recover Testing: bzip2 compressed Uncompressing Kernel Image ... OK Uncompressing Kernel Image ... Image too large: increase CONFIG_SYS_BOOTM_LEN Must RESET board to recover Uncompressing Kernel Image ... bzip2 compressed: uncompress error -4 Must RESET board to recover Testing: lzma compressed Uncompressing Kernel Image ... OK Uncompressing Kernel Image ... Image too large: increase CONFIG_SYS_BOOTM_LEN Must RESET board to recover Uncompressing Kernel Image ... lzma compressed: uncompress error 1 Must RESET board to recover Testing: lzo compressed Uncompressing Kernel Image ... OK Uncompressing Kernel Image ... Image too large: increase CONFIG_SYS_BOOTM_LEN Must RESET board to recover Uncompressing Kernel Image ... lzo compressed: uncompress error -6 Must RESET board to recover Testing: uncompressed Loading Kernel Image ... OK Loading Kernel Image ... Image too large: increase CONFIG_SYS_BOOTM_LEN Must RESET board to recover ut_image_decomp ok
Changes in v2: - Add tests to cover the code changes - Correct the overflow check for bzip2
Simon Glass (12): lzma: fix buffer bound check error further bootm: Move compression progress/error messages into a function sandbox: Correct ordering of 'sb save' commands test: Add DEBUG output option to test-fit.py bootm: Export bootm_decomp_image() test: Rename test_compression to ut_compression test: Add unit tests for bootm image decompression bootm: Use print_decomp_msg() in all cases bootm: Factor out common parts of image decompression code bzlib: Update destLen even on error gunzip: Update lenp even on error lzo: Update dst_len even on error
common/bootm.c | 150 ++++++++++++++++++++++++++------------------- common/cmd_sandbox.c | 2 +- include/bootm.h | 17 +++++ lib/bzlib.c | 2 +- lib/gunzip.c | 7 ++- lib/lzma/LzmaTools.c | 4 +- lib/lzo/lzo1x_decompress.c | 4 +- test/compression.c | 93 ++++++++++++++++++++++++++-- test/dm/sf.c | 2 +- test/image/test-fit.py | 16 ++++- 10 files changed, 218 insertions(+), 79 deletions(-)

Commit 4d3b8a0d fixed a problem with lzma decompress where it would run out of bytes to decompress. The algorithm needs to know how many uncompressed bytes it is expected to produce.
However, the fix introduced a potential buffer overrun, and causes the compression test to fail (test_compression command in sandbox).
The correct fix seems to be to use the minimum of the expected number of uncompressed bytes and the amount of output space available. That way things work normally when there is enough space, and return an error (without overrunning available space) when there is not.
Signed-off-by: Antonios Vamporakis ant@area128.com CC: Kees Cook keescook@chromium.org CC: Simon Glass sjg@chromium.org CC: Daniel Schwierzeck daniel.schwierzeck@gmail.com CC: Luka Perkov luka@openwrt.org
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
lib/lzma/LzmaTools.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/lzma/LzmaTools.c b/lib/lzma/LzmaTools.c index cfc7cb0..f88629b 100644 --- a/lib/lzma/LzmaTools.c +++ b/lib/lzma/LzmaTools.c @@ -102,7 +102,7 @@ int lzmaBuffToBuffDecompress (unsigned char *outStream, SizeT *uncompressedSize, return SZ_ERROR_OUTPUT_EOF;
/* Decompress */ - outProcessed = outSizeFull; + outProcessed = min(outSizeFull, *uncompressedSize);
WATCHDOG_RESET();
@@ -112,7 +112,7 @@ int lzmaBuffToBuffDecompress (unsigned char *outStream, SizeT *uncompressedSize, inStream, LZMA_PROPS_SIZE, LZMA_FINISH_END, &state, &g_Alloc); *uncompressedSize = outProcessed;
- debug("LZMA: Uncompresed ................ 0x%zx\n", outProcessed); + debug("LZMA: Uncompressed ............... 0x%zx\n", outProcessed);
if (res != SZ_OK) { return res;

On Tue, Dec 02, 2014 at 01:17:29PM -0700, Simon Glass wrote:
Commit 4d3b8a0d fixed a problem with lzma decompress where it would run out of bytes to decompress. The algorithm needs to know how many uncompressed bytes it is expected to produce.
However, the fix introduced a potential buffer overrun, and causes the compression test to fail (test_compression command in sandbox).
The correct fix seems to be to use the minimum of the expected number of uncompressed bytes and the amount of output space available. That way things work normally when there is enough space, and return an error (without overrunning available space) when there is not.
Signed-off-by: Antonios Vamporakis ant@area128.com CC: Kees Cook keescook@chromium.org CC: Simon Glass sjg@chromium.org CC: Daniel Schwierzeck daniel.schwierzeck@gmail.com CC: Luka Perkov luka@openwrt.org
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

This code is repeated in several places, and does not detect a common fault where the image is too large. Move it into its own function and provide a more helpful messages in this case, for compression schemes which support this.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: - Add tests to cover the code changes - Correct the overflow check for bzip2
common/bootm.c | 71 ++++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 47 insertions(+), 24 deletions(-)
diff --git a/common/bootm.c b/common/bootm.c index 6b3ea8c..915d537 100644 --- a/common/bootm.c +++ b/common/bootm.c @@ -264,7 +264,30 @@ static int bootm_find_other(cmd_tbl_t *cmdtp, int flag, int argc,
return 0; } -#endif /* USE_HOSTCC */ +#endif /* USE_HOSTC */ + +#if defined(CONFIG_GZIP) || defined(CONFIG_GZIP) || defined(CONFIG_BZIP2) || \ + defined(CONFIG_LZMA) || defined(CONFIG_LZO) +static void print_decomp_msg(const char *type_name) +{ + printf(" Uncompressing %s ... ", type_name); +} + +static int handle_decomp_error(const char *algo, size_t size, size_t unc_len, + int ret) +{ + if (size >= unc_len) + puts("Image too large: increase CONFIG_SYS_BOOTM_LEN\n"); + else + printf("%s: uncompress or overwrite error %d\n", algo, ret); + puts("Must RESET board to recover\n"); +#ifndef USE_HOSTCC + bootstage_error(BOOTSTAGE_ID_DECOMP_IMAGE); +#endif + + return BOOTM_ERR_RESET; +} +#endif
/** * decomp_image() - decompress the operating system @@ -296,19 +319,25 @@ static int decomp_image(int comp, ulong load, ulong image_start, int type, *load_end = load + image_len; break; #ifdef CONFIG_GZIP - case IH_COMP_GZIP: - printf(" Uncompressing %s ... ", type_name); - if (gunzip(load_buf, unc_len, image_buf, &image_len) != 0) { - puts("GUNZIP: uncompress, out-of-mem or overwrite error - must RESET board to recover\n"); - return BOOTM_ERR_RESET; + case IH_COMP_GZIP: { + int ret; + + print_decomp_msg(type_name); + ret = gunzip(load_buf, unc_len, image_buf, &image_len); + if (ret != 0) { + return handle_decomp_error("GUNZIP", image_len, + unc_len, ret); }
*load_end = load + image_len; break; + } #endif /* CONFIG_GZIP */ #ifdef CONFIG_BZIP2 - case IH_COMP_BZIP2: - printf(" Uncompressing %s ... ", type_name); + case IH_COMP_BZIP2: { + size_t size = unc_len; + + print_decomp_msg(type_name); /* * If we've got less than 4 MB of malloc() space, * use slower decompression algorithm which requires @@ -318,30 +347,27 @@ static int decomp_image(int comp, ulong load, ulong image_start, int type, image_buf, image_len, CONFIG_SYS_MALLOC_LEN < (4096 * 1024), 0); if (i != BZ_OK) { - printf("BUNZIP2: uncompress or overwrite error %d - must RESET board to recover\n", - i); - return BOOTM_ERR_RESET; + return handle_decomp_error("BUNZIP2", size, unc_len, + i); }
*load_end = load + unc_len; break; + } #endif /* CONFIG_BZIP2 */ #ifdef CONFIG_LZMA case IH_COMP_LZMA: { SizeT lzma_len = unc_len; int ret;
- printf(" Uncompressing %s ... ", type_name); - + print_decomp_msg(type_name); ret = lzmaBuffToBuffDecompress(load_buf, &lzma_len, image_buf, image_len); - unc_len = lzma_len; if (ret != SZ_OK) { - printf("LZMA: uncompress or overwrite error %d - must RESET board to recover\n", - ret); - bootstage_error(BOOTSTAGE_ID_DECOMP_IMAGE); - return BOOTM_ERR_RESET; + return handle_decomp_error("LZMA", lzma_len, unc_len, + ret); } + unc_len = lzma_len; *load_end = load + unc_len; break; } @@ -351,14 +377,11 @@ static int decomp_image(int comp, ulong load, ulong image_start, int type, size_t size = unc_len; int ret;
- printf(" Uncompressing %s ... ", type_name); + print_decomp_msg(type_name);
ret = lzop_decompress(image_buf, image_len, load_buf, &size); - if (ret != LZO_E_OK) { - printf("LZO: uncompress or overwrite error %d - must RESET board to recover\n", - ret); - return BOOTM_ERR_RESET; - } + if (ret != LZO_E_OK) + return handle_decomp_error("LZO", size, unc_len, ret);
*load_end = load + size; break;

On Tue, Dec 02, 2014 at 01:17:30PM -0700, Simon Glass wrote:
This code is repeated in several places, and does not detect a common fault where the image is too large. Move it into its own function and provide a more helpful messages in this case, for compression schemes which support this.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Prior to commit d455d87 there was an inconsistency between the position of the 'address' parameter in 'sb load' and 'sb save'. This was corrected but it broke some tests. Fix the tests and also the help for 'sb save'.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
common/cmd_sandbox.c | 2 +- test/dm/sf.c | 2 +- test/image/test-fit.py | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/common/cmd_sandbox.c b/common/cmd_sandbox.c index 3d9fce7..4286969 100644 --- a/common/cmd_sandbox.c +++ b/common/cmd_sandbox.c @@ -117,7 +117,7 @@ U_BOOT_CMD( "load hostfs - <addr> <filename> [<bytes> <offset>] - " "load a file from host\n" "sb ls hostfs - <filename> - list files on host\n" - "sb save hostfs - <filename> <addr> <bytes> [<offset>] - " + "sb save hostfs - <addr> <filename> <bytes> [<offset>] - " "save a file to host\n" "sb bind <dev> [<filename>] - bind "host" device to file\n" "sb info [<dev>] - show device binding & info\n" diff --git a/test/dm/sf.c b/test/dm/sf.c index 57dd134..08098a1 100644 --- a/test/dm/sf.c +++ b/test/dm/sf.c @@ -29,7 +29,7 @@ static int dm_test_spi_flash(struct dm_test_state *dms) * benefit is worth the extra complexity. */ ut_asserteq(0, run_command_list( - "sb save hostfs - spi.bin 0 200000;" + "sb save hostfs - 0 spi.bin 200000;" "sf probe;" "sf test 0 10000", -1, 0)); /* diff --git a/test/image/test-fit.py b/test/image/test-fit.py index b065fcb..0eb424d 100755 --- a/test/image/test-fit.py +++ b/test/image/test-fit.py @@ -97,9 +97,9 @@ sb load hostfs 0 %(fit_addr)x %(fit)s fdt addr %(fit_addr)x bootm start %(fit_addr)x bootm loados -sb save hostfs 0 %(kernel_out)s %(kernel_addr)x %(kernel_size)x -sb save hostfs 0 %(fdt_out)s %(fdt_addr)x %(fdt_size)x -sb save hostfs 0 %(ramdisk_out)s %(ramdisk_addr)x %(ramdisk_size)x +sb save hostfs 0 %(kernel_addr)x %(kernel_out)s %(kernel_size)x +sb save hostfs 0 %(fdt_addr)x %(fdt_out)s %(fdt_size)x +sb save hostfs 0 %(ramdisk_addr)x %(ramdisk_out)s %(ramdisk_size)x reset '''

On Tue, Dec 02, 2014 at 01:17:31PM -0700, Simon Glass wrote:
Prior to commit d455d87 there was an inconsistency between the position of the 'address' parameter in 'sb load' and 'sb save'. This was corrected but it broke some tests. Fix the tests and also the help for 'sb save'.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Sometimes it is useful to see the output from U-Boot, so add an option to make this easier.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
test/image/test-fit.py | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/test/image/test-fit.py b/test/image/test-fit.py index 0eb424d..e9e756a 100755 --- a/test/image/test-fit.py +++ b/test/image/test-fit.py @@ -20,6 +20,9 @@ import struct import sys import tempfile
+# Enable printing of all U-Boot output +DEBUG = True + # The 'command' library in patman is convenient for running commands base_path = os.path.dirname(sys.argv[0]) patman = os.path.join(base_path, '../../tools/patman') @@ -103,6 +106,10 @@ sb save hostfs 0 %(ramdisk_addr)x %(ramdisk_out)s %(ramdisk_size)x reset '''
+def debug_stdout(stdout): + if DEBUG: + print stdout + def make_fname(leaf): """Make a temporary filename
@@ -328,6 +335,7 @@ def run_fit_test(mkimage, u_boot): # We could perhaps reduce duplication with some loss of readability set_test('Kernel load') stdout = command.Output(u_boot, '-d', control_dtb, '-c', cmd) + debug_stdout(stdout) if read_file(kernel) != read_file(kernel_out): fail('Kernel not loaded', stdout) if read_file(control_dtb) == read_file(fdt_out): @@ -352,6 +360,7 @@ def run_fit_test(mkimage, u_boot): params['fdt_load'] = 'load = <%#x>;' % params['fdt_addr'] fit = make_fit(mkimage, params) stdout = command.Output(u_boot, '-d', control_dtb, '-c', cmd) + debug_stdout(stdout) if read_file(kernel) != read_file(kernel_out): fail('Kernel not loaded', stdout) if read_file(control_dtb) != read_file(fdt_out): @@ -365,6 +374,7 @@ def run_fit_test(mkimage, u_boot): params['ramdisk_load'] = 'load = <%#x>;' % params['ramdisk_addr'] fit = make_fit(mkimage, params) stdout = command.Output(u_boot, '-d', control_dtb, '-c', cmd) + debug_stdout(stdout) if read_file(ramdisk) != read_file(ramdisk_out): fail('Ramdisk not loaded', stdout)

On Tue, Dec 02, 2014 at 01:17:32PM -0700, Simon Glass wrote:
Sometimes it is useful to see the output from U-Boot, so add an option to make this easier.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Export this function for testing. Also add a parameter so that values other than CONFIG_SYS_BOOTM_LEN can be used for the maximum uncompressed size.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
common/bootm.c | 29 ++++++++++------------------- include/bootm.h | 17 +++++++++++++++++ 2 files changed, 27 insertions(+), 19 deletions(-)
diff --git a/common/bootm.c b/common/bootm.c index 915d537..10c15ef 100644 --- a/common/bootm.c +++ b/common/bootm.c @@ -289,23 +289,11 @@ static int handle_decomp_error(const char *algo, size_t size, size_t unc_len, } #endif
-/** - * decomp_image() - decompress the operating system - * - * @comp: Compression algorithm that is used (IH_COMP_...) - * @load: Destination load address in U-Boot memory - * @image_start Image start address (where we are decompressing from) - * @type: OS type (IH_OS_...) - * @load_bug: Place to decompress to - * @image_buf: Address to decompress from - * @return 0 if OK, -ve on error (BOOTM_ERR_...) - */ -static int decomp_image(int comp, ulong load, ulong image_start, int type, - void *load_buf, void *image_buf, ulong image_len, - ulong *load_end) +int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, + void *load_buf, void *image_buf, ulong image_len, + uint unc_len, ulong *load_end) { const char *type_name = genimg_get_type_name(type); - __attribute__((unused)) uint unc_len = CONFIG_SYS_BOOTM_LEN;
*load_end = load; switch (comp) { @@ -413,8 +401,9 @@ static int bootm_load_os(bootm_headers_t *images, unsigned long *load_end,
load_buf = map_sysmem(load, 0); image_buf = map_sysmem(os.image_start, image_len); - err = decomp_image(os.comp, load, os.image_start, os.type, load_buf, - image_buf, image_len, load_end); + err = bootm_decomp_image(os.comp, load, os.image_start, os.type, + load_buf, image_buf, image_len, + CONFIG_SYS_BOOTM_LEN, load_end); if (err) { bootstage_error(BOOTSTAGE_ID_DECOMP_IMAGE); return err; @@ -905,9 +894,11 @@ static int bootm_host_load_image(const void *fit, int req_image_type)
/* Allow the image to expand by a factor of 4, should be safe */ load_buf = malloc((1 << 20) + len * 4); - ret = decomp_image(imape_comp, 0, data, image_type, load_buf, - (void *)data, len, &load_end); + ret = bootm_decomp_image(imape_comp, 0, data, image_type, load_buf, + (void *)data, len, CONFIG_SYS_BOOTM_LEN, + &load_end); free(load_buf); + if (ret && ret != BOOTM_ERR_UNIMPLEMENTED) return ret;
diff --git a/include/bootm.h b/include/bootm.h index b3d1a62..6181488 100644 --- a/include/bootm.h +++ b/include/bootm.h @@ -56,4 +56,21 @@ int do_bootm_states(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
void arch_preboot_os(void);
+/** + * bootm_decomp_image() - decompress the operating system + * + * @comp: Compression algorithm that is used (IH_COMP_...) + * @load: Destination load address in U-Boot memory + * @image_start Image start address (where we are decompressing from) + * @type: OS type (IH_OS_...) + * @load_bug: Place to decompress to + * @image_buf: Address to decompress from + * @image_len: Number of bytes in @image_buf to decompress + * @unc_len: Available space for decompression + * @return 0 if OK, -ve on error (BOOTM_ERR_...) + */ +int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, + void *load_buf, void *image_buf, ulong image_len, + uint unc_len, ulong *load_end); + #endif

On Tue, Dec 02, 2014 at 01:17:33PM -0700, Simon Glass wrote:
Export this function for testing. Also add a parameter so that values other than CONFIG_SYS_BOOTM_LEN can be used for the maximum uncompressed size.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Try to keep the names of the unit test commands consistent.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
test/compression.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/test/compression.c b/test/compression.c index 139ea01..c460567 100644 --- a/test/compression.c +++ b/test/compression.c @@ -313,9 +313,8 @@ out: return ret; }
- -static int do_test_compression(cmd_tbl_t *cmdtp, int flag, int argc, - char * const argv[]) +static int do_ut_compression(cmd_tbl_t *cmdtp, int flag, int argc, + char *const argv[]) { int err = 0;
@@ -324,12 +323,12 @@ static int do_test_compression(cmd_tbl_t *cmdtp, int flag, int argc, err += run_test("lzma", compress_using_lzma, uncompress_using_lzma); err += run_test("lzo", compress_using_lzo, uncompress_using_lzo);
- printf("test_compression %s\n", err == 0 ? "ok" : "FAILED"); + printf("ut_compression %s\n", err == 0 ? "ok" : "FAILED");
return err; }
U_BOOT_CMD( - test_compression, 5, 1, do_test_compression, + ut_compression, 5, 1, do_ut_compression, "Basic test of compressors: gzip bzip2 lzma lzo", "" );

On Tue, Dec 02, 2014 at 01:17:34PM -0700, Simon Glass wrote:
Try to keep the names of the unit test commands consistent.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Use each compression method (including uncompressed). Test for normal operation, insufficient space and corrupted data.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
test/compression.c | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+)
diff --git a/test/compression.c b/test/compression.c index c460567..ea2e4ad 100644 --- a/test/compression.c +++ b/test/compression.c @@ -7,8 +7,10 @@ #define DEBUG
#include <common.h> +#include <bootm.h> #include <command.h> #include <malloc.h> +#include <asm/io.h>
#include <u-boot/zlib.h> #include <bzlib.h> @@ -328,7 +330,89 @@ static int do_ut_compression(cmd_tbl_t *cmdtp, int flag, int argc, return err; }
+static int compress_using_none(void *in, unsigned long in_size, + void *out, unsigned long out_max, + unsigned long *out_size) +{ + /* Here we just copy */ + memcpy(out, in, in_size); + *out_size = in_size; + + return 0; +} + +/** + * run_bootm_test() - Run tests on the bootm decopmression function + * + * @comp_type: Compression type to test + * @compress: Our function to compress data + * @return 0 if OK, non-zero on failure + */ +static int run_bootm_test(int comp_type, mutate_func compress) +{ + ulong compress_size = 1024; + void *compress_buff; + int unc_len; + int err = 0; + const ulong image_start = 0; + const ulong load_addr = 0x1000; + ulong load_end; + + printf("Testing: %s\n", genimg_get_comp_name(comp_type)); + compress_buff = map_sysmem(image_start, 0); + unc_len = strlen(plain); + compress((void *)plain, unc_len, compress_buff, compress_size, + &compress_size); + err = bootm_decomp_image(comp_type, load_addr, image_start, + IH_TYPE_KERNEL, map_sysmem(load_addr, 0), + compress_buff, compress_size, unc_len, + &load_end); + if (err) + return err; + err = bootm_decomp_image(comp_type, load_addr, image_start, + IH_TYPE_KERNEL, map_sysmem(load_addr, 0), + compress_buff, compress_size, unc_len - 1, + &load_end); + if (!err) + return -EINVAL; + + /* We can't detect corruption when not decompressing */ + if (comp_type == IH_COMP_NONE) + return 0; + memset(compress_buff + compress_size / 2, '\x49', + compress_size / 2); + err = bootm_decomp_image(comp_type, load_addr, image_start, + IH_TYPE_KERNEL, map_sysmem(load_addr, 0), + compress_buff, compress_size, 0x10000, + &load_end); + if (!err) + return -EINVAL; + + return 0; +} + +static int do_ut_image_decomp(cmd_tbl_t *cmdtp, int flag, int argc, + char *const argv[]) +{ + int err = 0; + + err = run_bootm_test(IH_COMP_GZIP, compress_using_gzip); + err |= run_bootm_test(IH_COMP_BZIP2, compress_using_bzip2); + err |= run_bootm_test(IH_COMP_LZMA, compress_using_lzma); + err |= run_bootm_test(IH_COMP_LZO, compress_using_lzo); + err |= run_bootm_test(IH_COMP_NONE, compress_using_none); + + printf("ut_image_decomp %s\n", err == 0 ? "ok" : "FAILED"); + + return 0; +} + U_BOOT_CMD( ut_compression, 5, 1, do_ut_compression, "Basic test of compressors: gzip bzip2 lzma lzo", "" ); + +U_BOOT_CMD( + ut_image_decomp, 5, 1, do_ut_image_decomp, + "Basic test of bootm decompression", "" +);

On Tue, Dec 02, 2014 at 01:17:35PM -0700, Simon Glass wrote:
Use each compression method (including uncompressed). Test for normal operation, insufficient space and corrupted data.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Refactor to allow this function to be used to announce the image being loaded regardless of compression type and even when there is no decompression.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
common/bootm.c | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-)
diff --git a/common/bootm.c b/common/bootm.c index 10c15ef..4a5c5ea 100644 --- a/common/bootm.c +++ b/common/bootm.c @@ -266,13 +266,25 @@ static int bootm_find_other(cmd_tbl_t *cmdtp, int flag, int argc, } #endif /* USE_HOSTC */
-#if defined(CONFIG_GZIP) || defined(CONFIG_GZIP) || defined(CONFIG_BZIP2) || \ - defined(CONFIG_LZMA) || defined(CONFIG_LZO) -static void print_decomp_msg(const char *type_name) +/** + * print_decomp_msg() - Print a suitable decompression/loading message + * + * @type: OS type (IH_OS_...) + * @comp_type: Compression type being used (IH_COMP_...) + * @is_xip: true if the load address matches the image start + */ +static void print_decomp_msg(int comp_type, int type, bool is_xip) { - printf(" Uncompressing %s ... ", type_name); + const char *name = genimg_get_type_name(type); + + if (comp_type == IH_COMP_NONE) + printf(" %s %s ... ", is_xip ? "XIP" : "Loading", name); + else + printf(" Uncompressing %s ... ", name); }
+#if defined(CONFIG_GZIP) || defined(CONFIG_GZIP) || defined(CONFIG_BZIP2) || \ + defined(CONFIG_LZMA) || defined(CONFIG_LZO) static int handle_decomp_error(const char *algo, size_t size, size_t unc_len, int ret) { @@ -293,24 +305,18 @@ int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, void *load_buf, void *image_buf, ulong image_len, uint unc_len, ulong *load_end) { - const char *type_name = genimg_get_type_name(type); - *load_end = load; + print_decomp_msg(comp, type, load == image_start); switch (comp) { case IH_COMP_NONE: - if (load == image_start) { - printf(" XIP %s ... ", type_name); - } else { - printf(" Loading %s ... ", type_name); + if (load != image_start) memmove_wd(load_buf, image_buf, image_len, CHUNKSZ); - } *load_end = load + image_len; break; #ifdef CONFIG_GZIP case IH_COMP_GZIP: { int ret;
- print_decomp_msg(type_name); ret = gunzip(load_buf, unc_len, image_buf, &image_len); if (ret != 0) { return handle_decomp_error("GUNZIP", image_len, @@ -325,7 +331,6 @@ int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, case IH_COMP_BZIP2: { size_t size = unc_len;
- print_decomp_msg(type_name); /* * If we've got less than 4 MB of malloc() space, * use slower decompression algorithm which requires @@ -348,7 +353,6 @@ int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, SizeT lzma_len = unc_len; int ret;
- print_decomp_msg(type_name); ret = lzmaBuffToBuffDecompress(load_buf, &lzma_len, image_buf, image_len); if (ret != SZ_OK) { @@ -365,8 +369,6 @@ int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, size_t size = unc_len; int ret;
- print_decomp_msg(type_name); - ret = lzop_decompress(image_buf, image_len, load_buf, &size); if (ret != LZO_E_OK) return handle_decomp_error("LZO", size, unc_len, ret);

On Tue, Dec 02, 2014 at 01:17:36PM -0700, Simon Glass wrote:
Refactor to allow this function to be used to announce the image being loaded regardless of compression type and even when there is no decompression.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

Adjust the code so that the error reporting can all be done at the end, and is the same for each decompression method. Try to detect when decompression fails due to lack of space. Keep the behaviour of resetting on failure even though there should be no memory corruption now.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
common/bootm.c | 88 ++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 49 insertions(+), 39 deletions(-)
diff --git a/common/bootm.c b/common/bootm.c index 4a5c5ea..e2dc164 100644 --- a/common/bootm.c +++ b/common/bootm.c @@ -283,97 +283,103 @@ static void print_decomp_msg(int comp_type, int type, bool is_xip) printf(" Uncompressing %s ... ", name); }
-#if defined(CONFIG_GZIP) || defined(CONFIG_GZIP) || defined(CONFIG_BZIP2) || \ - defined(CONFIG_LZMA) || defined(CONFIG_LZO) -static int handle_decomp_error(const char *algo, size_t size, size_t unc_len, - int ret) +/** + * handle_decomp_error() - display a decompression error + * + * This function tries to produce a useful message. In the case where the + * uncompressed size is the same as the available space, we can assume that + * the image is too large for the buffer. + * + * @comp_type: Compression type being used (IH_COMP_...) + * @uncomp_size: Number of bytes uncompressed + * @unc_len: Amount of space available for decompression + * @ret: Error code to report + * @return BOOTM_ERR_RESET, indicating that the board must be reset + */ +static int handle_decomp_error(int comp_type, size_t uncomp_size, + size_t unc_len, int ret) { - if (size >= unc_len) - puts("Image too large: increase CONFIG_SYS_BOOTM_LEN\n"); + const char *name = genimg_get_comp_name(comp_type); + + if (uncomp_size >= unc_len) + printf("Image too large: increase CONFIG_SYS_BOOTM_LEN\n"); else - printf("%s: uncompress or overwrite error %d\n", algo, ret); - puts("Must RESET board to recover\n"); + printf("%s: uncompress error %d\n", name, ret); + + /* + * The decompression routines are now safe, so will not write beyond + * their bounds. Probably it is not necessary to reset, but maintain + * the current behaviour for now. + */ + printf("Must RESET board to recover\n"); #ifndef USE_HOSTCC bootstage_error(BOOTSTAGE_ID_DECOMP_IMAGE); #endif
return BOOTM_ERR_RESET; } -#endif
int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, void *load_buf, void *image_buf, ulong image_len, uint unc_len, ulong *load_end) { + int ret = 0; + *load_end = load; print_decomp_msg(comp, type, load == image_start); + + /* + * Load the image to the right place, decompressing if needed. After + * this, image_len will be set to the number of uncompressed bytes + * loaded, ret will be non-zero on error. + */ switch (comp) { case IH_COMP_NONE: - if (load != image_start) + if (load == image_start) + break; + if (image_len <= unc_len) memmove_wd(load_buf, image_buf, image_len, CHUNKSZ); - *load_end = load + image_len; + else + ret = 1; break; #ifdef CONFIG_GZIP case IH_COMP_GZIP: { - int ret; - ret = gunzip(load_buf, unc_len, image_buf, &image_len); - if (ret != 0) { - return handle_decomp_error("GUNZIP", image_len, - unc_len, ret); - } - - *load_end = load + image_len; break; } #endif /* CONFIG_GZIP */ #ifdef CONFIG_BZIP2 case IH_COMP_BZIP2: { - size_t size = unc_len; + uint size = unc_len;
/* * If we've got less than 4 MB of malloc() space, * use slower decompression algorithm which requires * at most 2300 KB of memory. */ - int i = BZ2_bzBuffToBuffDecompress(load_buf, &unc_len, + ret = BZ2_bzBuffToBuffDecompress(load_buf, &size, image_buf, image_len, CONFIG_SYS_MALLOC_LEN < (4096 * 1024), 0); - if (i != BZ_OK) { - return handle_decomp_error("BUNZIP2", size, unc_len, - i); - } - - *load_end = load + unc_len; + image_len = size; break; } #endif /* CONFIG_BZIP2 */ #ifdef CONFIG_LZMA case IH_COMP_LZMA: { SizeT lzma_len = unc_len; - int ret;
ret = lzmaBuffToBuffDecompress(load_buf, &lzma_len, image_buf, image_len); - if (ret != SZ_OK) { - return handle_decomp_error("LZMA", lzma_len, unc_len, - ret); - } - unc_len = lzma_len; - *load_end = load + unc_len; + image_len = lzma_len; break; } #endif /* CONFIG_LZMA */ #ifdef CONFIG_LZO case IH_COMP_LZO: { size_t size = unc_len; - int ret;
ret = lzop_decompress(image_buf, image_len, load_buf, &size); - if (ret != LZO_E_OK) - return handle_decomp_error("LZO", size, unc_len, ret); - - *load_end = load + size; + image_len = size; break; } #endif /* CONFIG_LZO */ @@ -382,6 +388,10 @@ int bootm_decomp_image(int comp, ulong load, ulong image_start, int type, return BOOTM_ERR_UNIMPLEMENTED; }
+ if (ret) + return handle_decomp_error(comp, image_len, unc_len, ret); + *load_end = load + image_len; + puts("OK\n");
return 0;

On Tue, Dec 02, 2014 at 01:17:37PM -0700, Simon Glass wrote:
Adjust the code so that the error reporting can all be done at the end, and is the same for each decompression method. Try to detect when decompression fails due to lack of space. Keep the behaviour of resetting on failure even though there should be no memory corruption now.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

This allows the caller to easily detect how much of the destination buffer has been used.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
lib/bzlib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/bzlib.c b/lib/bzlib.c index 5844e18..9262e40 100644 --- a/lib/bzlib.c +++ b/lib/bzlib.c @@ -1350,11 +1350,11 @@ int BZ_API(BZ2_bzBuffToBuffDecompress) strm.avail_out = *destLen;
ret = BZ2_bzDecompress ( &strm ); + *destLen -= strm.avail_out; if (ret == BZ_OK) goto output_overflow_or_eof; if (ret != BZ_STREAM_END) goto errhandler;
/* normal termination */ - *destLen -= strm.avail_out; BZ2_bzDecompressEnd ( &strm ); return BZ_OK;

On Tue, Dec 02, 2014 at 01:17:38PM -0700, Simon Glass wrote:
This allows the caller to easily detect how much of the destination buffer has been used.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

This allows the caller to easily detect how much of the destination buffer has been used.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
lib/gunzip.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lib/gunzip.c b/lib/gunzip.c index 35abfb3..f469fcbe 100644 --- a/lib/gunzip.c +++ b/lib/gunzip.c @@ -73,6 +73,7 @@ int zunzip(void *dst, int dstlen, unsigned char *src, unsigned long *lenp, int stoponerr, int offset) { z_stream s; + int err = 0; int r;
s.zalloc = gzalloc; @@ -92,13 +93,13 @@ int zunzip(void *dst, int dstlen, unsigned char *src, unsigned long *lenp, if (stoponerr == 1 && r != Z_STREAM_END && (s.avail_out == 0 || r != Z_BUF_ERROR)) { printf("Error: inflate() returned %d\n", r); - inflateEnd(&s); - return -1; + err = -1; + break; } s.avail_in = *lenp - offset - (int)(s.next_out - (unsigned char*)dst); } while (r == Z_BUF_ERROR); *lenp = s.next_out - (unsigned char *) dst; inflateEnd(&s);
- return 0; + return err; }

On Tue, Dec 02, 2014 at 01:17:39PM -0700, Simon Glass wrote:
This allows the caller to easily detect how much of the destination buffer has been used.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!

This allows the caller to easily detect how much of the destination buffer has been used.
Signed-off-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
lib/lzo/lzo1x_decompress.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/lzo/lzo1x_decompress.c b/lib/lzo/lzo1x_decompress.c index 35f3793..ebdf10b 100644 --- a/lib/lzo/lzo1x_decompress.c +++ b/lib/lzo/lzo1x_decompress.c @@ -102,8 +102,10 @@ int lzop_decompress(const unsigned char *src, size_t src_len, tmp = dlen; r = lzo1x_decompress_safe((u8 *) src, slen, dst, &tmp);
- if (r != LZO_E_OK) + if (r != LZO_E_OK) { + *dst_len = dst - start; return r; + }
if (dlen != tmp) return LZO_E_ERROR;

On Tue, Dec 02, 2014 at 01:17:40PM -0700, Simon Glass wrote:
This allows the caller to easily detect how much of the destination buffer has been used.
Signed-off-by: Simon Glass sjg@chromium.org
Applied to u-boot/master, thanks!
participants (2)
-
Simon Glass
-
Tom Rini