[PATCH 11/16] lib: enable lzma decompression support for SPL build
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao weijie.gao@mediatek.com --- lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.
+config SPL_LZMA + bool "Enable LZMA decompression support for SPL build" + help + This enables support for LZMA compression altorithm for SPL boot. + config SPL_LZO bool "Enable LZO decompression support in SPL" help diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
On 1/8/20 4:01 AM, Weijie Gao wrote:
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao weijie.gao@mediatek.com
lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.
+config SPL_LZMA
- bool "Enable LZMA decompression support for SPL build"
- help
This enables support for LZMA compression altorithm for SPL boot.
%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
- config SPL_LZO bool "Enable LZO decompression support in SPL" help
diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
On 1/8/20 4:01 AM, Weijie Gao wrote:
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao weijie.gao@mediatek.com
lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.
+config SPL_LZMA
- bool "Enable LZMA decompression support for SPL build"
- help
This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
- config SPL_LZO bool "Enable LZO decompression support in SPL" help
diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2...
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Best Regards,
Weijie
On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote:
On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
On 1/8/20 4:01 AM, Weijie Gao wrote:
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao weijie.gao@mediatek.com
lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.
+config SPL_LZMA
- bool "Enable LZMA decompression support for SPL build"
- help
This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
- config SPL_LZO bool "Enable LZO decompression support in SPL" help
diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2...
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and report back? Thanks!
On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote:
On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote:
On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
On 1/8/20 4:01 AM, Weijie Gao wrote:
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao weijie.gao@mediatek.com
lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.
+config SPL_LZMA
- bool "Enable LZMA decompression support for SPL build"
- help
This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
- config SPL_LZO bool "Enable LZO decompression support in SPL" help
diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2...
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and report back? Thanks!
My purpose is to reduce the size of u-boot binary, and my plan is to compress the payload and the SPL decompress it at runtime. I need smallest size here, not fastest decompression speed.
So there are two points must be taken into consideration: the size of the decompressor (i.e. u-boot-spl.bin) and the size of the compressed payload (u-boot.bin.gz/bz2/...).
Below is my test results, including all compression algorithms supported by uImage, plus zstd:
Platform: mips - mt7628
Uncompressed size of u-boot.bin: 302736 (295KiB)
Commands for compression: gzip -c -f -k -9 bzip2 -c -f -k -9 lzma -c -z -k -9 lzop -c -f -9 lz4 -f -9 zstd --ultra -f
Results: algo. | spl.bin size | compressed size | % of uncomp | total size ------------------------------------------------------------------------ gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB) ------------------------------------------------------------------------ bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB) ------------------------------------------------------------------------ lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB) ------------------------------------------------------------------------ lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB) ------------------------------------------------------------------------ lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB) ------------------------------------------------------------------------ zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB)
As you can see lzma has the best compression ratio. The is the reason I chose lzma here.
Best Regards
On Fri, Jan 10, 2020 at 04:21:35PM +0800, Weijie Gao wrote:
On Thu, 2020-01-09 at 12:45 -0500, Tom Rini wrote:
On Thu, Jan 09, 2020 at 04:10:12PM +0800, Weijie Gao wrote:
On Wed, 2020-01-08 at 08:22 +0100, Heinrich Schuchardt wrote:
On 1/8/20 4:01 AM, Weijie Gao wrote:
This patch enables LZMA decompression support for SPL build
Signed-off-by: Weijie Gao weijie.gao@mediatek.com
lib/Kconfig | 5 +++++ lib/Makefile | 1 + 2 files changed, 6 insertions(+)
diff --git a/lib/Kconfig b/lib/Kconfig index 965cf7bc03..2f38e37802 100644 --- a/lib/Kconfig +++ b/lib/Kconfig @@ -418,6 +418,11 @@ config SPL_LZ4 fast compression and decompression speed. It belongs to the LZ77 family of byte-oriented compression schemes.
+config SPL_LZMA
- bool "Enable LZMA decompression support for SPL build"
- help
This enables support for LZMA compression altorithm for SPL boot.%s/altorithm/algorithm/g
Why do we need yet another algorithm for compression?
Doesn't Zstandard provide a much faster decompression? Cf. https://sites.google.com/site/powturbo/home/web-compression
Best regards
Heinrich
- config SPL_LZO bool "Enable LZO decompression support in SPL" help
diff --git a/lib/Makefile b/lib/Makefile index 1fb650cd90..8e51eec01c 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -64,6 +64,7 @@ obj-$(CONFIG_$(SPL_)ZLIB) += zlib/ obj-$(CONFIG_$(SPL_)ZSTD) += zstd/ obj-$(CONFIG_$(SPL_)GZIP) += gunzip.o obj-$(CONFIG_$(SPL_)LZO) += lzo/ +obj-$(CONFIG_$(SPL_)LZMA) += lzma/ obj-$(CONFIG_$(SPL_)LZ4) += lz4_wrapper.o
obj-$(CONFIG_LIBAVB) += libavb/
Hi Heinrich,
I need an algorithm with the maximum compression ratio to reduce the size of the u-boot payload as much as possible.
I believe currently lzma and xz are the best choices. Refer to: https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2...
Although they will take longer time for compression and decompression, considering that the u-boot payload is usually small (<= 1MiB), there will not be significant effect on decompression time.
Where does zstd fall in the list however? Can you please try that and report back? Thanks!
My purpose is to reduce the size of u-boot binary, and my plan is to compress the payload and the SPL decompress it at runtime. I need smallest size here, not fastest decompression speed.
So there are two points must be taken into consideration: the size of the decompressor (i.e. u-boot-spl.bin) and the size of the compressed payload (u-boot.bin.gz/bz2/...).
Below is my test results, including all compression algorithms supported by uImage, plus zstd:
Platform: mips - mt7628
Uncompressed size of u-boot.bin: 302736 (295KiB)
Commands for compression: gzip -c -f -k -9 bzip2 -c -f -k -9 lzma -c -z -k -9 lzop -c -f -9 lz4 -f -9 zstd --ultra -f
Results: algo. | spl.bin size | compressed size | % of uncomp | total size
gzip | 26232 (25.6K) | 141207 (137KiB) | 46.64% | 167439 (163KiB)
bzip2 | 30616 (29.8K) | 135403 (132KiB) | 44.72% | 166019 (162KiB)
lzma | 18920 (18.4K) | 112660 (110KiB) | 37.21% | 131580 (128KiB)
lzo | 13288 (12.9K) | 152409 (148KiB) | 50.34% | 165697 (161KiB)
lz4 | 12408 (12.1K) | 167311 (163KiB) | 55.27% | 179719 (175KiB)
zstd | 44824 (43.7K) | 149765 (146KiB) | 49.47% | 194589 (190KiB)
As you can see lzma has the best compression ratio. The is the reason I chose lzma here.
Thanks, that's exactly the kind of data I was looking for.
Reviewed-by: Tom Rini trini@konsulko.com
participants (3)
-
Heinrich Schuchardt -
Tom Rini -
Weijie Gao