[U-Boot] [PATCH] LZO: Add a LZO compression feature

In some case, LZO compression function is needed in U-Boot.
This patch supports LZO compression feature which is originated from Linux kernel.
Thanks. -Donggeun Kim
Signed-off-by: Donggeun Kim dg77.kim@samsung.com Signed-off-by: Kyungmin Park kyungmin.park@samsung.com --- lib/lzo/Makefile | 2 +- lib/lzo/lzo1x_compress.c | 219 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 220 insertions(+), 1 deletions(-) create mode 100644 lib/lzo/lzo1x_compress.c
diff --git a/lib/lzo/Makefile b/lib/lzo/Makefile index 5dd1bf5..0e22a49 100644 --- a/lib/lzo/Makefile +++ b/lib/lzo/Makefile @@ -27,7 +27,7 @@ LIB = $(obj)liblzo.a
SOBJS =
-COBJS-$(CONFIG_LZO) += lzo1x_decompress.o +COBJS-$(CONFIG_LZO) += lzo1x_decompress.o lzo1x_compress.o
COBJS = $(COBJS-y) SRCS := $(SOBJS:.o=.S) $(COBJS:.o=.c) diff --git a/lib/lzo/lzo1x_compress.c b/lib/lzo/lzo1x_compress.c new file mode 100644 index 0000000..b1b2637 --- /dev/null +++ b/lib/lzo/lzo1x_compress.c @@ -0,0 +1,219 @@ +/* + * LZO1X Compressor from MiniLZO + * + * Copyright (C) 1996-2005 Markus F.X.J. Oberhumer markus@oberhumer.com + * + * The full LZO package can be found at: + * http://www.oberhumer.com/opensource/lzo/ + * + * Changed for kernel use by: + * Nitin Gupta nitingupta910@gmail.com + * Richard Purdie rpurdie@openedhand.com + */ +#include <common.h> +#include <linux/lzo.h> +#include <asm/unaligned.h> +#include "lzodefs.h" + +static size_t +_lzo1x_1_do_compress(const unsigned char *in, size_t in_len, + unsigned char *out, size_t *out_len, void *wrkmem) +{ + const unsigned char * const in_end = in + in_len; + const unsigned char * const ip_end = in + in_len - M2_MAX_LEN - 5; + const unsigned char ** const dict = wrkmem; + const unsigned char *ip = in, *ii = ip; + const unsigned char *end, *m, *m_pos; + size_t m_off, m_len, dindex; + unsigned char *op = out; + + ip += 4; + + for (;;) { + dindex = ((size_t)(0x21 * DX3(ip, 5, 5, 6)) >> 5) & D_MASK; + m_pos = dict[dindex]; + + if (m_pos < in) + goto literal; + + if (ip == m_pos || ((size_t)(ip - m_pos) > M4_MAX_OFFSET)) + goto literal; + + m_off = ip - m_pos; + if (m_off <= M2_MAX_OFFSET || m_pos[3] == ip[3]) + goto try_match; + + dindex = (dindex & (D_MASK & 0x7ff)) ^ (D_HIGH | 0x1f); + m_pos = dict[dindex]; + + if (m_pos < in) + goto literal; + + if (ip == m_pos || ((size_t)(ip - m_pos) > M4_MAX_OFFSET)) + goto literal; + + m_off = ip - m_pos; + if (m_off <= M2_MAX_OFFSET || m_pos[3] == ip[3]) + goto try_match; + + goto literal; + +try_match: + if (get_unaligned((const unsigned short *)m_pos) + == get_unaligned((const unsigned short *)ip)) { + if (likely(m_pos[2] == ip[2])) + goto match; + } + +literal: + dict[dindex] = ip; + ++ip; + if (unlikely(ip >= ip_end)) + break; + continue; + +match: + dict[dindex] = ip; + if (ip != ii) { + size_t t = ip - ii; + + if (t <= 3) { + op[-2] |= t; + } else if (t <= 18) { + *op++ = (t - 3); + } else { + size_t tt = t - 18; + + *op++ = 0; + while (tt > 255) { + tt -= 255; + *op++ = 0; + } + *op++ = tt; + } + do { + *op++ = *ii++; + } while (--t > 0); + } + + ip += 3; + if (m_pos[3] != *ip++ || m_pos[4] != *ip++ + || m_pos[5] != *ip++ || m_pos[6] != *ip++ + || m_pos[7] != *ip++ || m_pos[8] != *ip++) { + --ip; + m_len = ip - ii; + + if (m_off <= M2_MAX_OFFSET) { + m_off -= 1; + *op++ = (((m_len - 1) << 5) + | ((m_off & 7) << 2)); + *op++ = (m_off >> 3); + } else if (m_off <= M3_MAX_OFFSET) { + m_off -= 1; + *op++ = (M3_MARKER | (m_len - 2)); + goto m3_m4_offset; + } else { + m_off -= 0x4000; + + *op++ = (M4_MARKER | ((m_off & 0x4000) >> 11) + | (m_len - 2)); + goto m3_m4_offset; + } + } else { + end = in_end; + m = m_pos + M2_MAX_LEN + 1; + + while (ip < end && *m == *ip) { + m++; + ip++; + } + m_len = ip - ii; + + if (m_off <= M3_MAX_OFFSET) { + m_off -= 1; + if (m_len <= 33) { + *op++ = (M3_MARKER | (m_len - 2)); + } else { + m_len -= 33; + *op++ = M3_MARKER | 0; + goto m3_m4_len; + } + } else { + m_off -= 0x4000; + if (m_len <= M4_MAX_LEN) { + *op++ = (M4_MARKER + | ((m_off & 0x4000) >> 11) + | (m_len - 2)); + } else { + m_len -= M4_MAX_LEN; + *op++ = (M4_MARKER + | ((m_off & 0x4000) >> 11)); +m3_m4_len: + while (m_len > 255) { + m_len -= 255; + *op++ = 0; + } + + *op++ = (m_len); + } + } +m3_m4_offset: + *op++ = ((m_off & 63) << 2); + *op++ = (m_off >> 6); + } + + ii = ip; + if (unlikely(ip >= ip_end)) + break; + } + + *out_len = op - out; + return in_end - ii; +} + +int lzo1x_1_compress(const unsigned char *in, size_t in_len, unsigned char *out, + size_t *out_len, void *wrkmem) +{ + const unsigned char *ii; + unsigned char *op = out; + size_t t; + + if (unlikely(in_len <= M2_MAX_LEN + 5)) { + t = in_len; + } else { + t = _lzo1x_1_do_compress(in, in_len, op, out_len, wrkmem); + op += *out_len; + } + + if (t > 0) { + ii = in + in_len - t; + + if (op == out && t <= 238) { + *op++ = (17 + t); + } else if (t <= 3) { + op[-2] |= t; + } else if (t <= 18) { + *op++ = (t - 3); + } else { + size_t tt = t - 18; + + *op++ = 0; + while (tt > 255) { + tt -= 255; + *op++ = 0; + } + + *op++ = tt; + } + do { + *op++ = *ii++; + } while (--t > 0); + } + + *op++ = M4_MARKER | 1; + *op++ = 0; + *op++ = 0; + + *out_len = op - out; + return LZO_E_OK; +}

Dear Donggeun Kim,
In message 4CA5B732.3030706@samsung.com you wrote:
In some case, LZO compression function is needed in U-Boot.
Can you please explain what use cases you have in mind?
So far, we provide only uncompression functions, and it seems nobody ever needed compression inside U-Boot yet?
If we really add this, then please make it separately configurable, so that not everybody who wants LZO decompression always has to accept the compression functions, too.
This patch supports LZO compression feature which is originated from Linux kernel.
...
--- /dev/null +++ b/lib/lzo/lzo1x_compress.c @@ -0,0 +1,219 @@ +/*
- LZO1X Compressor from MiniLZO
- Copyright (C) 1996-2005 Markus F.X.J. Oberhumer markus@oberhumer.com
- The full LZO package can be found at:
- Changed for kernel use by:
- Nitin Gupta nitingupta910@gmail.com
- Richard Purdie rpurdie@openedhand.com
- */
What exactly are the licensing terms?
Best regards,
Wolfgang Denk

Hi Wolfgang.
On Fri, Oct 1, 2010 at 7:53 PM, Wolfgang Denk wd@denx.de wrote:
Dear Donggeun Kim,
In message 4CA5B732.3030706@samsung.com you wrote:
In some case, LZO compression function is needed in U-Boot.
Can you please explain what use cases you have in mind?
There's request from other team. They want to transfer the binary but it's written as ubifs filesystem. So download the binary to RAM and then mkfs.ubifs at this address and ubinize it finally. Then write it to flash. When mkfs.ubifs, it requires it.
So far, we provide only uncompression functions, and it seems nobody ever needed compression inside U-Boot yet?
If we really add this, then please make it separately configurable, so that not everybody who wants LZO decompression always has to accept the compression functions, too.
Agree, It will use the CONFIG_LZO_COMPRESSION
This patch supports LZO compression feature which is originated from Linux kernel.
...
--- /dev/null +++ b/lib/lzo/lzo1x_compress.c @@ -0,0 +1,219 @@ +/*
- LZO1X Compressor from MiniLZO
- Copyright (C) 1996-2005 Markus F.X.J. Oberhumer markus@oberhumer.com
- The full LZO package can be found at:
- Changed for kernel use by:
- Nitin Gupta nitingupta910@gmail.com
- Richard Purdie rpurdie@openedhand.com
- */
What exactly are the licensing terms?
It's from kernel code. do you want to add the GPL v2?
Thank you, Kyungmin Park
Best regards,
Wolfgang Denk
-- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de Am I your nanny? The kernel is there to support user programs, but it's a _resource_ handler, not a baby feeder. - Linus Torvalds in Pine.LNX.3.91.960425074845.22041C-100000@linux.cs.Helsinki.FI _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot

Dear Kyungmin Park,
In message AANLkTi=HNkqfunOZMrWGG_=GhJoR6EfiGwWUkGgzc=N4@mail.gmail.com you wrote:
There's request from other team. They want to transfer the binary but it's written as ubifs filesystem. So download the binary to RAM and then mkfs.ubifs at this address and ubinize it finally. Then write it to flash. When mkfs.ubifs, it requires it.
You mean you want to add both support for ubinize and mkfs.ubifs to U-Boot? Would it not make more sense to boot a (minimal) Linux kernel for that purposes?
What exactly are the licensing terms?
It's from kernel code. do you want to add the GPL v2?
I think we should do that (if that's what the original code is released under; I'm not so certain), to make things clear.
Best regards,
Wolfgang Denk

On Fri, Oct 1, 2010 at 10:54 PM, Wolfgang Denk wd@denx.de wrote:
Dear Kyungmin Park,
In message AANLkTi=HNkqfunOZMrWGG_=GhJoR6EfiGwWUkGgzc=N4@mail.gmail.com you wrote:
There's request from other team. They want to transfer the binary but it's written as ubifs filesystem. So download the binary to RAM and then mkfs.ubifs at this address and ubinize it finally. Then write it to flash. When mkfs.ubifs, it requires it.
You mean you want to add both support for ubinize and mkfs.ubifs to U-Boot? Would it not make more sense to boot a (minimal) Linux kernel for that purposes?
Umm it's maybe difficult. This team don't use the linux and don't have linux environment. Only generate binary it and remaining job are done by each product team. The even worse there's no console when they are use it.
Enter the download mode and send the image to target.
Of course this method is not visible to their. and no behavior changed to our environment.
Thank you, Kyungmin Park
What exactly are the licensing terms?
It's from kernel code. do you want to add the GPL v2?
I think we should do that (if that's what the original code is released under; I'm not so certain), to make things clear.
Then just add the "It's from linux kernel"
Best regards,
Wolfgang Denk
-- DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de "There's always been Tower of Babel sort of bickering inside Unix, but this is the most extreme form ever. This means at least several years of confusion." - Bill Gates, founder and chairman of Microsoft, about the Open Systems Foundation _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot

On Fri, Oct 1, 2010 at 10:11 AM, Kyungmin Park wrote:
On Fri, Oct 1, 2010 at 10:54 PM, Wolfgang Denk wrote:
Kyungmin Park wrote:
There's request from other team. They want to transfer the binary but it's written as ubifs filesystem. So download the binary to RAM and then mkfs.ubifs at this address and ubinize it finally. Then write it to flash. When mkfs.ubifs, it requires it.
You mean you want to add both support for ubinize and mkfs.ubifs to U-Boot? Would it not make more sense to boot a (minimal) Linux kernel for that purposes?
Umm it's maybe difficult. This team don't use the linux and don't have linux environment. Only generate binary it and remaining job are done by each product team. The even worse there's no console when they are use it.
Enter the download mode and send the image to target.
so why dont you generate the ubifs on the development system and just transfer the image to the board ? -mike
participants (4)
-
Donggeun Kim
-
Kyungmin Park
-
Mike Frysinger
-
Wolfgang Denk