[U-Boot] [RFC PATCH 0/6] Some speed improvements to U-Boot JFFS2 code (updated)
Hello everybody,
here is a set of changes we made to improve U-Boot JFFS2 code performance. We still can't reach Linux's performance but improvements are significant.
Patches are againt current u-boot git tree.
Any comments are welcome.
Regards, Ilya.
We need to update i_version inside cycle to find really latest version inside jffs2_1pass_list_inodes(). With that fixed we can use isize inside dump_inode() instead of calling expensive jffs2_1pass_read_inode().
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 920d2fd..35743fc 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -940,9 +940,7 @@ static inline u32 dump_inode(struct b_lists * pL, struct jffs2_raw_dirent *d, st st.st_mtime = i->mtime; st.st_mode = i->mode; st.st_ino = i->ino; - - /* neither dsize nor isize help us.. do it the long way */ - st.st_size = jffs2_1pass_read_inode(pL, i->ino, NULL); + st.st_size = i->isize;
dump_stat(&st, fname);
@@ -976,6 +974,7 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino) jNode = (struct jffs2_raw_inode *) get_fl_mem(b2->offset, sizeof(ojNode), &ojNode); if (jNode->ino == jDir->ino && jNode->version >= i_version) { + i_version = jNode->version; if (i) put_fl_mem(i);
Dear Ilya Yanok,
In message 1226594976-6087-2-git-send-email-yanok@emcraft.com you wrote:
We need to update i_version inside cycle to find really latest version inside jffs2_1pass_list_inodes(). With that fixed we can use isize inside dump_inode() instead of calling expensive jffs2_1pass_read_inode().
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
This patch adds sector_size field to part_info structure (used by new JFFS2 code).
Signed-off-by: Ilya Yanok yanok@emcraft.com --- common/cmd_jffs2.c | 20 +++++++++++++++++--- include/jffs2/load_kernel.h | 1 + 2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index 791a572..e2bf198 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -339,11 +339,15 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) extern flash_info_t flash_info[]; flash_info_t *flash; int offset_aligned; - u32 end_offset; + u32 end_offset, sector_size = 0; int i;
flash = &flash_info[id->num];
+ /* size of last sector */ + part->sector_size = flash->size - + (flash->start[flash->sector_count-1] - flash->start[0]); + offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) { if ((flash->start[i] - flash->start[0]) == part->offset) { @@ -358,12 +362,18 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) }
end_offset = part->offset + part->size; + offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) { + if (i) { + sector_size = flash->start[i] - flash->start[i-1]; + if (part->sector_size < sector_size) + part->sector_size = sector_size; + } if ((flash->start[i] - flash->start[0]) == end_offset) - return 0; + offset_aligned = 1; }
- if (flash->size == end_offset) + if (offset_aligned || flash->size == end_offset) return 0;
printf("%s%d: partition (%s) size alignment incorrect\n", @@ -389,6 +399,8 @@ static int part_validate_nand(struct mtdids *id, struct part_info *part)
nand = &nand_info[id->num];
+ part->sector_size = nand->erasesize; + if ((unsigned long)(part->offset) % nand->erasesize) { printf("%s%d: partition (%s) start offset alignment incorrect\n", MTD_DEV_TYPE(id->type), id->num, part->name); @@ -424,6 +436,8 @@ static int part_validate_onenand(struct mtdids *id, struct part_info *part)
mtd = &onenand_mtd;
+ part->sector_size = mtd->erasesize; + if ((unsigned long)(part->offset) % mtd->erasesize) { printf("%s%d: partition (%s) start offset" "alignment incorrect\n", diff --git a/include/jffs2/load_kernel.h b/include/jffs2/load_kernel.h index 551fd0c..c0442a2 100644 --- a/include/jffs2/load_kernel.h +++ b/include/jffs2/load_kernel.h @@ -50,6 +50,7 @@ struct part_info { u32 offset; /* offset within device */ void *jffs2_priv; /* used internaly by jffs2 */ u32 mask_flags; /* kernel MTD mask flags */ + u32 sector_size; /* size of sector */ struct mtd_device *dev; /* parent device */ };
On 19:49 Thu 13 Nov , Ilya Yanok wrote:
This patch adds sector_size field to part_info structure (used by new JFFS2 code).
Signed-off-by: Ilya Yanok yanok@emcraft.com
common/cmd_jffs2.c | 20 +++++++++++++++++--- include/jffs2/load_kernel.h | 1 + 2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index 791a572..e2bf198 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -339,11 +339,15 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) extern flash_info_t flash_info[]; flash_info_t *flash; int offset_aligned;
- u32 end_offset;
- u32 end_offset, sector_size = 0;
please spit in two lines
Best Regards, J.
Jean-Christophe PLAGNIOL-VILLARD wrote:
On 19:49 Thu 13 Nov , Ilya Yanok wrote:
This patch adds sector_size field to part_info structure (used by new JFFS2 code).
Signed-off-by: Ilya Yanok yanok@emcraft.com
common/cmd_jffs2.c | 20 +++++++++++++++++--- include/jffs2/load_kernel.h | 1 + 2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index 791a572..e2bf198 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -339,11 +339,15 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) extern flash_info_t flash_info[]; flash_info_t *flash; int offset_aligned;
- u32 end_offset;
- u32 end_offset, sector_size = 0;
please spit in two lines
Why do you waste peoples' time with this pedantic nit-picking? There's absolutely nothing wrong with having them both on the same line, especially if it's the only thing you can object to in the patch.
regards, Ben
Dear Ilya Yanok,
In message 1226594976-6087-3-git-send-email-yanok@emcraft.com you wrote:
This patch adds sector_size field to part_info structure (used by new JFFS2 code).
Signed-off-by: Ilya Yanok yanok@emcraft.com
common/cmd_jffs2.c | 20 +++++++++++++++++--- include/jffs2/load_kernel.h | 1 + 2 files changed, 18 insertions(+), 3 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
Rewrites jffs2_1pass_build_lists() function in style of Linux's jffs2_scan_medium() and jffs2_scan_eraseblock(). This includes: - Caching flash acceses - Smart dealing with free space
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 227 ++++++++++++++++++++++++++++++++++++------------ 1 files changed, 171 insertions(+), 56 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 35743fc..30a6dba 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -389,6 +389,12 @@ static inline void *get_fl_mem_nor(u32 off) return (void*)addr; }
+static inline void *get_fl_mem_nor_copy(u32 off, u32 size, void *ext_buf) +{ + memcpy(ext_buf, get_fl_mem_nor(off), size); + return ext_buf; +} + static inline void *get_node_mem_nor(u32 off) { return (void*)get_fl_mem_nor(off); @@ -405,8 +411,11 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf) struct mtdids *id = current_part->dev->id;
#if defined(CONFIG_CMD_FLASH) - if (id->type == MTD_DEV_TYPE_NOR) + if (id->type == MTD_DEV_TYPE_NOR) { + if (ext_buf) + return get_fl_mem_nor_copy(off, size, ext_buf); return get_fl_mem_nor(off); + } #endif
#if defined(CONFIG_JFFS2_NAND) && defined(CONFIG_CMD_NAND) @@ -478,9 +487,6 @@ static char *compr_names[] = { #endif };
-/* Spinning wheel */ -static char spinner[] = { '|', '/', '-', '\' }; - /* Memory management */ struct mem_block { u32 index; @@ -651,23 +657,6 @@ static int compare_dirents(struct b_node *new, struct b_node *old) } #endif
-static u32 -jffs2_scan_empty(u32 start_offset, struct part_info *part) -{ - char *max = (char *)(part->offset + part->size - sizeof(struct jffs2_raw_inode)); - char *offset = (char *)(part->offset + start_offset); - u32 off; - - while (offset < max && - *(u32*)get_fl_mem((u32)offset, sizeof(u32), &off) == 0xFFFFFFFF) { - offset += sizeof(u32); - /* return if spinning is due */ - if (((u32)offset & ((1 << SPIN_BLKSIZE)-1)) == 0) break; - } - - return (u32)offset - part->offset; -} - void jffs2_free_cache(struct part_info *part) { @@ -766,6 +755,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) put_fl_mem(jNode); continue; } + if (!data_crc(jNode)) { + put_fl_mem(jNode); + continue; + }
lDest = (uchar *) (dest + jNode->offset); #if 0 @@ -1268,17 +1261,33 @@ dump_dirents(struct b_lists *pL) } #endif
+#define min_t(type, x, y) ({ \ + type __min1 = (x); \ + type __min2 = (y); \ + __min1 < __min2 ? __min1: __min2; }) + +#define DEFAULT_EMPTY_SCAN_SIZE 4096 + +static inline uint32_t EMPTY_SCAN_SIZE(uint32_t sector_size) +{ + if (sector_size < DEFAULT_EMPTY_SCAN_SIZE) + return sector_size; + else + return DEFAULT_EMPTY_SCAN_SIZE; +} + static u32 jffs2_1pass_build_lists(struct part_info * part) { struct b_lists *pL; struct jffs2_unknown_node *node; - u32 offset, oldoffset = 0; - u32 max = part->size - sizeof(struct jffs2_raw_inode); - u32 counter = 0; + u32 nr_sectors = part->size/part->sector_size; + u32 i; u32 counter4 = 0; u32 counterF = 0; u32 counterN = 0; + u32 buf_size = DEFAULT_EMPTY_SCAN_SIZE; + char *buf;
/* turn off the lcd. Refreshing the lcd adds 50% overhead to the */ /* jffs2 list building enterprise nope. in newer versions the overhead is */ @@ -1288,70 +1297,176 @@ jffs2_1pass_build_lists(struct part_info * part) /* if we are building a list we need to refresh the cache. */ jffs_init_1pass_list(part); pL = (struct b_lists *)part->jffs2_priv; - offset = 0; + buf = malloc(buf_size); puts ("Scanning JFFS2 FS: ");
/* start at the beginning of the partition */ - while (offset < max) { - if ((oldoffset >> SPIN_BLKSIZE) != (offset >> SPIN_BLKSIZE)) { - printf("\b\b%c ", spinner[counter++ % sizeof(spinner)]); - oldoffset = offset; - } + for (i = 0; i < nr_sectors; i++) { + uint32_t sector_ofs = i * part->sector_size; + uint32_t buf_ofs = sector_ofs; + uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size); + uint32_t ofs, prevofs;
WATCHDOG_RESET(); + get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
- node = (struct jffs2_unknown_node *) get_node_mem((u32)part->offset + offset); - if (node->magic == JFFS2_MAGIC_BITMASK && hdr_crc(node)) { + /* We temporarily use 'ofs' as a pointer into the buffer/jeb */ + ofs = 0; + + /* Scan only 4KiB of 0xFF before declaring it's empty */ + while (ofs < EMPTY_SCAN_SIZE(part->sector_size) && + *(uint32_t *)(&buf[ofs]) == 0xFFFFFFFF) + ofs += 4; + + if (ofs == EMPTY_SCAN_SIZE(part->sector_size)) + continue; + + ofs += sector_ofs; + prevofs = ofs - 1; + + scan_more: + while (ofs < sector_ofs + part->sector_size) { + if (ofs == prevofs) { + printf("offset %08x already seen, skip\n", ofs); + ofs += 4; + counter4++; + continue; + } + prevofs = ofs; + if (sector_ofs + part->sector_size < + ofs + sizeof(*node)) + break; + if (buf_ofs + buf_len < ofs + sizeof(*node)) { + buf_len = min_t(uint32_t, buf_size, sector_ofs + + part->sector_size - ofs); + get_fl_mem((u32)part->offset + ofs, buf_len, + buf); + buf_ofs = ofs; + } + + node = (struct jffs2_unknown_node *)&buf[ofs-buf_ofs]; + + if (*(uint32_t *)(&buf[ofs-buf_ofs]) == 0xffffffff) { + uint32_t inbuf_ofs; + uint32_t empty_start, scan_end; + + empty_start = ofs; + ofs += 4; + scan_end = min_t(uint32_t, EMPTY_SCAN_SIZE( + part->sector_size)/8, + buf_len); + more_empty: + inbuf_ofs = ofs - buf_ofs; + while (inbuf_ofs < scan_end) { + if (*(uint32_t *)(&buf[inbuf_ofs]) != + 0xffffffff) + goto scan_more; + + inbuf_ofs += 4; + ofs += 4; + } + /* Ran off end. */ + + /* See how much more there is to read in this + * eraseblock... + */ + buf_len = min_t(uint32_t, buf_size, + sector_ofs + + part->sector_size - ofs); + if (!buf_len) { + /* No more to read. Break out of main + * loop without marking this range of + * empty space as dirty (because it's + * not) + */ + break; + } + scan_end = buf_len; + get_fl_mem((u32)part->offset + ofs, buf_len, + buf); + buf_ofs = ofs; + goto more_empty; + } + if (node->magic != JFFS2_MAGIC_BITMASK || + !hdr_crc(node)) { + ofs += 4; + counter4++; + continue; + } + if (ofs + node->totlen > + sector_ofs + part->sector_size) { + ofs += 4; + counter4++; + continue; + } /* if its a fragment add it */ - if (node->nodetype == JFFS2_NODETYPE_INODE && - inode_crc((struct jffs2_raw_inode *) node) && - data_crc((struct jffs2_raw_inode *) node)) { + switch (node->nodetype) { + case JFFS2_NODETYPE_INODE: + if (buf_ofs + buf_len < ofs + sizeof(struct + jffs2_raw_inode)) { + get_fl_mem((u32)part->offset + ofs, + buf_len, buf); + buf_ofs = ofs; + node = (void *)buf; + } + if (!inode_crc((struct jffs2_raw_inode *) node)) + break; + if (insert_node(&pL->frag, (u32) part->offset + - offset) == NULL) { - put_fl_mem(node); + ofs) == NULL) return 0; + break; + case JFFS2_NODETYPE_DIRENT: + if (buf_ofs + buf_len < ofs + sizeof(struct + jffs2_raw_dirent) + + ((struct + jffs2_raw_dirent *) + node)->nsize) { + get_fl_mem((u32)part->offset + ofs, + buf_len, buf); + buf_ofs = ofs; + node = (void *)buf; } - } else if (node->nodetype == JFFS2_NODETYPE_DIRENT && - dirent_crc((struct jffs2_raw_dirent *) node) && - dirent_name_crc((struct jffs2_raw_dirent *) node)) { + + if (!dirent_crc((struct jffs2_raw_dirent *) + node) || + !dirent_name_crc( + (struct + jffs2_raw_dirent *) + node)) + break; if (! (counterN%100)) puts ("\b\b. "); if (insert_node(&pL->dir, (u32) part->offset + - offset) == NULL) { - put_fl_mem(node); + ofs) == NULL) return 0; - } counterN++; - } else if (node->nodetype == JFFS2_NODETYPE_CLEANMARKER) { + break; + case JFFS2_NODETYPE_CLEANMARKER: if (node->totlen != sizeof(struct jffs2_unknown_node)) printf("OOPS Cleanmarker has bad size " "%d != %zu\n", node->totlen, sizeof(struct jffs2_unknown_node)); - } else if (node->nodetype == JFFS2_NODETYPE_PADDING) { + break; + case JFFS2_NODETYPE_PADDING: if (node->totlen < sizeof(struct jffs2_unknown_node)) printf("OOPS Padding has bad size " "%d < %zu\n", node->totlen, sizeof(struct jffs2_unknown_node)); - } else { + break; + default: printf("Unknown node type: %x len %d offset 0x%x\n", node->nodetype, - node->totlen, offset); + node->totlen, ofs); } - offset += ((node->totlen + 3) & ~3); + ofs += ((node->totlen + 3) & ~3); counterF++; - } else if (node->magic == JFFS2_EMPTY_BITMASK && - node->nodetype == JFFS2_EMPTY_BITMASK) { - offset = jffs2_scan_empty(offset, part); - } else { /* if we know nothing, we just step and look. */ - offset += 4; - counter4++; } -/* printf("unknown node magic %4.4x %4.4x @ %lx\n", node->magic, node->nodetype, (unsigned long)node); */ - put_fl_mem(node); }
+ free(buf); putstr("\b\b done.\r\n"); /* close off the dots */ /* turn the lcd back on. */ /* splash(); */
/* start at the beginning of the partition */
- while (offset < max) {
if ((oldoffset >> SPIN_BLKSIZE) != (offset >> SPIN_BLKSIZE)) {printf("\b\b%c ", spinner[counter++ % sizeof(spinner)]);oldoffset = offset;}
for (i = 0; i < nr_sectors; i++) {
uint32_t sector_ofs = i * part->sector_size;uint32_t buf_ofs = sector_ofs;uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size);uint32_t ofs, prevofs;WATCHDOG_RESET();
get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
node = (struct jffs2_unknown_node *) get_node_mem((u32)part->offset + offset);if (node->magic == JFFS2_MAGIC_BITMASK && hdr_crc(node)) {
/* We temporarily use 'ofs' as a pointer into the buffer/jeb */ofs = 0;/* Scan only 4KiB of 0xFF before declaring it's empty */while (ofs < EMPTY_SCAN_SIZE(part->sector_size) &&*(uint32_t *)(&buf[ofs]) == 0xFFFFFFFF)
please use readx/writex
ofs += 4;if (ofs == EMPTY_SCAN_SIZE(part->sector_size))continue;ofs += sector_ofs;prevofs = ofs - 1;- scan_more:
while (ofs < sector_ofs + part->sector_size) {if (ofs == prevofs) {printf("offset %08x already seen, skip\n", ofs);ofs += 4;counter4++;continue;}prevofs = ofs;if (sector_ofs + part->sector_size <ofs + sizeof(*node))break;if (buf_ofs + buf_len < ofs + sizeof(*node)) {buf_len = min_t(uint32_t, buf_size, sector_ofs+ part->sector_size - ofs);get_fl_mem((u32)part->offset + ofs, buf_len,buf);buf_ofs = ofs;}node = (struct jffs2_unknown_node *)&buf[ofs-buf_ofs];if (*(uint32_t *)(&buf[ofs-buf_ofs]) == 0xffffffff) {
please use readx/writex
uint32_t inbuf_ofs;uint32_t empty_start, scan_end;empty_start = ofs;ofs += 4;scan_end = min_t(uint32_t, EMPTY_SCAN_SIZE(part->sector_size)/8,buf_len);more_empty:inbuf_ofs = ofs - buf_ofs;while (inbuf_ofs < scan_end) {if (*(uint32_t *)(&buf[inbuf_ofs]) !=0xffffffff)
please use readx/writex and so on
Best Regards, J.
Hello Jean-Christophe,
Jean-Christophe PLAGNIOL-VILLARD wrote:
/* start at the beginning of the partition */
- while (offset < max) {
if ((oldoffset >> SPIN_BLKSIZE) != (offset >> SPIN_BLKSIZE)) {printf("\b\b%c ", spinner[counter++ % sizeof(spinner)]);oldoffset = offset;}
for (i = 0; i < nr_sectors; i++) {
uint32_t sector_ofs = i * part->sector_size;uint32_t buf_ofs = sector_ofs;uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size);uint32_t ofs, prevofs;WATCHDOG_RESET();
get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
node = (struct jffs2_unknown_node *) get_node_mem((u32)part->offset + offset);if (node->magic == JFFS2_MAGIC_BITMASK && hdr_crc(node)) {
/* We temporarily use 'ofs' as a pointer into the buffer/jeb */ofs = 0;/* Scan only 4KiB of 0xFF before declaring it's empty */while (ofs < EMPTY_SCAN_SIZE(part->sector_size) &&*(uint32_t *)(&buf[ofs]) == 0xFFFFFFFF)please use readx/writex
Why? it's memory buffer actually.
Regards, Ilya.
Dear Ilya Yanok,
In message 1226594976-6087-4-git-send-email-yanok@emcraft.com you wrote:
Rewrites jffs2_1pass_build_lists() function in style of Linux's jffs2_scan_medium() and jffs2_scan_eraseblock(). This includes:
- Caching flash acceses
- Smart dealing with free space
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 227 ++++++++++++++++++++++++++++++++++++------------ 1 files changed, 171 insertions(+), 56 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
Hi Ilya,
On Thursday 13 November 2008, Ilya Yanok wrote:
Rewrites jffs2_1pass_build_lists() function in style of Linux's jffs2_scan_medium() and jffs2_scan_eraseblock(). This includes:
- Caching flash acceses
- Smart dealing with free space
After this patch is now applied to the main u-boot repo, I get the following warning when compiling my new board with OneNAND & JFFS2 support:
[stefan@kubuntu u-boot (vcth-onenand-ubi-ver3)]$ make -s -j5 jffs2_1pass.c:1411:1: warning: "min_t" redefined In file included from /home/stefan/git/u-boot/u-boot/include/linux/mtd/mtd-abi.h:11, from /home/stefan/git/u-boot/u-boot/include/linux/mtd/mtd.h:11, from jffs2_1pass.c:275: /home/stefan/git/u-boot/u-boot/include/linux/mtd/compat.h:34:1: warning: this is the location of the previous definition
As it seems you added the "min_t" define to this file (see below). Why was this needed? I can safely remove it from my version. Do you see a problem with removing it on your test platform?
Thanks.
<snip>
@@ -1268,17 +1261,33 @@ dump_dirents(struct b_lists *pL) } #endif
+#define min_t(type, x, y) ({ \
- type __min1 = (x); \
- type __min2 = (y); \
- __min1 < __min2 ? __min1: __min2; })
+#define DEFAULT_EMPTY_SCAN_SIZE 4096
...
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
Hi Stefan,
Stefan Roese wrote:
After this patch is now applied to the main u-boot repo, I get the following warning when compiling my new board with OneNAND & JFFS2 support:
[stefan@kubuntu u-boot (vcth-onenand-ubi-ver3)]$ make -s -j5 jffs2_1pass.c:1411:1: warning: "min_t" redefined In file included from /home/stefan/git/u-boot/u-boot/include/linux/mtd/mtd-abi.h:11, from /home/stefan/git/u-boot/u-boot/include/linux/mtd/mtd.h:11, from jffs2_1pass.c:275: /home/stefan/git/u-boot/u-boot/include/linux/mtd/compat.h:34:1: warning: this is the location of the previous definition
As it seems you added the "min_t" define to this file (see below). Why was this needed? I can safely remove it from my version. Do you see a problem with removing it on your test platform?
Thanks for reporting. That's obviously because we already have min_t definition in <linux/mtd/compat.h> and OneNAND support includes this header from somewhere while NOR support doesn't. I've changed jffs2_1pass.c to directly include <linux/mtd/compat.h> directly instead of defining own min_t. Please see patch in the follow up message.
Regards, Ilya.
Include <linux/mtd/compat.h> header for min_t definition instead of providing our own one. Removes warnings in case of OneNAND support enabled.
Although I thinks it's a bit silly to include <linux/mtd/compat.h> just for min_t...
Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 6 +----- 1 files changed, 1 insertions(+), 5 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index be7c1a1..11b66ab 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -119,6 +119,7 @@ #include <watchdog.h> #include <jffs2/jffs2.h> #include <jffs2/jffs2_1pass.h> +#include <linux/mtd/compat.h>
#include "jffs2_private.h"
@@ -1408,11 +1409,6 @@ dump_dirents(struct b_lists *pL) } #endif
-#define min_t(type, x, y) ({ \ - type __min1 = (x); \ - type __min2 = (y); \ - __min1 < __min2 ? __min1: __min2; }) - #define DEFAULT_EMPTY_SCAN_SIZE 4096
static inline uint32_t EMPTY_SCAN_SIZE(uint32_t sector_size)
On Thursday 11 December 2008, Ilya Yanok wrote:
Include <linux/mtd/compat.h> header for min_t definition instead of providing our own one. Removes warnings in case of OneNAND support enabled.
Although I thinks it's a bit silly to include <linux/mtd/compat.h> just for min_t...
Signed-off-by: Ilya Yanok yanok@emcraft.com
Acked-by: Stefan Roese sr@denx.de
Thanks.
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
Dear Ilya Yanok,
In message 1228963917-11820-1-git-send-email-yanok@emcraft.com you wrote:
Include <linux/mtd/compat.h> header for min_t definition instead of providing our own one. Removes warnings in case of OneNAND support enabled.
Although I thinks it's a bit silly to include <linux/mtd/compat.h> just for min_t...
Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 6 +----- 1 files changed, 1 insertions(+), 5 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
With this patch JFFS2 code allocates memory buffer of max_totlen size (size of the largest node, calculated during scan time) and uses it to store entire node. Speeds up loading. If malloc fails we use old ways to do things.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 116 ++++++++++++++++++++++++++++----------------- fs/jffs2/jffs2_private.h | 2 +- 2 files changed, 73 insertions(+), 45 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 30a6dba..73d3ddc 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -245,7 +245,7 @@ static void *get_fl_mem_nand(u32 off, u32 size, void *ext_buf) return buf; }
-static void *get_node_mem_nand(u32 off) +static void *get_node_mem_nand(u32 off, void *ext_buf) { struct jffs2_unknown_node node; void *ret = NULL; @@ -255,7 +255,7 @@ static void *get_node_mem_nand(u32 off)
if (!(ret = get_fl_mem_nand(off, node.magic == JFFS2_MAGIC_BITMASK ? node.totlen : sizeof(node), - NULL))) { + ext_buf))) { printf("off = %#x magic %#x type %#x node.totlen = %d\n", off, node.magic, node.nodetype, node.totlen); } @@ -344,7 +344,7 @@ static void *get_fl_mem_onenand(u32 off, u32 size, void *ext_buf) return buf; }
-static void *get_node_mem_onenand(u32 off) +static void *get_node_mem_onenand(u32 off, void *ext_buf) { struct jffs2_unknown_node node; void *ret = NULL; @@ -354,7 +354,7 @@ static void *get_node_mem_onenand(u32 off)
ret = get_fl_mem_onenand(off, node.magic == JFFS2_MAGIC_BITMASK ? node.totlen : sizeof(node), - NULL); + ext_buf); if (!ret) { printf("off = %#x magic %#x type %#x node.totlen = %d\n", off, node.magic, node.nodetype, node.totlen); @@ -377,7 +377,7 @@ static void put_fl_mem_onenand(void *buf) * NOR flash memory is mapped in processor's address space, * just return address. */ -static inline void *get_fl_mem_nor(u32 off) +static inline void *get_fl_mem_nor(u32 off, u32 size, void *ext_buf) { u32 addr = off; struct mtdids *id = current_part->dev->id; @@ -386,18 +386,22 @@ static inline void *get_fl_mem_nor(u32 off) flash_info_t *flash = &flash_info[id->num];
addr += flash->start[0]; + if (ext_buf) { + memcpy(ext_buf, (void *)addr, size); + return ext_buf; + } return (void*)addr; }
-static inline void *get_fl_mem_nor_copy(u32 off, u32 size, void *ext_buf) +static inline void *get_node_mem_nor(u32 off, void *ext_buf) { - memcpy(ext_buf, get_fl_mem_nor(off), size); - return ext_buf; -} + struct jffs2_unknown_node *pNode;
-static inline void *get_node_mem_nor(u32 off) -{ - return (void*)get_fl_mem_nor(off); + /* pNode will point directly to flash - don't provide external buffer + and don't care about size */ + pNode = get_fl_mem_nor(off, 0, NULL); + return (void *)get_fl_mem_nor(off, pNode->magic == JFFS2_MAGIC_BITMASK ? + pNode->totlen : sizeof(*pNode), ext_buf); } #endif
@@ -412,9 +416,7 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf)
#if defined(CONFIG_CMD_FLASH) if (id->type == MTD_DEV_TYPE_NOR) { - if (ext_buf) - return get_fl_mem_nor_copy(off, size, ext_buf); - return get_fl_mem_nor(off); + return get_fl_mem_nor(off, size, ext_buf); } #endif
@@ -432,34 +434,38 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf) return (void*)off; }
-static inline void *get_node_mem(u32 off) +static inline void *get_node_mem(u32 off, void *ext_buf) { struct mtdids *id = current_part->dev->id;
#if defined(CONFIG_CMD_FLASH) if (id->type == MTD_DEV_TYPE_NOR) - return get_node_mem_nor(off); + return get_node_mem_nor(off, ext_buf); #endif
#if defined(CONFIG_JFFS2_NAND) && \ defined(CONFIG_CMD_NAND) if (id->type == MTD_DEV_TYPE_NAND) - return get_node_mem_nand(off); + return get_node_mem_nand(off, ext_buf); #endif
#if defined(CONFIG_CMD_ONENAND) if (id->type == MTD_DEV_TYPE_ONENAND) - return get_node_mem_onenand(off); + return get_node_mem_onenand(off, ext_buf); #endif
printf("get_node_mem: unknown device type, using raw offset!\n"); return (void*)off; }
-static inline void put_fl_mem(void *buf) +static inline void put_fl_mem(void *buf, void *ext_buf) { struct mtdids *id = current_part->dev->id;
+ /* If buf is the same as ext_buf, it was provided by the caller - + we shouldn't free it then. */ + if (buf == ext_buf) + return; switch (id->type) { #if defined(CONFIG_JFFS2_NAND) && defined(CONFIG_CMD_NAND) case MTD_DEV_TYPE_NAND: @@ -666,6 +672,7 @@ jffs2_free_cache(struct part_info *part) pL = (struct b_lists *)part->jffs2_priv; free_nodes(&pL->frag); free_nodes(&pL->dir); + free(pL->readbuf); free(pL); } } @@ -712,7 +719,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) */ for (b = pL->frag.listHead; b != NULL; b = b->next) { jNode = (struct jffs2_raw_inode *) get_fl_mem(b->offset, - sizeof(struct jffs2_raw_inode), NULL); + sizeof(struct jffs2_raw_inode), pL->readbuf); if ((inode == jNode->ino)) { /* get actual file length from the newest node */ if (jNode->version >= latestVersion) { @@ -720,12 +727,13 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) latestVersion = jNode->version; } } - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); } #endif
for (b = pL->frag.listHead; b != NULL; b = b->next) { - jNode = (struct jffs2_raw_inode *) get_node_mem(b->offset); + jNode = (struct jffs2_raw_inode *) get_node_mem(b->offset, + pL->readbuf); if ((inode == jNode->ino)) { #if 0 putLabeledWord("\r\n\r\nread_inode: totlen = ", jNode->totlen); @@ -752,11 +760,11 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) src = ((uchar *) jNode) + sizeof(struct jffs2_raw_inode); /* ignore data behind latest known EOF */ if (jNode->offset > totalSize) { - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); continue; } if (!data_crc(jNode)) { - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); continue; }
@@ -797,7 +805,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) default: /* unknown */ putLabeledWord("UNKOWN COMPRESSION METHOD = ", jNode->compr); - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); return -1; break; } @@ -809,7 +817,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) #endif } counter++; - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); }
#if 0 @@ -835,12 +843,13 @@ jffs2_1pass_find_inode(struct b_lists * pL, const char *name, u32 pino) counter = 0; /* we need to search all and return the inode with the highest version */ for(b = pL->dir.listHead; b; b = b->next, counter++) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); if ((pino == jDir->pino) && (len == jDir->nsize) && (jDir->ino) && /* 0 for unlink */ (!strncmp((char *)jDir->name, name, len))) { /* a match */ if (jDir->version < version) { - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); continue; }
@@ -862,7 +871,7 @@ jffs2_1pass_find_inode(struct b_lists * pL, const char *name, u32 pino) putLabeledWord("b = ", (u32) b); putLabeledWord("counter = ", counter); #endif - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } return inode; } @@ -956,7 +965,8 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino) struct jffs2_raw_dirent *jDir;
for (b = pL->dir.listHead; b; b = b->next) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); if ((pino == jDir->pino) && (jDir->ino)) { /* ino=0 -> unlink */ u32 i_version = 0; struct jffs2_raw_inode ojNode; @@ -969,20 +979,23 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino) if (jNode->ino == jDir->ino && jNode->version >= i_version) { i_version = jNode->version; if (i) - put_fl_mem(i); + put_fl_mem(i, NULL);
if (jDir->type == DT_LNK) - i = get_node_mem(b2->offset); + i = get_node_mem(b2->offset, + NULL); else - i = get_fl_mem(b2->offset, sizeof(*i), NULL); + i = get_fl_mem(b2->offset, + sizeof(*i), + NULL); } b2 = b2->next; }
dump_inode(pL, jDir, i); - put_fl_mem(i); + put_fl_mem(i, NULL); } - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } return pino; } @@ -1060,10 +1073,11 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino)
/* we need to search all and return the inode with the highest version */ for(b = pL->dir.listHead; b; b = b->next) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); if (ino == jDir->ino) { if (jDir->version < version) { - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); continue; }
@@ -1080,7 +1094,7 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino) jDirFoundPino = jDir->pino; version = jDir->version; } - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } /* now we found the right entry again. (shoulda returned inode*) */ if (jDirFoundType != DT_LNK) @@ -1089,7 +1103,8 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino) /* it's a soft link so we follow it again. */ b2 = pL->frag.listHead; while (b2) { - jNode = (struct jffs2_raw_inode *) get_node_mem(b2->offset); + jNode = (struct jffs2_raw_inode *) get_node_mem(b2->offset, + pL->readbuf); if (jNode->ino == jDirFoundIno) { src = (unsigned char *)jNode + sizeof(struct jffs2_raw_inode);
@@ -1101,11 +1116,11 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino) #endif strncpy(tmp, (char *)src, jNode->dsize); tmp[jNode->dsize] = '\0'; - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); break; } b2 = b2->next; - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); } /* ok so the name of the new file to find is in tmp */ /* if it starts with a slash it is root based else shared dirs */ @@ -1240,7 +1255,8 @@ dump_dirents(struct b_lists *pL) putstr("\r\n\r\n******The directory Entries******\r\n"); b = pL->dir.listHead; while (b) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); putstr("\r\n"); putnstr(jDir->name, jDir->nsize); putLabeledWord("\r\n\tbuild_list: magic = ", jDir->magic); @@ -1256,7 +1272,7 @@ dump_dirents(struct b_lists *pL) putLabeledWord("\tbuild_list: name_crc = ", jDir->name_crc); putLabeledWord("\tbuild_list: offset = ", b->offset); /* FIXME: ? [RS] */ b = b->next; - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } } #endif @@ -1286,6 +1302,7 @@ jffs2_1pass_build_lists(struct part_info * part) u32 counter4 = 0; u32 counterF = 0; u32 counterN = 0; + u32 max_totlen = 0; u32 buf_size = DEFAULT_EMPTY_SCAN_SIZE; char *buf;
@@ -1415,6 +1432,8 @@ jffs2_1pass_build_lists(struct part_info * part) if (insert_node(&pL->frag, (u32) part->offset + ofs) == NULL) return 0; + if (max_totlen < node->totlen) + max_totlen = node->totlen; break; case JFFS2_NODETYPE_DIRENT: if (buf_ofs + buf_len < ofs + sizeof(struct @@ -1440,6 +1459,8 @@ jffs2_1pass_build_lists(struct part_info * part) if (insert_node(&pL->dir, (u32) part->offset + ofs) == NULL) return 0; + if (max_totlen < node->totlen) + max_totlen = node->totlen; counterN++; break; case JFFS2_NODETYPE_CLEANMARKER: @@ -1468,6 +1489,13 @@ jffs2_1pass_build_lists(struct part_info * part)
free(buf); putstr("\b\b done.\r\n"); /* close off the dots */ + + /* We don't care if malloc failed - then each read operation will + * allocate its own buffer as necessary (NAND) or will read directly + * from flash (NOR). + */ + pL->readbuf = malloc(max_totlen); + /* turn the lcd back on. */ /* splash(); */
diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h index 9745762..3633dea 100644 --- a/fs/jffs2/jffs2_private.h +++ b/fs/jffs2/jffs2_private.h @@ -24,7 +24,7 @@ struct b_list { struct b_lists { struct b_list dir; struct b_list frag; - + void *readbuf; };
struct b_compr_info {
Dear Ilya Yanok,
In message 1226594976-6087-5-git-send-email-yanok@emcraft.com you wrote:
With this patch JFFS2 code allocates memory buffer of max_totlen size (size of the largest node, calculated during scan time) and uses it to store entire node. Speeds up loading. If malloc fails we use old ways to do things.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 116 ++++++++++++++++++++++++++++----------------- fs/jffs2/jffs2_private.h | 2 +- 2 files changed, 73 insertions(+), 45 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
This patch adds support for reading fs information from summary node instead of scanning full eraseblock.
Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++- fs/jffs2/summary.h | 163 +++++++++++++++++++++++++++++++++++++++++ include/jffs2/jffs2.h | 19 +++++ 3 files changed, 368 insertions(+), 1 deletions(-) create mode 100644 fs/jffs2/summary.h
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 73d3ddc..4e49a05 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -138,6 +138,8 @@ # define DEBUGF(fmt,args...) #endif
+#include "summary.h" + /* keeps pointer to currentlu processed partition */ static struct part_info *current_part;
@@ -1214,6 +1216,132 @@ jffs2_1pass_rescan_needed(struct part_info *part) return 0; }
+#define dbg_summary(...) do {} while (0); +/* Process the stored summary information - helper function for + * jffs2_sum_scan_sumnode() + */ + +static int jffs2_sum_process_sum_data(struct part_info *part, uint32_t offset, + struct jffs2_raw_summary *summary, + struct b_lists *pL) +{ + void *sp; + int i; + + sp = summary->sum; + + for (i = 0; i < summary->sum_num; i++) { + dbg_summary("processing summary index %d\n", i); + + switch (((struct jffs2_sum_unknown_flash *)sp)->nodetype) { + case JFFS2_NODETYPE_INODE: { + struct jffs2_sum_inode_flash *spi; + spi = sp; + + dbg_summary("Inode at 0x%08x-0x%08x\n", + offset + spi->offset, + offset + spi->offset + spi->totlen); + + if (insert_node(&pL->frag, (u32) part->offset + + offset + spi->offset) == NULL) + return -1; + + sp += JFFS2_SUMMARY_INODE_SIZE; + + break; + } + + case JFFS2_NODETYPE_DIRENT: { + struct jffs2_sum_dirent_flash *spd; + spd = sp; + + dbg_summary("Dirent at 0x%08x-0x%08x\n", + offset + spd->offset, + offset + spd->offset + spd->totlen); + + if (insert_node(&pL->dir, (u32) part->offset + + offset + spd->offset) == NULL) + return -1; + + sp += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); + + break; + } + default : { + uint16_t nodetype = + ((struct jffs2_sum_unknown_flash *) + sp)->nodetype; + printf("Unsupported node type %x found in " + "summary!\n", nodetype); + break; + } + } + } + return 0; +} + +/* Process the summary node - called from jffs2_scan_eraseblock() */ +int jffs2_sum_scan_sumnode(struct part_info *part, uint32_t offset, + struct jffs2_raw_summary *summary, uint32_t sumsize, + struct b_lists *pL) +{ + struct jffs2_unknown_node crcnode; + int ret, ofs; + uint32_t crc; + + ofs = part->sector_size - sumsize; + + dbg_summary("summary found for 0x%08x at 0x%08x (0x%x bytes)\n", + offset, offset + ofs, sumsize); + + /* OK, now check for node validity and CRC */ + crcnode.magic = JFFS2_MAGIC_BITMASK; + crcnode.nodetype = JFFS2_NODETYPE_SUMMARY; + crcnode.totlen = summary->totlen; + crc = crc32_no_comp(0, (uchar *)&crcnode, sizeof(crcnode)-4); + + if (summary->hdr_crc != crc) { + dbg_summary("Summary node header is corrupt (bad CRC or " + "no summary at all)\n"); + goto crc_err; + } + + if (summary->totlen != sumsize) { + dbg_summary("Summary node is corrupt (wrong erasesize?)\n"); + goto crc_err; + } + + crc = crc32_no_comp(0, (uchar *)summary, + sizeof(struct jffs2_raw_summary)-8); + + if (summary->node_crc != crc) { + dbg_summary("Summary node is corrupt (bad CRC)\n"); + goto crc_err; + } + + crc = crc32_no_comp(0, (uchar *)summary->sum, + sumsize - sizeof(struct jffs2_raw_summary)); + + if (summary->sum_crc != crc) { + dbg_summary("Summary node data is corrupt (bad CRC)\n"); + goto crc_err; + } + + if (summary->cln_mkr) + dbg_summary("Summary : CLEANMARKER node \n"); + + ret = jffs2_sum_process_sum_data(part, offset, summary, pL); + if (ret) + return ret; /* real error */ + + return 1; + +crc_err: + putstr("Summary node crc error, skipping summary information.\n"); + + return 0; +} + #ifdef DEBUG_FRAGMENTS static void dump_fragments(struct b_lists *pL) @@ -1321,10 +1449,65 @@ jffs2_1pass_build_lists(struct part_info * part) for (i = 0; i < nr_sectors; i++) { uint32_t sector_ofs = i * part->sector_size; uint32_t buf_ofs = sector_ofs; - uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size); + uint32_t buf_len; uint32_t ofs, prevofs; + struct jffs2_sum_marker *sm; + void *sumptr = NULL; + uint32_t sumlen; + int ret;
WATCHDOG_RESET(); + + buf_len = sizeof(*sm); + + /* Read as much as we want into the _end_ of the preallocated + * buffer + */ + get_fl_mem(part->offset + sector_ofs + part->sector_size - + buf_len, buf_len, buf + buf_size - buf_len); + + sm = (void *)buf + buf_size - sizeof(*sm); + if (sm->magic == JFFS2_SUM_MAGIC) { + sumlen = part->sector_size - sm->offset; + sumptr = buf + buf_size - sumlen; + + /* Now, make sure the summary itself is available */ + if (sumlen > buf_size) { + /* Need to kmalloc for this. */ + sumptr = malloc(sumlen); + if (!sumptr) { + putstr("Can't get memory for summary " + "node!\n"); + return 0; + } + memcpy(sumptr + sumlen - buf_len, buf + + buf_size - buf_len, buf_len); + } + if (buf_len < sumlen) { + /* Need to read more so that the entire summary + * node is present + */ + get_fl_mem(part->offset + sector_ofs + + part->sector_size - sumlen, + sumlen - buf_len, sumptr); + } + } + + if (sumptr) { + ret = jffs2_sum_scan_sumnode(part, sector_ofs, sumptr, + sumlen, pL); + + if (buf_size && sumlen > buf_size) + free(sumptr); + if (ret < 0) + return 0; + if (ret) + continue; + + } + + buf_len = EMPTY_SCAN_SIZE(part->sector_size); + get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
/* We temporarily use 'ofs' as a pointer into the buffer/jeb */ @@ -1477,6 +1660,8 @@ jffs2_1pass_build_lists(struct part_info * part) node->totlen, sizeof(struct jffs2_unknown_node)); break; + case JFFS2_NODETYPE_SUMMARY: + break; default: printf("Unknown node type: %x len %d offset 0x%x\n", node->nodetype, diff --git a/fs/jffs2/summary.h b/fs/jffs2/summary.h new file mode 100644 index 0000000..834933c --- /dev/null +++ b/fs/jffs2/summary.h @@ -0,0 +1,163 @@ +/* + * JFFS2 -- Journalling Flash File System, Version 2. + * + * Copyright © 2004 Ferenc Havasi havasi@inf.u-szeged.hu, + * Zoltan Sogor weth@inf.u-szeged.hu, + * Patrik Kluba pajko@halom.u-szeged.hu, + * University of Szeged, Hungary + * + * For licensing information, see the file 'LICENCE' in this directory. + * + */ + +#ifndef JFFS2_SUMMARY_H +#define JFFS2_SUMMARY_H + +#define BLK_STATE_ALLFF 0 +#define BLK_STATE_CLEAN 1 +#define BLK_STATE_PARTDIRTY 2 +#define BLK_STATE_CLEANMARKER 3 +#define BLK_STATE_ALLDIRTY 4 +#define BLK_STATE_BADBLOCK 5 + +#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff +#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash)) +#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x)) +#define JFFS2_SUMMARY_XATTR_SIZE (sizeof(struct jffs2_sum_xattr_flash)) +#define JFFS2_SUMMARY_XREF_SIZE (sizeof(struct jffs2_sum_xref_flash)) + +/* Summary structures used on flash */ + +struct jffs2_sum_unknown_flash +{ + __u16 nodetype; /* node type */ +}; + +struct jffs2_sum_inode_flash +{ + __u16 nodetype; /* node type */ + __u32 inode; /* inode number */ + __u32 version; /* inode version */ + __u32 offset; /* offset on jeb */ + __u32 totlen; /* record length */ +} __attribute__((packed)); + +struct jffs2_sum_dirent_flash +{ + __u16 nodetype; /* == JFFS_NODETYPE_DIRENT */ + __u32 totlen; /* record length */ + __u32 offset; /* offset on jeb */ + __u32 pino; /* parent inode */ + __u32 version; /* dirent version */ + __u32 ino; /* == zero for unlink */ + uint8_t nsize; /* dirent name size */ + uint8_t type; /* dirent type */ + uint8_t name[0]; /* dirent name */ +} __attribute__((packed)); + +struct jffs2_sum_xattr_flash +{ + __u16 nodetype; /* == JFFS2_NODETYPE_XATR */ + __u32 xid; /* xattr identifier */ + __u32 version; /* version number */ + __u32 offset; /* offset on jeb */ + __u32 totlen; /* node length */ +} __attribute__((packed)); + +struct jffs2_sum_xref_flash +{ + __u16 nodetype; /* == JFFS2_NODETYPE_XREF */ + __u32 offset; /* offset on jeb */ +} __attribute__((packed)); + +union jffs2_sum_flash +{ + struct jffs2_sum_unknown_flash u; + struct jffs2_sum_inode_flash i; + struct jffs2_sum_dirent_flash d; + struct jffs2_sum_xattr_flash x; + struct jffs2_sum_xref_flash r; +}; + +/* Summary structures used in the memory */ + +struct jffs2_sum_unknown_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; /* node type */ +}; + +struct jffs2_sum_inode_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; /* node type */ + __u32 inode; /* inode number */ + __u32 version; /* inode version */ + __u32 offset; /* offset on jeb */ + __u32 totlen; /* record length */ +} __attribute__((packed)); + +struct jffs2_sum_dirent_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; /* == JFFS_NODETYPE_DIRENT */ + __u32 totlen; /* record length */ + __u32 offset; /* ofset on jeb */ + __u32 pino; /* parent inode */ + __u32 version; /* dirent version */ + __u32 ino; /* == zero for unlink */ + uint8_t nsize; /* dirent name size */ + uint8_t type; /* dirent type */ + uint8_t name[0]; /* dirent name */ +} __attribute__((packed)); + +struct jffs2_sum_xattr_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; + __u32 xid; + __u32 version; + __u32 offset; + __u32 totlen; +} __attribute__((packed)); + +struct jffs2_sum_xref_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; + __u32 offset; +} __attribute__((packed)); + +union jffs2_sum_mem +{ + struct jffs2_sum_unknown_mem u; + struct jffs2_sum_inode_mem i; + struct jffs2_sum_dirent_mem d; + struct jffs2_sum_xattr_mem x; + struct jffs2_sum_xref_mem r; +}; + +/* Summary related information stored in superblock */ + +struct jffs2_summary +{ + uint32_t sum_size; /* collected summary information for nextblock */ + uint32_t sum_num; + uint32_t sum_padded; + union jffs2_sum_mem *sum_list_head; + union jffs2_sum_mem *sum_list_tail; + + __u32 *sum_buf; /* buffer for writing out summary */ +}; + +/* Summary marker is stored at the end of every sumarized erase block */ + +struct jffs2_sum_marker +{ + __u32 offset; /* offset of the summary node in the jeb */ + __u32 magic; /* == JFFS2_SUM_MAGIC */ +}; + +#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_raw_summary) + sizeof(struct jffs2_sum_marker)) + +#endif /* JFFS2_SUMMARY_H */ diff --git a/include/jffs2/jffs2.h b/include/jffs2/jffs2.h index d142cd1..ed96bab 100644 --- a/include/jffs2/jffs2.h +++ b/include/jffs2/jffs2.h @@ -50,6 +50,9 @@ #define JFFS2_EMPTY_BITMASK 0xffff #define JFFS2_DIRTY_BITMASK 0x0000
+/* Summary node MAGIC marker */ +#define JFFS2_SUM_MAGIC 0x02851885 + /* We only allow a single char for length, and 0xFF is empty flash so we don't want it confused with a real length. Hence max 254. */ @@ -89,6 +92,7 @@ #define JFFS2_NODETYPE_INODE (JFFS2_FEATURE_INCOMPAT | JFFS2_NODE_ACCURATE | 2) #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4) +#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
/* Maybe later... */ /*#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) */ @@ -166,9 +170,24 @@ struct jffs2_raw_inode /* __u8 data[dsize]; */ } __attribute__((packed));
+struct jffs2_raw_summary +{ + __u16 magic; + __u16 nodetype; /* = JFFS2_NODETYPE_SUMMARY */ + __u32 totlen; + __u32 hdr_crc; + __u32 sum_num; /* number of sum entries*/ + __u32 cln_mkr; /* clean marker size, 0 = no cleanmarker */ + __u32 padded; /* sum of the size of padding nodes */ + __u32 sum_crc; /* summary information crc */ + __u32 node_crc; /* node crc */ + __u32 sum[0]; /* inode summary info */ +}; + union jffs2_node_union { struct jffs2_raw_inode i; struct jffs2_raw_dirent d; + struct jffs2_raw_summary s; struct jffs2_unknown_node u; } __attribute__((packed));
Dear Ilya Yanok,
In message 1226594976-6087-6-git-send-email-yanok@emcraft.com you wrote:
This patch adds support for reading fs information from summary node instead of scanning full eraseblock.
Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++- fs/jffs2/summary.h | 163 +++++++++++++++++++++++++++++++++++++++++ include/jffs2/jffs2.h | 19 +++++ 3 files changed, 368 insertions(+), 1 deletions(-) create mode 100644 fs/jffs2/summary.h
Applied, thanks.
Best regards,
Wolfgang Denk
As we moved data_crc() invocation from jffs2_1pass_build_lists() to jffs2_1pass_read_inode() data_crc is going to be calculated on each inode access. This patch adds caching of data_crc() results. There is no significant improvement in speed (because of flash access caching added in previous patch I think, crc in RAM is really fast) but this patch impacts memory usage -- every b_node structure uses 12 bytes instead of 8.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 5 ++++- fs/jffs2/jffs2_private.h | 1 + 2 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 4e49a05..be7c1a1 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -765,7 +765,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) put_fl_mem(jNode, pL->readbuf); continue; } - if (!data_crc(jNode)) { + if (b->datacrc == CRC_UNKNOWN) + b->datacrc = data_crc(jNode) ? + CRC_OK : CRC_BAD; + if (b->datacrc == CRC_BAD) { put_fl_mem(jNode, pL->readbuf); continue; } diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h index 3633dea..658b325 100644 --- a/fs/jffs2/jffs2_private.h +++ b/fs/jffs2/jffs2_private.h @@ -7,6 +7,7 @@ struct b_node { u32 offset; struct b_node *next; + enum { CRC_UNKNOWN = 0, CRC_OK, CRC_BAD } datacrc; };
struct b_list {
Dear Ilya Yanok,
In message 1226594976-6087-7-git-send-email-yanok@emcraft.com you wrote:
As we moved data_crc() invocation from jffs2_1pass_build_lists() to jffs2_1pass_read_inode() data_crc is going to be calculated on each inode access. This patch adds caching of data_crc() results. There is no significant improvement in speed (because of flash access caching added in previous patch I think, crc in RAM is really fast) but this patch impacts memory usage -- every b_node structure uses 12 bytes instead of 8.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 5 ++++- fs/jffs2/jffs2_private.h | 1 + 2 files changed, 5 insertions(+), 1 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
participants (5)
-
Ben Warren -
Ilya Yanok -
Jean-Christophe PLAGNIOL-VILLARD -
Stefan Roese -
Wolfgang Denk