[U-Boot] [PATCH/RFC] Some speed improvements to U-Boot JFFS2 code
Hello everybody,
here is a set of changes we made to improve U-Boot JFFS2 code performance. We still can't reach Linux's performance but improvements are significant.
Any comments are welcome.
Regards, Ilya.
This patch adds sector_size field to part_info structure (used by new JFFS2 code).
Signed-off-by: Ilya Yanok yanok@emcraft.com --- common/cmd_jffs2.c | 13 ++++++++++++- include/jffs2/load_kernel.h | 1 + 2 files changed, 13 insertions(+), 1 deletions(-)
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index c6920c9..e7f07bf 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -339,11 +339,13 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) extern flash_info_t flash_info[]; flash_info_t *flash; int offset_aligned; - u32 end_offset; + u32 end_offset, sector_size = 0; int i;
flash = &flash_info[id->num];
+ part->sector_size = 0; + offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) { if ((flash->start[i] - flash->start[0]) == part->offset) { @@ -359,6 +361,11 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
end_offset = part->offset + part->size; for (i = 0; i < flash->sector_count; i++) { + if (i) { + sector_size = flash->start[i] - flash->start[i-1]; + if (part->sector_size < sector_size) + part->sector_size = sector_size; + } if ((flash->start[i] - flash->start[0]) == end_offset) return 0; } @@ -389,6 +396,8 @@ static int part_validate_nand(struct mtdids *id, struct part_info *part)
nand = &nand_info[id->num];
+ part->sector_size = nand->erasesize; + if ((unsigned long)(part->offset) % nand->erasesize) { printf("%s%d: partition (%s) start offset alignment incorrect\n", MTD_DEV_TYPE(id->type), id->num, part->name); @@ -424,6 +433,8 @@ static int part_validate_onenand(struct mtdids *id, struct part_info *part)
mtd = &onenand_mtd;
+ part->sector_size = mtd->erasesize; + if ((unsigned long)(part->offset) % mtd->erasesize) { printf("%s%d: partition (%s) start offset" "alignment incorrect\n", diff --git a/include/jffs2/load_kernel.h b/include/jffs2/load_kernel.h index 551fd0c..c0442a2 100644 --- a/include/jffs2/load_kernel.h +++ b/include/jffs2/load_kernel.h @@ -50,6 +50,7 @@ struct part_info { u32 offset; /* offset within device */ void *jffs2_priv; /* used internaly by jffs2 */ u32 mask_flags; /* kernel MTD mask flags */ + u32 sector_size; /* size of sector */ struct mtd_device *dev; /* parent device */ };
Dear Ilya,
In message 1223877234-720-2-git-send-email-yanok@emcraft.com you wrote:
This patch adds sector_size field to part_info structure (used by new JFFS2 code).
...
@@ -359,6 +361,11 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
end_offset = part->offset + part->size; for (i = 0; i < flash->sector_count; i++) {
if (i) {sector_size = flash->start[i] - flash->start[i-1];if (part->sector_size < sector_size)part->sector_size = sector_size;}
What about the size of the last sector?
Best regards,
Wolfgang Denk
Hi Wolfgang,
Wolfgang Denk wrote:
@@ -359,6 +361,11 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
end_offset = part->offset + part->size; for (i = 0; i < flash->sector_count; i++) {
if (i) {sector_size = flash->start[i] - flash->start[i-1];if (part->sector_size < sector_size)part->sector_size = sector_size;}What about the size of the last sector?
Well, you are right, I've lost last sector... I'll fix this.
Regards, Ilya.
Rewrites jffs2_1pass_build_lists() function in style of Linux's jffs2_scan_medium() and jffs2_scan_eraseblock(). This includes: - Caching flash acceses - Smart dealing with free space
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 227 ++++++++++++++++++++++++++++++++++++------------ 1 files changed, 171 insertions(+), 56 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 57f5582..61a0459 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -389,6 +389,12 @@ static inline void *get_fl_mem_nor(u32 off) return (void*)addr; }
+static inline void *get_fl_mem_nor_copy(u32 off, u32 size, void *ext_buf) +{ + memcpy(ext_buf, get_fl_mem_nor(off), size); + return ext_buf; +} + static inline void *get_node_mem_nor(u32 off) { return (void*)get_fl_mem_nor(off); @@ -405,8 +411,11 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf) struct mtdids *id = current_part->dev->id;
#if defined(CONFIG_CMD_FLASH) - if (id->type == MTD_DEV_TYPE_NOR) + if (id->type == MTD_DEV_TYPE_NOR) { + if (ext_buf) + return get_fl_mem_nor_copy(off, size, ext_buf); return get_fl_mem_nor(off); + } #endif
#if defined(CONFIG_JFFS2_NAND) && defined(CONFIG_CMD_NAND) @@ -480,9 +489,6 @@ static char *compr_names[] = { #endif };
-/* Spinning wheel */ -static char spinner[] = { '|', '/', '-', '\' }; - /* Memory management */ struct mem_block { u32 index; @@ -653,23 +659,6 @@ static int compare_dirents(struct b_node *new, struct b_node *old) } #endif
-static u32 -jffs2_scan_empty(u32 start_offset, struct part_info *part) -{ - char *max = (char *)(part->offset + part->size - sizeof(struct jffs2_raw_inode)); - char *offset = (char *)(part->offset + start_offset); - u32 off; - - while (offset < max && - *(u32*)get_fl_mem((u32)offset, sizeof(u32), &off) == 0xFFFFFFFF) { - offset += sizeof(u32); - /* return if spinning is due */ - if (((u32)offset & ((1 << SPIN_BLKSIZE)-1)) == 0) break; - } - - return (u32)offset - part->offset; -} - void jffs2_free_cache(struct part_info *part) { @@ -768,6 +757,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) put_fl_mem(jNode); continue; } + if (!data_crc(jNode)) { + put_fl_mem(jNode); + continue; + }
lDest = (uchar *) (dest + jNode->offset); #if 0 @@ -1271,17 +1264,33 @@ dump_dirents(struct b_lists *pL) } #endif
+#define min_t(type, x, y) ({ \ + type __min1 = (x); \ + type __min2 = (y); \ + __min1 < __min2 ? __min1: __min2; }) + +#define DEFAULT_EMPTY_SCAN_SIZE 4096 + +static inline uint32_t EMPTY_SCAN_SIZE(uint32_t sector_size) +{ + if (sector_size < DEFAULT_EMPTY_SCAN_SIZE) + return sector_size; + else + return DEFAULT_EMPTY_SCAN_SIZE; +} + static u32 jffs2_1pass_build_lists(struct part_info * part) { struct b_lists *pL; struct jffs2_unknown_node *node; - u32 offset, oldoffset = 0; - u32 max = part->size - sizeof(struct jffs2_raw_inode); - u32 counter = 0; + u32 nr_sectors = part->size/part->sector_size; + u32 i; u32 counter4 = 0; u32 counterF = 0; u32 counterN = 0; + u32 buf_size = DEFAULT_EMPTY_SCAN_SIZE; + char *buf;
/* turn off the lcd. Refreshing the lcd adds 50% overhead to the */ /* jffs2 list building enterprise nope. in newer versions the overhead is */ @@ -1291,70 +1300,176 @@ jffs2_1pass_build_lists(struct part_info * part) /* if we are building a list we need to refresh the cache. */ jffs_init_1pass_list(part); pL = (struct b_lists *)part->jffs2_priv; - offset = 0; + buf = malloc(buf_size); puts ("Scanning JFFS2 FS: ");
/* start at the beginning of the partition */ - while (offset < max) { - if ((oldoffset >> SPIN_BLKSIZE) != (offset >> SPIN_BLKSIZE)) { - printf("\b\b%c ", spinner[counter++ % sizeof(spinner)]); - oldoffset = offset; - } + for (i = 0; i < nr_sectors; i++) { + uint32_t sector_ofs = i * part->sector_size; + uint32_t buf_ofs = sector_ofs; + uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size); + uint32_t ofs, prevofs;
WATCHDOG_RESET(); + get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
- node = (struct jffs2_unknown_node *) get_node_mem((u32)part->offset + offset); - if (node->magic == JFFS2_MAGIC_BITMASK && hdr_crc(node)) { + /* We temporarily use 'ofs' as a pointer into the buffer/jeb */ + ofs = 0; + + /* Scan only 4KiB of 0xFF before declaring it's empty */ + while (ofs < EMPTY_SCAN_SIZE(part->sector_size) && + *(uint32_t *)(&buf[ofs]) == 0xFFFFFFFF) + ofs += 4; + + if (ofs == EMPTY_SCAN_SIZE(part->sector_size)) + continue; + + ofs += sector_ofs; + prevofs = ofs - 1; + + scan_more: + while (ofs < sector_ofs + part->sector_size) { + if (ofs == prevofs) { + printf("offset %08x already seen, skip\n", ofs); + ofs += 4; + counter4++; + continue; + } + prevofs = ofs; + if (sector_ofs + part->sector_size < + ofs + sizeof(*node)) + break; + if (buf_ofs + buf_len < ofs + sizeof(*node)) { + buf_len = min_t(uint32_t, buf_size, sector_ofs + + part->sector_size - ofs); + get_fl_mem((u32)part->offset + ofs, buf_len, + buf); + buf_ofs = ofs; + } + + node = (struct jffs2_unknown_node *)&buf[ofs-buf_ofs]; + + if (*(uint32_t *)(&buf[ofs-buf_ofs]) == 0xffffffff) { + uint32_t inbuf_ofs; + uint32_t empty_start, scan_end; + + empty_start = ofs; + ofs += 4; + scan_end = min_t(uint32_t, EMPTY_SCAN_SIZE( + part->sector_size)/8, + buf_len); + more_empty: + inbuf_ofs = ofs - buf_ofs; + while (inbuf_ofs < scan_end) { + if (*(uint32_t *)(&buf[inbuf_ofs]) != + 0xffffffff) + goto scan_more; + + inbuf_ofs += 4; + ofs += 4; + } + /* Ran off end. */ + + /* See how much more there is to read in this + * eraseblock... + */ + buf_len = min_t(uint32_t, buf_size, + sector_ofs + + part->sector_size - ofs); + if (!buf_len) { + /* No more to read. Break out of main + * loop without marking this range of + * empty space as dirty (because it's + * not) + */ + break; + } + scan_end = buf_len; + get_fl_mem((u32)part->offset + ofs, buf_len, + buf); + buf_ofs = ofs; + goto more_empty; + } + if (node->magic != JFFS2_MAGIC_BITMASK || + !hdr_crc(node)) { + ofs += 4; + counter4++; + continue; + } + if (ofs + node->totlen > + sector_ofs + part->sector_size) { + ofs += 4; + counter4++; + continue; + } /* if its a fragment add it */ - if (node->nodetype == JFFS2_NODETYPE_INODE && - inode_crc((struct jffs2_raw_inode *) node) && - data_crc((struct jffs2_raw_inode *) node)) { + switch (node->nodetype) { + case JFFS2_NODETYPE_INODE: + if (buf_ofs + buf_len < ofs + sizeof(struct + jffs2_raw_inode)) { + get_fl_mem((u32)part->offset + ofs, + buf_len, buf); + buf_ofs = ofs; + node = (void *)buf; + } + if (!inode_crc((struct jffs2_raw_inode *) node)) + break; + if (insert_node(&pL->frag, (u32) part->offset + - offset) == NULL) { - put_fl_mem(node); + ofs) == NULL) return 0; + break; + case JFFS2_NODETYPE_DIRENT: + if (buf_ofs + buf_len < ofs + sizeof(struct + jffs2_raw_dirent) + + ((struct + jffs2_raw_dirent *) + node)->nsize) { + get_fl_mem((u32)part->offset + ofs, + buf_len, buf); + buf_ofs = ofs; + node = (void *)buf; } - } else if (node->nodetype == JFFS2_NODETYPE_DIRENT && - dirent_crc((struct jffs2_raw_dirent *) node) && - dirent_name_crc((struct jffs2_raw_dirent *) node)) { + + if (!dirent_crc((struct jffs2_raw_dirent *) + node) || + !dirent_name_crc( + (struct + jffs2_raw_dirent *) + node)) + break; if (! (counterN%100)) puts ("\b\b. "); if (insert_node(&pL->dir, (u32) part->offset + - offset) == NULL) { - put_fl_mem(node); + ofs) == NULL) return 0; - } counterN++; - } else if (node->nodetype == JFFS2_NODETYPE_CLEANMARKER) { + break; + case JFFS2_NODETYPE_CLEANMARKER: if (node->totlen != sizeof(struct jffs2_unknown_node)) printf("OOPS Cleanmarker has bad size " "%d != %zu\n", node->totlen, sizeof(struct jffs2_unknown_node)); - } else if (node->nodetype == JFFS2_NODETYPE_PADDING) { + break; + case JFFS2_NODETYPE_PADDING: if (node->totlen < sizeof(struct jffs2_unknown_node)) printf("OOPS Padding has bad size " "%d < %zu\n", node->totlen, sizeof(struct jffs2_unknown_node)); - } else { + break; + default: printf("Unknown node type: %x len %d offset 0x%x\n", node->nodetype, - node->totlen, offset); + node->totlen, ofs); } - offset += ((node->totlen + 3) & ~3); + ofs += ((node->totlen + 3) & ~3); counterF++; - } else if (node->magic == JFFS2_EMPTY_BITMASK && - node->nodetype == JFFS2_EMPTY_BITMASK) { - offset = jffs2_scan_empty(offset, part); - } else { /* if we know nothing, we just step and look. */ - offset += 4; - counter4++; } -/* printf("unknown node magic %4.4x %4.4x @ %lx\n", node->magic, node->nodetype, (unsigned long)node); */ - put_fl_mem(node); }
+ free(buf); putstr("\b\b done.\r\n"); /* close off the dots */ /* turn the lcd back on. */ /* splash(); */
This patch adds support for reading fs information from summary node instead of scanning full eraseblock.
Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++- fs/jffs2/summary.h | 163 +++++++++++++++++++++++++++++++++++++++++ include/jffs2/jffs2.h | 19 +++++ 3 files changed, 368 insertions(+), 1 deletions(-) create mode 100644 fs/jffs2/summary.h
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 61a0459..58298c0 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -138,6 +138,8 @@ # define DEBUGF(fmt,args...) #endif
+#include "summary.h" + /* keeps pointer to currentlu processed partition */ static struct part_info *current_part;
@@ -1202,6 +1204,132 @@ jffs2_1pass_rescan_needed(struct part_info *part) return 0; }
+#define dbg_summary(...) do {} while (0); +/* Process the stored summary information - helper function for + * jffs2_sum_scan_sumnode() + */ + +static int jffs2_sum_process_sum_data(struct part_info *part, uint32_t offset, + struct jffs2_raw_summary *summary, + struct b_lists *pL) +{ + void *sp; + int i; + + sp = summary->sum; + + for (i = 0; i < summary->sum_num; i++) { + dbg_summary("processing summary index %d\n", i); + + switch (((struct jffs2_sum_unknown_flash *)sp)->nodetype) { + case JFFS2_NODETYPE_INODE: { + struct jffs2_sum_inode_flash *spi; + spi = sp; + + dbg_summary("Inode at 0x%08x-0x%08x\n", + offset + spi->offset, + offset + spi->offset + spi->totlen); + + if (insert_node(&pL->frag, (u32) part->offset + + offset + spi->offset) == NULL) + return -1; + + sp += JFFS2_SUMMARY_INODE_SIZE; + + break; + } + + case JFFS2_NODETYPE_DIRENT: { + struct jffs2_sum_dirent_flash *spd; + spd = sp; + + dbg_summary("Dirent at 0x%08x-0x%08x\n", + offset + spd->offset, + offset + spd->offset + spd->totlen); + + if (insert_node(&pL->dir, (u32) part->offset + + offset + spd->offset) == NULL) + return -1; + + sp += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); + + break; + } + default : { + uint16_t nodetype = + ((struct jffs2_sum_unknown_flash *) + sp)->nodetype; + printf("Unsupported node type %x found in " + "summary!\n", nodetype); + break; + } + } + } + return 0; +} + +/* Process the summary node - called from jffs2_scan_eraseblock() */ +int jffs2_sum_scan_sumnode(struct part_info *part, uint32_t offset, + struct jffs2_raw_summary *summary, uint32_t sumsize, + struct b_lists *pL) +{ + struct jffs2_unknown_node crcnode; + int ret, ofs; + uint32_t crc; + + ofs = part->sector_size - sumsize; + + dbg_summary("summary found for 0x%08x at 0x%08x (0x%x bytes)\n", + offset, offset + ofs, sumsize); + + /* OK, now check for node validity and CRC */ + crcnode.magic = JFFS2_MAGIC_BITMASK; + crcnode.nodetype = JFFS2_NODETYPE_SUMMARY; + crcnode.totlen = summary->totlen; + crc = crc32_no_comp(0, (uchar *)&crcnode, sizeof(crcnode)-4); + + if (summary->hdr_crc != crc) { + dbg_summary("Summary node header is corrupt (bad CRC or " + "no summary at all)\n"); + goto crc_err; + } + + if (summary->totlen != sumsize) { + dbg_summary("Summary node is corrupt (wrong erasesize?)\n"); + goto crc_err; + } + + crc = crc32_no_comp(0, (uchar *)summary, + sizeof(struct jffs2_raw_summary)-8); + + if (summary->node_crc != crc) { + dbg_summary("Summary node is corrupt (bad CRC)\n"); + goto crc_err; + } + + crc = crc32_no_comp(0, (uchar *)summary->sum, + sumsize - sizeof(struct jffs2_raw_summary)); + + if (summary->sum_crc != crc) { + dbg_summary("Summary node data is corrupt (bad CRC)\n"); + goto crc_err; + } + + if (summary->cln_mkr) + dbg_summary("Summary : CLEANMARKER node \n"); + + ret = jffs2_sum_process_sum_data(part, offset, summary, pL); + if (ret) + return ret; /* real error */ + + return 1; + +crc_err: + putstr("Summary node crc error, skipping summary information.\n"); + + return 0; +} + #ifdef DEBUG_FRAGMENTS static void dump_fragments(struct b_lists *pL) @@ -1307,10 +1435,65 @@ jffs2_1pass_build_lists(struct part_info * part) for (i = 0; i < nr_sectors; i++) { uint32_t sector_ofs = i * part->sector_size; uint32_t buf_ofs = sector_ofs; - uint32_t buf_len = EMPTY_SCAN_SIZE(part->sector_size); + uint32_t buf_len; uint32_t ofs, prevofs; + struct jffs2_sum_marker *sm; + void *sumptr = NULL; + uint32_t sumlen; + int ret;
WATCHDOG_RESET(); + + buf_len = sizeof(*sm); + + /* Read as much as we want into the _end_ of the preallocated + * buffer + */ + get_fl_mem(part->offset + sector_ofs + part->sector_size - + buf_len, buf_len, buf + buf_size - buf_len); + + sm = (void *)buf + buf_size - sizeof(*sm); + if (sm->magic == JFFS2_SUM_MAGIC) { + sumlen = part->sector_size - sm->offset; + sumptr = buf + buf_size - sumlen; + + /* Now, make sure the summary itself is available */ + if (sumlen > buf_size) { + /* Need to kmalloc for this. */ + sumptr = malloc(sumlen); + if (!sumptr) { + putstr("Can't get memory for summary " + "node!\n"); + return 0; + } + memcpy(sumptr + sumlen - buf_len, buf + + buf_size - buf_len, buf_len); + } + if (buf_len < sumlen) { + /* Need to read more so that the entire summary + * node is present + */ + get_fl_mem(part->offset + sector_ofs + + part->sector_size - sumlen, + sumlen - buf_len, sumptr); + } + } + + if (sumptr) { + ret = jffs2_sum_scan_sumnode(part, sector_ofs, sumptr, + sumlen, pL); + + if (buf_size && sumlen > buf_size) + free(sumptr); + if (ret < 0) + return 0; + if (ret) + continue; + + } + + buf_len = EMPTY_SCAN_SIZE(part->sector_size); + get_fl_mem((u32)part->offset + buf_ofs, buf_len, buf);
/* We temporarily use 'ofs' as a pointer into the buffer/jeb */ @@ -1459,6 +1642,8 @@ jffs2_1pass_build_lists(struct part_info * part) node->totlen, sizeof(struct jffs2_unknown_node)); break; + case JFFS2_NODETYPE_SUMMARY: + break; default: printf("Unknown node type: %x len %d offset 0x%x\n", node->nodetype, diff --git a/fs/jffs2/summary.h b/fs/jffs2/summary.h new file mode 100644 index 0000000..834933c --- /dev/null +++ b/fs/jffs2/summary.h @@ -0,0 +1,163 @@ +/* + * JFFS2 -- Journalling Flash File System, Version 2. + * + * Copyright © 2004 Ferenc Havasi havasi@inf.u-szeged.hu, + * Zoltan Sogor weth@inf.u-szeged.hu, + * Patrik Kluba pajko@halom.u-szeged.hu, + * University of Szeged, Hungary + * + * For licensing information, see the file 'LICENCE' in this directory. + * + */ + +#ifndef JFFS2_SUMMARY_H +#define JFFS2_SUMMARY_H + +#define BLK_STATE_ALLFF 0 +#define BLK_STATE_CLEAN 1 +#define BLK_STATE_PARTDIRTY 2 +#define BLK_STATE_CLEANMARKER 3 +#define BLK_STATE_ALLDIRTY 4 +#define BLK_STATE_BADBLOCK 5 + +#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff +#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash)) +#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x)) +#define JFFS2_SUMMARY_XATTR_SIZE (sizeof(struct jffs2_sum_xattr_flash)) +#define JFFS2_SUMMARY_XREF_SIZE (sizeof(struct jffs2_sum_xref_flash)) + +/* Summary structures used on flash */ + +struct jffs2_sum_unknown_flash +{ + __u16 nodetype; /* node type */ +}; + +struct jffs2_sum_inode_flash +{ + __u16 nodetype; /* node type */ + __u32 inode; /* inode number */ + __u32 version; /* inode version */ + __u32 offset; /* offset on jeb */ + __u32 totlen; /* record length */ +} __attribute__((packed)); + +struct jffs2_sum_dirent_flash +{ + __u16 nodetype; /* == JFFS_NODETYPE_DIRENT */ + __u32 totlen; /* record length */ + __u32 offset; /* offset on jeb */ + __u32 pino; /* parent inode */ + __u32 version; /* dirent version */ + __u32 ino; /* == zero for unlink */ + uint8_t nsize; /* dirent name size */ + uint8_t type; /* dirent type */ + uint8_t name[0]; /* dirent name */ +} __attribute__((packed)); + +struct jffs2_sum_xattr_flash +{ + __u16 nodetype; /* == JFFS2_NODETYPE_XATR */ + __u32 xid; /* xattr identifier */ + __u32 version; /* version number */ + __u32 offset; /* offset on jeb */ + __u32 totlen; /* node length */ +} __attribute__((packed)); + +struct jffs2_sum_xref_flash +{ + __u16 nodetype; /* == JFFS2_NODETYPE_XREF */ + __u32 offset; /* offset on jeb */ +} __attribute__((packed)); + +union jffs2_sum_flash +{ + struct jffs2_sum_unknown_flash u; + struct jffs2_sum_inode_flash i; + struct jffs2_sum_dirent_flash d; + struct jffs2_sum_xattr_flash x; + struct jffs2_sum_xref_flash r; +}; + +/* Summary structures used in the memory */ + +struct jffs2_sum_unknown_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; /* node type */ +}; + +struct jffs2_sum_inode_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; /* node type */ + __u32 inode; /* inode number */ + __u32 version; /* inode version */ + __u32 offset; /* offset on jeb */ + __u32 totlen; /* record length */ +} __attribute__((packed)); + +struct jffs2_sum_dirent_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; /* == JFFS_NODETYPE_DIRENT */ + __u32 totlen; /* record length */ + __u32 offset; /* ofset on jeb */ + __u32 pino; /* parent inode */ + __u32 version; /* dirent version */ + __u32 ino; /* == zero for unlink */ + uint8_t nsize; /* dirent name size */ + uint8_t type; /* dirent type */ + uint8_t name[0]; /* dirent name */ +} __attribute__((packed)); + +struct jffs2_sum_xattr_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; + __u32 xid; + __u32 version; + __u32 offset; + __u32 totlen; +} __attribute__((packed)); + +struct jffs2_sum_xref_mem +{ + union jffs2_sum_mem *next; + __u16 nodetype; + __u32 offset; +} __attribute__((packed)); + +union jffs2_sum_mem +{ + struct jffs2_sum_unknown_mem u; + struct jffs2_sum_inode_mem i; + struct jffs2_sum_dirent_mem d; + struct jffs2_sum_xattr_mem x; + struct jffs2_sum_xref_mem r; +}; + +/* Summary related information stored in superblock */ + +struct jffs2_summary +{ + uint32_t sum_size; /* collected summary information for nextblock */ + uint32_t sum_num; + uint32_t sum_padded; + union jffs2_sum_mem *sum_list_head; + union jffs2_sum_mem *sum_list_tail; + + __u32 *sum_buf; /* buffer for writing out summary */ +}; + +/* Summary marker is stored at the end of every sumarized erase block */ + +struct jffs2_sum_marker +{ + __u32 offset; /* offset of the summary node in the jeb */ + __u32 magic; /* == JFFS2_SUM_MAGIC */ +}; + +#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_raw_summary) + sizeof(struct jffs2_sum_marker)) + +#endif /* JFFS2_SUMMARY_H */ diff --git a/include/jffs2/jffs2.h b/include/jffs2/jffs2.h index d142cd1..ed96bab 100644 --- a/include/jffs2/jffs2.h +++ b/include/jffs2/jffs2.h @@ -50,6 +50,9 @@ #define JFFS2_EMPTY_BITMASK 0xffff #define JFFS2_DIRTY_BITMASK 0x0000
+/* Summary node MAGIC marker */ +#define JFFS2_SUM_MAGIC 0x02851885 + /* We only allow a single char for length, and 0xFF is empty flash so we don't want it confused with a real length. Hence max 254. */ @@ -89,6 +92,7 @@ #define JFFS2_NODETYPE_INODE (JFFS2_FEATURE_INCOMPAT | JFFS2_NODE_ACCURATE | 2) #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4) +#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
/* Maybe later... */ /*#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) */ @@ -166,9 +170,24 @@ struct jffs2_raw_inode /* __u8 data[dsize]; */ } __attribute__((packed));
+struct jffs2_raw_summary +{ + __u16 magic; + __u16 nodetype; /* = JFFS2_NODETYPE_SUMMARY */ + __u32 totlen; + __u32 hdr_crc; + __u32 sum_num; /* number of sum entries*/ + __u32 cln_mkr; /* clean marker size, 0 = no cleanmarker */ + __u32 padded; /* sum of the size of padding nodes */ + __u32 sum_crc; /* summary information crc */ + __u32 node_crc; /* node crc */ + __u32 sum[0]; /* inode summary info */ +}; + union jffs2_node_union { struct jffs2_raw_inode i; struct jffs2_raw_dirent d; + struct jffs2_raw_summary s; struct jffs2_unknown_node u; } __attribute__((packed));
We need to update i_version inside cycle to find really latest version inside jffs2_1pass_list_inodes(). With that fixed we can use isize inside dump_inode() instead of calling expensive jffs2_1pass_read_inode().
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 58298c0..94f8468 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -937,9 +937,7 @@ static inline u32 dump_inode(struct b_lists * pL, struct jffs2_raw_dirent *d, st st.st_mtime = i->mtime; st.st_mode = i->mode; st.st_ino = i->ino; - - /* neither dsize nor isize help us.. do it the long way */ - st.st_size = jffs2_1pass_read_inode(pL, i->ino, NULL); + st.st_size = i->isize;
dump_stat(&st, fname);
@@ -973,6 +971,7 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino) jNode = (struct jffs2_raw_inode *) get_fl_mem(b2->offset, sizeof(ojNode), &ojNode); if (jNode->ino == jDir->ino && jNode->version >= i_version) { + i_version = jNode->version; if (i) put_fl_mem(i);
With this patch JFFS2 code allocates memory buffer of max_totlen size (size of the largest node, calculated during scan time) and uses it to store entire node. Speeds up loading. If malloc fails we use old ways to do things.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 121 ++++++++++++++++++++++++++++----------------- fs/jffs2/jffs2_private.h | 2 +- 2 files changed, 76 insertions(+), 47 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 94f8468..9d9ac1a 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -247,7 +247,7 @@ static void *get_fl_mem_nand(u32 off, u32 size, void *ext_buf) return buf; }
-static void *get_node_mem_nand(u32 off) +static void *get_node_mem_nand(u32 off, void *ext_buf) { struct jffs2_unknown_node node; void *ret = NULL; @@ -257,7 +257,7 @@ static void *get_node_mem_nand(u32 off)
if (!(ret = get_fl_mem_nand(off, node.magic == JFFS2_MAGIC_BITMASK ? node.totlen : sizeof(node), - NULL))) { + ext_buf))) { printf("off = %#x magic %#x type %#x node.totlen = %d\n", off, node.magic, node.nodetype, node.totlen); } @@ -346,7 +346,7 @@ static void *get_fl_mem_onenand(u32 off, u32 size, void *ext_buf) return buf; }
-static void *get_node_mem_onenand(u32 off) +static void *get_node_mem_onenand(u32 off, void *ext_buf) { struct jffs2_unknown_node node; void *ret = NULL; @@ -356,7 +356,7 @@ static void *get_node_mem_onenand(u32 off)
ret = get_fl_mem_onenand(off, node.magic == JFFS2_MAGIC_BITMASK ? node.totlen : sizeof(node), - NULL); + ext_buf); if (!ret) { printf("off = %#x magic %#x type %#x node.totlen = %d\n", off, node.magic, node.nodetype, node.totlen); @@ -379,7 +379,7 @@ static void put_fl_mem_onenand(void *buf) * NOR flash memory is mapped in processor's address space, * just return address. */ -static inline void *get_fl_mem_nor(u32 off) +static inline void *get_fl_mem_nor(u32 off, u32 size, void *ext_buf) { u32 addr = off; struct mtdids *id = current_part->dev->id; @@ -388,18 +388,22 @@ static inline void *get_fl_mem_nor(u32 off) flash_info_t *flash = &flash_info[id->num];
addr += flash->start[0]; + if (ext_buf) { + memcpy(ext_buf, (void *)addr, size); + return ext_buf; + } return (void*)addr; }
-static inline void *get_fl_mem_nor_copy(u32 off, u32 size, void *ext_buf) +static inline void *get_node_mem_nor(u32 off, void *ext_buf) { - memcpy(ext_buf, get_fl_mem_nor(off), size); - return ext_buf; -} + struct jffs2_unknown_node *pNode;
-static inline void *get_node_mem_nor(u32 off) -{ - return (void*)get_fl_mem_nor(off); + /* pNode will point directly to flash - don't provide external buffer + and don't care about size */ + pNode = get_fl_mem_nor(off, 0, NULL); + return (void *)get_fl_mem_nor(off, pNode->magic == JFFS2_MAGIC_BITMASK ? + pNode->totlen : sizeof(*pNode), ext_buf); } #endif
@@ -414,9 +418,7 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf)
#if defined(CONFIG_CMD_FLASH) if (id->type == MTD_DEV_TYPE_NOR) { - if (ext_buf) - return get_fl_mem_nor_copy(off, size, ext_buf); - return get_fl_mem_nor(off); + return get_fl_mem_nor(off, size, ext_buf); } #endif
@@ -434,44 +436,48 @@ static inline void *get_fl_mem(u32 off, u32 size, void *ext_buf) return (void*)off; }
-static inline void *get_node_mem(u32 off) +static inline void *get_node_mem(u32 off, void *ext_buf) { struct mtdids *id = current_part->dev->id;
#if defined(CONFIG_CMD_FLASH) if (id->type == MTD_DEV_TYPE_NOR) - return get_node_mem_nor(off); + return get_node_mem_nor(off, ext_buf); #endif
#if defined(CONFIG_JFFS2_NAND) && \ defined(CONFIG_CMD_NAND) if (id->type == MTD_DEV_TYPE_NAND) - return get_node_mem_nand(off); + return get_node_mem_nand(off, ext_buf); #endif
#if defined(CONFIG_CMD_ONENAND) if (id->type == MTD_DEV_TYPE_ONENAND) - return get_node_mem_onenand(off); + return get_node_mem_onenand(off, ext_buf); #endif
printf("get_node_mem: unknown device type, using raw offset!\n"); return (void*)off; }
-static inline void put_fl_mem(void *buf) +static inline void put_fl_mem(void *buf, void *ext_buf) { #if defined(CONFIG_JFFS2_NAND) && \ defined(CONFIG_CMD_NAND) struct mtdids *id = current_part->dev->id;
- if (id->type == MTD_DEV_TYPE_NAND) + /* If buf is the same as ext_buf, it was provided by the caller - + we shouldn't free it then. */ + if (id->type == MTD_DEV_TYPE_NAND && buf != ext_buf) return put_fl_mem_nand(buf); #endif
#if defined(CONFIG_CMD_ONENAND) struct mtdids *id = current_part->dev->id;
- if (id->type == MTD_DEV_TYPE_ONENAND) + /* If buf is the same as ext_buf, it was provided by the caller - + we shouldn't free it then. */ + if (id->type == MTD_DEV_TYPE_ONENAND && buf != ext_buf) return put_fl_mem_onenand(buf); #endif } @@ -670,6 +676,7 @@ jffs2_free_cache(struct part_info *part) pL = (struct b_lists *)part->jffs2_priv; free_nodes(&pL->frag); free_nodes(&pL->dir); + free(pL->readbuf); free(pL); } } @@ -716,7 +723,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) */ for (b = pL->frag.listHead; b != NULL; b = b->next) { jNode = (struct jffs2_raw_inode *) get_fl_mem(b->offset, - sizeof(struct jffs2_raw_inode), NULL); + sizeof(struct jffs2_raw_inode), pL->readbuf); if ((inode == jNode->ino)) { /* get actual file length from the newest node */ if (jNode->version >= latestVersion) { @@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) latestVersion = jNode->version; } } - put_fl_mem(jNode); + if (pL->readbuf == NULL) + put_fl_mem(jNode); } #endif
for (b = pL->frag.listHead; b != NULL; b = b->next) { - jNode = (struct jffs2_raw_inode *) get_node_mem(b->offset); + jNode = (struct jffs2_raw_inode *) get_node_mem(b->offset, + pL->readbuf); if ((inode == jNode->ino)) { #if 0 putLabeledWord("\r\n\r\nread_inode: totlen = ", jNode->totlen); @@ -756,11 +765,11 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) src = ((uchar *) jNode) + sizeof(struct jffs2_raw_inode); /* ignore data behind latest known EOF */ if (jNode->offset > totalSize) { - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); continue; } if (!data_crc(jNode)) { - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); continue; }
@@ -801,7 +810,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) default: /* unknown */ putLabeledWord("UNKOWN COMPRESSION METHOD = ", jNode->compr); - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); return -1; break; } @@ -813,7 +822,7 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) #endif } counter++; - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); }
#if 0 @@ -839,12 +848,13 @@ jffs2_1pass_find_inode(struct b_lists * pL, const char *name, u32 pino) counter = 0; /* we need to search all and return the inode with the highest version */ for(b = pL->dir.listHead; b; b = b->next, counter++) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); if ((pino == jDir->pino) && (len == jDir->nsize) && (jDir->ino) && /* 0 for unlink */ (!strncmp((char *)jDir->name, name, len))) { /* a match */ if (jDir->version < version) { - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); continue; }
@@ -866,7 +876,7 @@ jffs2_1pass_find_inode(struct b_lists * pL, const char *name, u32 pino) putLabeledWord("b = ", (u32) b); putLabeledWord("counter = ", counter); #endif - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } return inode; } @@ -960,7 +970,8 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino) struct jffs2_raw_dirent *jDir;
for (b = pL->dir.listHead; b; b = b->next) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); if ((pino == jDir->pino) && (jDir->ino)) { /* ino=0 -> unlink */ u32 i_version = 0; struct jffs2_raw_inode ojNode; @@ -973,20 +984,23 @@ jffs2_1pass_list_inodes(struct b_lists * pL, u32 pino) if (jNode->ino == jDir->ino && jNode->version >= i_version) { i_version = jNode->version; if (i) - put_fl_mem(i); + put_fl_mem(i, NULL);
if (jDir->type == DT_LNK) - i = get_node_mem(b2->offset); + i = get_node_mem(b2->offset, + NULL); else - i = get_fl_mem(b2->offset, sizeof(*i), NULL); + i = get_fl_mem(b2->offset, + sizeof(*i), + NULL); } b2 = b2->next; }
dump_inode(pL, jDir, i); - put_fl_mem(i); + put_fl_mem(i, NULL); } - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } return pino; } @@ -1064,10 +1078,11 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino)
/* we need to search all and return the inode with the highest version */ for(b = pL->dir.listHead; b; b = b->next) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); if (ino == jDir->ino) { if (jDir->version < version) { - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); continue; }
@@ -1084,7 +1099,7 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino) jDirFoundPino = jDir->pino; version = jDir->version; } - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } /* now we found the right entry again. (shoulda returned inode*) */ if (jDirFoundType != DT_LNK) @@ -1093,7 +1108,8 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino) /* it's a soft link so we follow it again. */ b2 = pL->frag.listHead; while (b2) { - jNode = (struct jffs2_raw_inode *) get_node_mem(b2->offset); + jNode = (struct jffs2_raw_inode *) get_node_mem(b2->offset, + pL->readbuf); if (jNode->ino == jDirFoundIno) { src = (unsigned char *)jNode + sizeof(struct jffs2_raw_inode);
@@ -1105,11 +1121,11 @@ jffs2_1pass_resolve_inode(struct b_lists * pL, u32 ino) #endif strncpy(tmp, (char *)src, jNode->dsize); tmp[jNode->dsize] = '\0'; - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); break; } b2 = b2->next; - put_fl_mem(jNode); + put_fl_mem(jNode, pL->readbuf); } /* ok so the name of the new file to find is in tmp */ /* if it starts with a slash it is root based else shared dirs */ @@ -1370,7 +1386,8 @@ dump_dirents(struct b_lists *pL) putstr("\r\n\r\n******The directory Entries******\r\n"); b = pL->dir.listHead; while (b) { - jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset); + jDir = (struct jffs2_raw_dirent *) get_node_mem(b->offset, + pL->readbuf); putstr("\r\n"); putnstr(jDir->name, jDir->nsize); putLabeledWord("\r\n\tbuild_list: magic = ", jDir->magic); @@ -1386,7 +1403,7 @@ dump_dirents(struct b_lists *pL) putLabeledWord("\tbuild_list: name_crc = ", jDir->name_crc); putLabeledWord("\tbuild_list: offset = ", b->offset); /* FIXME: ? [RS] */ b = b->next; - put_fl_mem(jDir); + put_fl_mem(jDir, pL->readbuf); } } #endif @@ -1416,6 +1433,7 @@ jffs2_1pass_build_lists(struct part_info * part) u32 counter4 = 0; u32 counterF = 0; u32 counterN = 0; + u32 max_totlen = 0; u32 buf_size = DEFAULT_EMPTY_SCAN_SIZE; char *buf;
@@ -1600,6 +1618,8 @@ jffs2_1pass_build_lists(struct part_info * part) if (insert_node(&pL->frag, (u32) part->offset + ofs) == NULL) return 0; + if (max_totlen < node->totlen) + max_totlen = node->totlen; break; case JFFS2_NODETYPE_DIRENT: if (buf_ofs + buf_len < ofs + sizeof(struct @@ -1625,6 +1645,8 @@ jffs2_1pass_build_lists(struct part_info * part) if (insert_node(&pL->dir, (u32) part->offset + ofs) == NULL) return 0; + if (max_totlen < node->totlen) + max_totlen = node->totlen; counterN++; break; case JFFS2_NODETYPE_CLEANMARKER: @@ -1655,6 +1677,13 @@ jffs2_1pass_build_lists(struct part_info * part)
free(buf); putstr("\b\b done.\r\n"); /* close off the dots */ + + /* We don't care if malloc failed - then each read operation will + * allocate its own buffer as necessary (NAND) or will read directly + * from flash (NOR). + */ + pL->readbuf = malloc(max_totlen); + /* turn the lcd back on. */ /* splash(); */
diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h index 46ed644..b3fab1c 100644 --- a/fs/jffs2/jffs2_private.h +++ b/fs/jffs2/jffs2_private.h @@ -24,7 +24,7 @@ struct b_list { struct b_lists { struct b_list dir; struct b_list frag; - + void *readbuf; };
struct b_compr_info {
As we moved data_crc() invocation from jffs2_1pass_build_lists() to jffs2_1pass_read_inode() data_crc is going to be calculated on each inode access. This patch adds caching of data_crc() results. There is no significant improvement in speed (because of flash access caching added in previous patch I think, crc in RAM is really fast) but this patch impacts memory usage -- every b_node structure uses 12 bytes instead of 8.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com --- fs/jffs2/jffs2_1pass.c | 5 ++++- fs/jffs2/jffs2_private.h | 1 + 2 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 9d9ac1a..b41c299 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -768,7 +768,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) put_fl_mem(jNode, pL->readbuf); continue; } - if (!data_crc(jNode)) { + if (b->datacrc == CRC_UNKNOWN) + b->datacrc = data_crc(jNode) ? + CRC_OK : CRC_BAD; + if (b->datacrc == CRC_BAD) { put_fl_mem(jNode, pL->readbuf); continue; } diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h index b3fab1c..7a9eda6 100644 --- a/fs/jffs2/jffs2_private.h +++ b/fs/jffs2/jffs2_private.h @@ -7,6 +7,7 @@ struct b_node { u32 offset; struct b_node *next; + enum { CRC_UNKNOWN = 0, CRC_OK, CRC_BAD } datacrc; };
struct b_list {
Ilya Yanok said the following:
As we moved data_crc() invocation from jffs2_1pass_build_lists() to jffs2_1pass_read_inode() data_crc is going to be calculated on each inode access. This patch adds caching of data_crc() results. There is no significant improvement in speed (because of flash access caching added in previous patch I think, crc in RAM is really fast) but this patch impacts memory usage -- every b_node structure uses 12 bytes instead of 8.
Signed-off-by: Alexey Neyman avn@emcraft.com Signed-off-by: Ilya Yanok yanok@emcraft.com
fs/jffs2/jffs2_1pass.c | 5 ++++- fs/jffs2/jffs2_private.h | 1 + 2 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/fs/jffs2/jffs2_1pass.c b/fs/jffs2/jffs2_1pass.c index 9d9ac1a..b41c299 100644 --- a/fs/jffs2/jffs2_1pass.c +++ b/fs/jffs2/jffs2_1pass.c @@ -768,7 +768,10 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) put_fl_mem(jNode, pL->readbuf); continue; }
if (!data_crc(jNode)) {
if (b->datacrc == CRC_UNKNOWN)b->datacrc = data_crc(jNode) ?CRC_OK : CRC_BAD;if (b->datacrc == CRC_BAD) { put_fl_mem(jNode, pL->readbuf); continue; }diff --git a/fs/jffs2/jffs2_private.h b/fs/jffs2/jffs2_private.h index b3fab1c..7a9eda6 100644 --- a/fs/jffs2/jffs2_private.h +++ b/fs/jffs2/jffs2_private.h @@ -7,6 +7,7 @@ struct b_node { u32 offset; struct b_node *next;
- enum { CRC_UNKNOWN = 0, CRC_OK, CRC_BAD } datacrc;
};
struct b_list {
Could be so kind to give us your source reference? I can't find above code in u-boot 1.3.3 nor 1.3.4
The same seems to apply to the other patches you've sent yesterday :-(
I'm eagerly waiting for summary patch ;-)
Regards, Michael
Dear Ilya,
In message 1223877234-720-1-git-send-email-yanok@emcraft.com you wrote:
here is a set of changes we made to improve U-Boot JFFS2 code performance. We still can't reach Linux's performance but improvements are significant.
Any comments are welcome.
Are these patches independent of each other, or are all of them needed, and if so, is there any specific order in which they have to be applied?
Best regards,
Wolfgang Denk
Hi Wolfgang,
Wolfgang Denk wrote:
here is a set of changes we made to improve U-Boot JFFS2 code performance. We still can't reach Linux's performance but improvements are significant.
Any comments are welcome.
Are these patches independent of each other, or are all of them needed, and if so, is there any specific order in which they have to be applied?
Argh... I forgot to create patches with xxx/nnn notation... Here is the order of patches: 0001-jffs2-add-sector_size-field-to-part_info-structure.patch 0002-jffs2-rewrite-jffs2-scanning-code-based-on-Linux-on.patch 0003-jffs2-summary-support.patch 0004-jffs2-fix-searching-for-latest-version-in-jffs2_1pa.patch 0005-jffs2-add-buffer-to-cache-flash-accesses.patch 0006-jffs2-cache-data_crc-results.patch
It's the order of changes in my tree and thus it's the simplest way to apply this patches. But actual dependencies are as follows: 0001 and 0004 don't depend on anything 0002 depends on 0001 0003, 0005 and 0006 depend on 0002
Regards, Ilya.
Ilya Yanok said the following:
Hi Wolfgang,
Wolfgang Denk wrote:
here is a set of changes we made to improve U-Boot JFFS2 code performance. We still can't reach Linux's performance but improvements are significant.
Any comments are welcome.
Are these patches independent of each other, or are all of them needed, and if so, is there any specific order in which they have to be applied?
Argh... I forgot to create patches with xxx/nnn notation... Here is the order of patches: 0001-jffs2-add-sector_size-field-to-part_info-structure.patch 0002-jffs2-rewrite-jffs2-scanning-code-based-on-Linux-on.patch 0003-jffs2-summary-support.patch 0004-jffs2-fix-searching-for-latest-version-in-jffs2_1pa.patch 0005-jffs2-add-buffer-to-cache-flash-accesses.patch 0006-jffs2-cache-data_crc-results.patch
It's the order of changes in my tree and thus it's the simplest way to apply this patches. But actual dependencies are as follows: 0001 and 0004 don't depend on anything 0002 depends on 0001 0003, 0005 and 0006 depend on 0002
Regards, Ilya.
I have put these patches to our board (MPC8548, NOR Flash, MTD partition size 48MB).
a) performance has improved (execution time of 'ls' reduced from ~16s to ~1.5s)
b) there is a bug in the patch 0005-jffs2-add-buffer-to-cache-flash-accesses: The passage
@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) latestVersion = jNode->version; } } - put_fl_mem(jNode); + if (pL->readbuf == NULL) + put_fl_mem(jNode); } #endif
should be
@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) latestVersion = jNode->version; } } - put_fl_mem(jNode); + if (pL->readbuf == NULL) + put_fl_mem(jNode, NULL); } #endif
c) I found no improvement with CFG_JFFS2_SORT_FRAGMENTS set (the way I detected b) ;-)
d) output of 'ls' is corrupted: no date and access right output, directories in / are shown multiple times.
Hello Michael,
Michael Lawnick wrote:
I have put these patches to our board (MPC8548, NOR Flash, MTD partition size 48MB).
a) performance has improved (execution time of 'ls' reduced from ~16s to ~1.5s)
Sounds good. That is what we were expecting.
b) there is a bug in the patch 0005-jffs2-add-buffer-to-cache-flash-accesses: The passage
@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) latestVersion = jNode->version; } }
put_fl_mem(jNode);
if (pL->readbuf == NULL)put_fl_mem(jNode);#endif
should be
@@ -724,12 +731,14 @@ jffs2_1pass_read_inode(struct b_lists *pL, u32 inode, char *dest) latestVersion = jNode->version; } }
put_fl_mem(jNode);
if (pL->readbuf == NULL)put_fl_mem(jNode, NULL);#endif
You are right. We haven't just tried to use CFG_JFFS2_SORT_FRAGMENTS...
c) I found no improvement with CFG_JFFS2_SORT_FRAGMENTS set (the way I detected b) ;-)
Same as above... We don't use CFG_JFFS2_SORT_FRAGMENTS... But it's very sad result really... I have no idea what is wrong with it...
d) output of 'ls' is corrupted: no date and access right output, directories in / are shown multiple times
Can you please be more specific on this? Is this with CFG_JFFS2_SORT_FRAGMENTS enabled or not? How did you create your FS? I can't reproduce any of described symptoms on our system...
The only thing I can tell you for now is that the first patch from my posting is wrong, the following patch fixes the problem:
From 38d2055f7b493af858c96d1db2aa5cef78adeae1 Mon Sep 17 00:00:00 2001
From: Ilya Yanok yanok@emcraft.com Date: Wed, 15 Oct 2008 16:22:35 +0200 Subject: jffs2: fix sector_size calculation in case of NOR flash
Need to take all sectors into account then calculating sector_size for NOR flashes.
Signed-off-by: Ilya Yanok yanok@emcraft.com --- common/cmd_jffs2.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index e7f07bf..2fc0b7d 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -344,7 +344,9 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
flash = &flash_info[id->num];
- part->sector_size = 0; + /* size of last sector */ + part->sector_size = flash->size - + (flash->start[flash->sector_count-1] - flash->start[0]);
offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) { @@ -360,6 +362,7 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) }
end_offset = part->offset + part->size; + offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) { if (i) { sector_size = flash->start[i] - flash->start[i-1]; @@ -367,10 +370,10 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part) part->sector_size = sector_size; } if ((flash->start[i] - flash->start[0]) == end_offset) - return 0; + offset_aligned = 1; }
- if (flash->size == end_offset) + if (offset_aligned || flash->size == end_offset) return 0;
printf("%s%d: partition (%s) size alignment incorrect\n",
Dear Ilya,
In message 490BAA2A.6020806@emcraft.com you wrote:
Hello Michael,
Michael Lawnick wrote:
I have put these patches to our board (MPC8548, NOR Flash, MTD partition size 48MB).
a) performance has improved (execution time of 'ls' reduced from ~16s to ~1.5s)
Sounds good. That is what we were expecting.
If you submit a patch, then please add such comments *below* the "---" line.
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index e7f07bf..2fc0b7d 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -344,7 +344,9 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
flash = &flash_info[id->num];
- part->sector_size = 0;
/* size of last sector */
part->sector_size = flash->size -
(flash->start[flash->sector_count-1] - flash->start[0]);offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) {
@@ -360,6 +362,7 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
^^^^^^^^^^^^^^^^^^^^^^^^^^
Patch is correcpted because your mailer wrapped long lines.
Please fix your mailer setup and resubmit.
Best regards,
Wolfgang Denk
Hello Wolfgang,
Wolfgang Denk wrote:
Sounds good. That is what we were expecting.
If you submit a patch, then please add such comments *below* the "---" line.
That wasn't a patch submission actually... I've just wanted Michael to test if this patch helps with his problem. I'll post the full updated patch series soon.
diff --git a/common/cmd_jffs2.c b/common/cmd_jffs2.c index e7f07bf..2fc0b7d 100644 --- a/common/cmd_jffs2.c +++ b/common/cmd_jffs2.c @@ -344,7 +344,9 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
flash = &flash_info[id->num];
- part->sector_size = 0;
/* size of last sector */
part->sector_size = flash->size -
(flash->start[flash->sector_count-1] - flash->start[0]);offset_aligned = 0; for (i = 0; i < flash->sector_count; i++) {
@@ -360,6 +362,7 @@ static int part_validate_nor(struct mtdids *id, struct part_info *part)
^^^^^^^^^^^^^^^^^^^^^^^^^^
Patch is correcpted because your mailer wrapped long lines.
Please fix your mailer setup and resubmit.
That's my fault... Usually I send patches with git-send-email... As I said before I'll post correct patches in a short time.
Regards, Ilya.
Ilya Yanok said the following:
Hello Michael,
Michael Lawnick wrote:
...
d) output of 'ls' is corrupted: no date and access right output, directories in / are shown multiple times
Can you please be more specific on this? Is this with CFG_JFFS2_SORT_FRAGMENTS enabled or not? How did you create your FS? I can't reproduce any of described symptoms on our system...
The only thing I can tell you for now is that the first patch from my posting is wrong, the following patch fixes the problem:
... I added your patch, it corrected date part of 'ls'. For both versions (CFG_JFFS2_SORT_FRAGMENTS set or not set) - device root based directories are printed out up to 9 times - output of access rights is still corrupted, e.g.: dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:08:17 2008 10 -ÿÿÿÿ-ÿÿ-ÿ 262144 Thu Jan 01 00:00:33 1970 u-boot.bin dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:08:32 2008 1 dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:00:58 2008 2
The difference is: *with* CFG_JFFS2_SORT_FRAGMENTS time for ls is ~ 16s *without* CFG_JFFS2_SORT_FRAGMENTS time for ls is ~ 2s
Setup scenario: U-boot flashed, Linux booted and jffs mounted on erased flash --> in background-process flash gets formatted. create dir '10' in root of flash, copy 2 files with ~1.2MB each into it. Run following script ~300 times: mkdir /flash/11 cp /flash/10/* /flash/11 rm -rf /flash/1 mv /flash/2 /flash/1 mv /flash/3 /flash/2 mv /flash/4 /flash/3 mv /flash/5 /flash/4 mv /flash/6 /flash/5 mv /flash/7 /flash/6 mv /flash/8 /flash/7 mv /flash/9 /flash/8 mv /flash/10 /flash/9 mv /flash/11 /flash/10 sync
This should mix up jffs image enough to see what I observe.
Michael Lawnick said the following: ...
- output of access rights is still corrupted, e.g.:
dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:08:17 2008 10 -ÿÿÿÿ-ÿÿ-ÿ 262144 Thu Jan 01 00:00:33 1970 u-boot.bin dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:08:32 2008 1 dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:00:58 2008 2
...
Ok, fixed this in my code: in jffs2_1pass.c @@ mkmodestr(unsigned long mode, char *str) you'll find declaration static const char *l = "xwr";
Although this is per se correct, it is not needed in context and screws up 'ls' in my case: Our u-boot is not placed in flash were it is located to on compilation time. static const char * points into flash space at some not properly initialized flash space. After modifying to char *l = "xwr"; output of access rights is fine now: drwxr-xr-x 0 Tue Oct 21 11:08:17 2008 10 -rwxr-xr-x 262144 Thu Jan 01 00:00:33 1970 u-boot.bin drwxr-xr-x 0 Tue Oct 21 11:08:32 2008 1 drwxr-xr-x 0 Tue Oct 21 11:00:58 2008 2
(Still directories are listed multiple times)
Could we modify this declaration to my non-static version in the trunk?
On Tue, 2008-11-04 at 13:31 +0100, Michael Lawnick wrote:
Michael Lawnick said the following: ...
- output of access rights is still corrupted, e.g.:
dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:08:17 2008 10 -ÿÿÿÿ-ÿÿ-ÿ 262144 Thu Jan 01 00:00:33 1970 u-boot.bin dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:08:32 2008 1 dÿÿÿÿ-ÿÿ-ÿ 0 Tue Oct 21 11:00:58 2008 2
...
Ok, fixed this in my code: in jffs2_1pass.c @@ mkmodestr(unsigned long mode, char *str) you'll find declaration static const char *l = "xwr";
Although this is per se correct, it is not needed in context and screws up 'ls' in my case: Our u-boot is not placed in flash were it is located to on compilation time. static const char * points into flash space at some not properly initialized flash space. After modifying to char *l = "xwr"; output of access rights is fine now: drwxr-xr-x 0 Tue Oct 21 11:08:17 2008 10 -rwxr-xr-x 262144 Thu Jan 01 00:00:33 1970 u-boot.bin drwxr-xr-x 0 Tue Oct 21 11:08:32 2008 1 drwxr-xr-x 0 Tue Oct 21 11:00:58 2008 2
(Still directories are listed multiple times)
Could we modify this declaration to my non-static version in the trunk?
Is this on PPC? If so it is due to missing fixup support. Grant Likely once sent a patch fixing this. It got applied, but later reverted because there was some problems that never got solved.
Joakim Tjernlund said the following:
On Tue, 2008-11-04 at 13:31 +0100, Michael Lawnick wrote:
Could we modify this declaration to my non-static version in the trunk?
Is this on PPC? If so it is due to missing fixup support. Grant Likely once sent a patch fixing this. It got applied, but later reverted because there was some problems that never got solved.
It is on PPC. IMHO such parts should be removed at least step by step, where it is proven that the dynamic version works. I can't see any benefit in the static solution here.
Michael,
I tried to reproduce the bug you mentioned (multiple appearance of directories in ls output) using the script you provided, to no avail: ls correctly showed the top-level directory, both with and without CFG_JFFS2_SORT_FRAGMENTS option.
I am inclined to suspect the same cause as you mentioned below:
Our u-boot is not placed in flash were it is located to on compilation time. static const char * points into flash space at some not properly initialized flash space.
This would affect all global and static variables, with unpredictable results. Could you try to reproduce this bug with u-boot placed at the address it was linked to?
As a side note, your fix for compilation with CFG_JFFS2_SORT_FRAGMENTS option turned on would be better as follows: replace
if (pL->readbuf == NULL) put_fl_mem(jNode);
with
put_fl_mem(jNode, pL->readbuf);
(put_fl_mem already performs this check)
Regards, Alexey.
On Mon, 2008-11-10 at 22:49 -0800, Alexey Neyman wrote:
Michael,
I tried to reproduce the bug you mentioned (multiple appearance of directories in ls output) using the script you provided, to no avail: ls correctly showed the top-level directory, both with and without CFG_JFFS2_SORT_FRAGMENTS option.
I am inclined to suspect the same cause as you mentioned below:
Our u-boot is not placed in flash were it is located to on compilation time. static const char * points into flash space at some not properly initialized flash space.
This would affect all global and static variables, with unpredictable results. Could you try to reproduce this bug with u-boot placed at the address it was linked to?
You can't do that. What you need is to enable fixup support. Try undoing this commit 1c3dd43338a077165e7e0309cb3994e65d2bdbf8
participants (5)
-
Alexey Neyman -
Ilya Yanok -
Joakim Tjernlund -
Michael Lawnick -
Wolfgang Denk