Re: [U-Boot] [PATCH v2] Enable journal replay for UBIFS

Hello Anton,

Am 19.01.2015 14:48, schrieb Anton Habegger:

During mount_ubifs the ubifs_replay_journal was disabled. This patch enables it again and fix some unrecoverable UBIFS volumes.

The following patch enables the error handling for ubifs_replay_journal as well. Please see discussion "Unreadable UBIFS partition after power cuts".

Signed-off-by: Anton Habegger anton.habegger@delta-es.com

fs/ubifs/Makefile | 2 +- fs/ubifs/gc.c | 987 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ubifs/replay.c | 8 +- fs/ubifs/super.c | 8 +- fs/ubifs/tnc.c | 7 +- 5 files changed, 997 insertions(+), 15 deletions(-) create mode 100644 fs/ubifs/gc.c

Thanks!

diff --git a/fs/ubifs/Makefile b/fs/ubifs/Makefile index 8c8c6ac..5efb349 100644 --- a/fs/ubifs/Makefile +++ b/fs/ubifs/Makefile @@ -12,4 +12,4 @@ obj-y := ubifs.o io.o super.o sb.o master.o lpt.o obj-y += lpt_commit.o scan.o lprops.o obj-y += tnc.o tnc_misc.o debug.o crc16.o budget.o -obj-y += log.o orphan.o recovery.o replay.o +obj-y += log.o orphan.o recovery.o replay.o gc.o

Hmm.... do we really need this in u-boot? Ah, the most functions are unneeded ... ok.

diff --git a/fs/ubifs/gc.c b/fs/ubifs/gc.c new file mode 100644 index 0000000..159ee67 --- /dev/null +++ b/fs/ubifs/gc.c @@ -0,0 +1,987 @@ +/*

• • This file is part of UBIFS.
• • Copyright (C) 2006-2008 Nokia Corporation.
• • This program is free software; you can redistribute it and/or modify it
• • under the terms of the GNU General Public License version 2 as published by
• • the Free Software Foundation.
• • This program is distributed in the hope that it will be useful, but WITHOUT
• • ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
• • FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
• • more details.
• • You should have received a copy of the GNU General Public License along with
• • this program; if not, write to the Free Software Foundation, Inc., 51
• • Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
• • Authors: Adrian Hunter

• •      Artem Bityutskiy (Битюцкий Артём)
    

• */

+/*

• • This file implements garbage collection. The procedure for garbage

collection

• • is different depending on whether a LEB as an index LEB (contains index
• • nodes) or not. For non-index LEBs, garbage collection finds a LEB which
• • contains a lot of dirty space (obsolete nodes), and copies the non-obsolete
• • nodes to the journal, at which point the garbage-collected LEB is free to be
• • reused. For index LEBs, garbage collection marks the non-obsolete

index nodes

• • dirty in the TNC, and after the next commit, the garbage-collected LEB is
• • to be reused. Garbage collection will cause the number of dirty index nodes
• • to grow, however sufficient space is reserved for the index to ensure the
• • commit will never run out of space.
• • Notes about dead watermark. At current UBIFS implementation we assume that
• • LEBs which have less than @c->dead_wm bytes of free + dirty space are full
• • and not worth garbage-collecting. The dead watermark is one min. I/O unit
• • size, or min. UBIFS node size, depending on what is greater. Indeed, UBIFS
• • Garbage Collector has to synchronize the GC head's write buffer before
• • returning, so this is about wasting one min. I/O unit. However, UBIFS GC can
• • actually reclaim even very small pieces of dirty space by garbage collecting
• • enough dirty LEBs, but we do not bother doing this at this implementation.
• • Notes about dark watermark. The results of GC work depends on how big are
• • the UBIFS nodes GC deals with. Large nodes make GC waste more space. Indeed,
• • if GC move data from LEB A to LEB B and nodes in LEB A are large, GC would
• • have to waste large pieces of free space at the end of LEB B, because nodes
• • from LEB A would not fit. And the worst situation is when all nodes are of
• • maximum size. So dark watermark is the amount of free + dirty space in LEB
• • which are guaranteed to be reclaimable. If LEB has less space, the GC might
• • be unable to reclaim it. So, LEBs with free + dirty greater than dark
• • watermark are "good" LEBs from GC's point of few. The other LEBs are not so
• • good, and GC takes extra care when moving them.
• */

+#ifndef __UBOOT__ +#include <linux/slab.h> +#include <linux/pagemap.h> +#include <linux/list_sort.h> +#endif +#include "ubifs.h"

+#ifndef __UBOOT__ +/*

• • GC may need to move more than one LEB to make progress. The below constants
• • define "soft" and "hard" limits on the number of LEBs the garbage collector
• • may move.
• */

+#define SOFT_LEBS_LIMIT 4 +#define HARD_LEBS_LIMIT 32

+/**

• • switch_gc_head - switch the garbage collection journal head.
• • @c: UBIFS file-system description object
• • @buf: buffer to write
• • @len: length of the buffer to write
• • @lnum: LEB number written is returned here
• • @offs: offset written is returned here
• • This function switch the GC head to the next LEB which is reserved in
• • @c->gc_lnum. Returns %0 in case of success, %-EAGAIN if commit is required,
• • and other negative error code in case of failures.
• */

+static int switch_gc_head(struct ubifs_info *c) +{

• int err, gc_lnum = c->gc_lnum;
• struct ubifs_wbuf *wbuf = &c->jheads[GCHD].wbuf;
• ubifs_assert(gc_lnum != -1);

Hmm.. something seems wrong with your mailer ... your patch is malformed. Could you fix this please?

[...]

diff --git a/fs/ubifs/replay.c b/fs/ubifs/replay.c index 7268b37..75b92ac 100644 --- a/fs/ubifs/replay.c +++ b/fs/ubifs/replay.c

[...]

@@ -1050,7 +1047,11 @@ int ubifs_replay_journal(struct ubifs_info *c)

• depend on it. This means we have to initialize it to make sure
• budgeting works properly.

*/ +#ifndef __UBOOT__ c->bi.uncommitted_idx = atomic_long_read(&c->dirty_zn_cnt); +#else

• c->bi.uncommitted_idx = c->dirty_zn_cnt;

+#endif

please introduce atomic_long_read() as mentioned to your other EMail.

c->bi.uncommitted_idx *= c->max_idx_node_sz;

ubifs_assert(c->bud_bytes <= c->max_bud_bytes || c->need_recovery); @@ -1063,4 +1064,3 @@ out: c->replaying = 0; return err; } -#endif diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c index 01d449a..10f8fff 100644 --- a/fs/ubifs/super.c +++ b/fs/ubifs/super.c @@ -1049,7 +1049,6 @@ static void free_orphans(struct ubifs_info *c) c->orph_buf = NULL; }

-#ifndef __UBOOT__ /**

• free_buds - free per-bud objects.
• @c: UBIFS file-system description object

@@ -1061,7 +1060,6 @@ static void free_buds(struct ubifs_info *c) rbtree_postorder_for_each_entry_safe(bud, n, &c->buds, rb) kfree(bud); } -#endif

/**

• check_volume_empty - check if the UBI volume is empty.

@@ -1242,6 +1240,7 @@ static int ubifs_parse_options(struct ubifs_info *c, char *options,

return 0; } +#endif

/**

• destroy_journal - destroy journal data structures.

@@ -1272,7 +1271,6 @@ static void destroy_journal(struct ubifs_info *c) ubifs_tnc_close(c); free_buds(c); } -#endif

/**

• bu_init - initialize bulk-read information.

@@ -1502,11 +1500,9 @@ static int mount_ubifs(struct ubifs_info *c) if (err) goto out_lpt;

-#ifndef __UBOOT__ err = ubifs_replay_journal(c); if (err) goto out_journal; -#endif

/* Calculate 'min_idx_lebs' after journal replay */ c->bi.min_idx_lebs = ubifs_calc_min_idx_lebs(c); @@ -1678,10 +1674,8 @@ out_infos: spin_unlock(&ubifs_infos_lock); out_orphans: free_orphans(c); -#ifndef __UBOOT__ out_journal: destroy_journal(c); -#endif out_lpt: ubifs_lpt_free(c, 0); out_master: diff --git a/fs/ubifs/tnc.c b/fs/ubifs/tnc.c index 95cae54..b6737db 100644 --- a/fs/ubifs/tnc.c +++ b/fs/ubifs/tnc.c @@ -2827,7 +2827,6 @@ out_unlock: return ERR_PTR(err); }

-#ifndef __UBOOT__ /**

• tnc_destroy_cnext - destroy left-over obsolete znodes from a failed commit.
• @c: UBIFS file-system description object

@@ -2850,7 +2849,6 @@ static void tnc_destroy_cnext(struct ubifs_info *c) kfree(znode); } while (cnext && cnext != c->cnext); }

• /**
  • ubifs_tnc_close - close TNC subsystem and free all related resources.
  • @c: UBIFS file-system description object

@@ -2859,17 +2857,20 @@ void ubifs_tnc_close(struct ubifs_info *c) { tnc_destroy_cnext(c); if (c->zroot.znode) { +#ifndef __UBOOT__ long n; +#endif

ubifs_destroy_tnc_subtree(c->zroot.znode); +#ifndef __UBOOT__ n = atomic_long_read(&c->clean_zn_cnt); atomic_long_sub(n, &ubifs_clean_zn_cnt); +#endif } kfree(c->gap_lebs); kfree(c->ilebs); destroy_old_idx(c); } -#endif

/**

• left_znode - get the znode to the left.

bye, Heiko