Hello Simon,

On Tue, Nov 4, 2014 at 5:04 PM, Simon Glass sjg@chromium.org wrote:

Hi,

On 3 November 2014 18:49, Suriyan Ramasami suriyan.r@gmail.com wrote:

...

Test size/read/write commands in a sandbox environment.

Signed-off-by: Suriyan Ramasami suriyan.r@gmail.com

---

Changes in v6:

• Update expected results in comments
• Simon comments -
• Remove README file and add it in start of script
• Drop blank lines in top of functions
• Put repeating constants in lower case variables
• Refer to U-Boot as U-Boot
• Move settings to top of file and code in functions or otherwise at bottom
• Create check_prereq function
• Use mkfs -t <type> -F <img> for ext4, but modify for fat

Changes in v5:

• Simon comments -
• Add README file to document how to run it
• Generate output in a sandbox environment
• Add one line comments on shell variables used
• Avoid camel case through out
• Path to UBOOT is variable at top
• Print PASSED or FAILED at end, and set return code, 0 if OK, 1 otherwise

test/fs/fs-test.sh | 562 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 562 insertions(+) create mode 100755 test/fs/fs-test.sh

diff --git a/test/fs/fs-test.sh b/test/fs/fs-test.sh new file mode 100755 index 0000000..a817de5 --- /dev/null +++ b/test/fs/fs-test.sh @@ -0,0 +1,562 @@ +#!/bin/bash +# +# (C) Copyright 2014 Suriyan Ramasami +# +# SPDX-License-Identifier: GPL-2.0+ +#

+# Invoke this test script from U-Boot base directory as ./test/fs/fs-test.sh +# It currently tests the fs/sb and native commands for ext4 and fat partitions +# Expected results are as follows: +# EXT4 tests: +# fs-test.sb.ext4.out: Summary: PASS: 17 FAIL: 2 +# fs-test.ext4.out: Summary: PASS: 11 FAIL: 8 +# fs-test.fs.ext4.out: Summary: PASS: 11 FAIL: 8 +# FAT tests: +# fs-test.sb.fat.out: Summary: PASS: 17 FAIL: 2 +# fs-test.fat.out: Summary: PASS: 19 FAIL: 0 +# fs-test.fs.fat.out: Summary: PASS: 19 FAIL: 0 +# Total Summary: TOTAL PASS: 94 TOTAL FAIL: 20

I found this does not work as I have mkfs.vfat but not mkfs.fat. Perhaps it needs to change to vfat? I've pushed a patch to u-boot-x86/fs-working which does this.

OK, I shall check up on that.

I have a minor comment below but it isn't important. I'm a bit worried that this script has grown so large it should be in Python, but we can worry about that another time. It does work so let's get it in.

I shall probably work on migrating this to Python.

There is one problem though - bisectability. Some patches change the API and thus U-Boot doesn't build.

My suggestion to reduce patch size/complexity was do break it into independent changes - e.g. adding the 'size' parameter in one patch and the 'actread/actwrite' parameter in another. Maybe that turned out to be too hard?

If you want to do each FS separately, you might be able to:

• patch each FS in separate patches to handle the two parameters

internally, but don't change the outside API function. For example leave file_fat_write() alone

• do this for each FS, and then have a final patch that changes all

the API functions and the FS layer

However if you want to squash things together a bit more, that is OK, I feel you have put enough effort into this already!

OK. Sorry about that. I just realized what a big idiot I am! I shall work on trying to segragate the patches in independent units keeping bisectability in mind this time around.

Thanks and Regards - Suriyan

./tools/buildman/buildman -b try-fs sandbox -se boards.cfg is up to date. Nothing to do. Summary of 8 commits for 1 boards (1 thread, 4 jobs per thread) 01: Merge branch 'master' of git://git.denx.de/u-boot-mips 02: sandbox: script for testing sandbox/ext4/fat/fs commands 03: fs: interface changes to accomodate files greater than 2GB sandbox: + sandbox +fs/fs.c:124:12: error: ‘ext4_write_file’ undeclared here (not in a function) +make[2]: *** [fs/fs.o] Error 1 +make[1]: *** [fs] Error 2 +make: *** [sub-make] Error 2 w+fs/fs.c:104:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:104:3: warning: (near initialization for ‘fstypes[0].size’) [enabled by default] w+fs/fs.c:105:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:105:3: warning: (near initialization for ‘fstypes[0].read’) [enabled by default] w+fs/fs.c:121:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:121:3: warning: (near initialization for ‘fstypes[1].size’) [enabled by default] w+fs/fs.c:122:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:122:3: warning: (near initialization for ‘fstypes[1].read’) [enabled by default] w+fs/fs.c:138:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:138:3: warning: (near initialization for ‘fstypes[2].size’) [enabled by default] w+fs/fs.c:139:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:139:3: warning: (near initialization for ‘fstypes[2].read’) [enabled by default] w+fs/fs.c:140:3: warning: initialization from incompatible pointer type [enabled by default] w+fs/fs.c:140:3: warning: (near initialization for ‘fstypes[2].write’) [enabled by default] 04: fat: interface changes to accomodate files greater than 2GB w-fs/fs.c:104:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:104:3: warning: (near initialization for ‘fstypes[0].size’) [enabled by default] w-fs/fs.c:105:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:105:3: warning: (near initialization for ‘fstypes[0].read’) [enabled by default] 05: ext4: interface changes to accomodate files greater than 2GB sandbox: sandbox -fs/fs.c:124:12: error: ‘ext4_write_file’ undeclared here (not in a function) -make[2]: *** [fs/fs.o] Error 1 -make[1]: *** [fs] Error 2 -make: *** [sub-make] Error 2 w-fs/fs.c:121:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:121:3: warning: (near initialization for ‘fstypes[1].size’) [enabled by default] w-fs/fs.c:122:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:122:3: warning: (near initialization for ‘fstypes[1].read’) [enabled by default] 06: sandbox: Use md5sum and fatwrite to enable testing of fs commands 07: sandbox: interface changes to accomodate files greater than 2GB sandbox: sandbox w-fs/fs.c:138:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:138:3: warning: (near initialization for ‘fstypes[2].size’) [enabled by default] w-fs/fs.c:139:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:139:3: warning: (near initialization for ‘fstypes[2].read’) [enabled by default] w-fs/fs.c:140:3: warning: initialization from incompatible pointer type [enabled by default] w-fs/fs.c:140:3: warning: (near initialization for ‘fstypes[2].write’) [enabled by default] 08: Change fat to vfat

...

+# pre-requisite binaries list. +PREREQ_BINS="md5sum mkfs mount umount dd fallocate mkdir"

+# All generated output files from this test will be in $OUT_DIR +# Hence everything is sandboxed. +OUT_DIR="sandbox/test/fs"

+# Location of generated sandbox u-boot +UBOOT="./sandbox/u-boot"

+# Our mount directory will be in the sandbox +MOUNT_DIR="${OUT_DIR}/mnt"

+# The file system image we create will have the $IMG prefix. +IMG="${OUT_DIR}/3GB"

+# $SMALL_FILE is the name of the 1MB file in the file system image +SMALL_FILE="1MB.file"

+# $BIG_FILE is the name of the 2.5GB file in the file system image +BIG_FILE="2.5GB.file"

+# $MD5_FILE will have the expected md5s when we do the test +# They shall have a suffix which represents their file system (ext4/fat) +MD5_FILE="${OUT_DIR}/md5s.list"

+# $OUT shall be the prefix of the test output. Their suffix will be .out +OUT="${OUT_DIR}/fs-test"

+# Full Path of the 1 MB file that shall be created in the fs image. +MB1="${MOUNT_DIR}/${SMALL_FILE}" +GB2p5="${MOUNT_DIR}/${BIG_FILE}"

+# ************************ +# * Functions start here * +# ************************

+# Check if the prereq binaries exist, or exit +function check_prereq() {

•   for prereq in $PREREQ_BINS; do
  

•           if [ ! -x `which $prereq` ]; then
  

•                   echo "Missing $prereq binary. Exiting!"
  

•                   exit
  

•           fi
  

•   done
  

•   # We use /dev/urandom to create files. Check if it exists.
  

•   if [ ! -c /dev/urandom ]; then
  

•           echo "Missing character special /dev/urandom. Exiting!"
  

•           exit
  

•   fi
  

+}

+# If 1st param is "clean", then clean out the generated files and exit +function check_clean() {

•   if [ "$1" = "clean" ]; then
  

•           rm -rf "$OUT_DIR"
  

•           echo "Cleaned up generated files. Exiting"
  

•           exit
  

•   fi
  

+}

+# Generate sandbox U-Boot - gleaned from /test/dm/test-dm.sh +function compile_sandbox() {

•   unset CROSS_COMPILE
  

•   NUM_CPUS=$(cat /proc/cpuinfo |grep -c processor)
  

•   make O=sandbox sandbox_config
  

•   make O=sandbox -s -j${NUM_CPUS}
  

•   # Check if U-Boot exists
  

•   if [ ! -x "$UBOOT" ]; then
  

•           echo "$UBOOT does not exist or is not executable"
  

•           echo "Build error?"
  

•           echo "Please run this script as ./test/fs/`basename $0`"
  

•           exit
  

•   fi
  

+}

+# Clean out all generated files other than the file system images +# We save time by not deleting and recreating the file system images +function prepare_env() {

•   rm -f ${MD5_FILE}.* ${OUT}.*
  

•   mkdir ${OUT_DIR}
  

+}

+# 1st parameter is the name of the image file to be created +# 2nd parameter is the filesystem - fat ext4 etc +# -F cant be used with fat as it means something else. +function create_image() {

•   # Create image if not already present - saves time, while debugging
  

•   if [ "$2" = "ext4" ]; then
  

•           MKFS_OPTION="-F"
  

•   else
  

•           MKFS_OPTION=""
  

•   fi
  

•   if [ ! -f "$1" ]; then
  

•           fallocate -l 3G "$1" &> /dev/null
  

•           mkfs -t "$2" $MKFS_OPTION "$1" &> /dev/null
  

•   fi
  

+}

+# 1st parameter is the FS type: fat/ext4 +# 2nd parameter is the name of small file +# Returns filename which can be used for fat or ext4 for writing +function fname_for_write() {

•   case $1 in
  

•           ext4)
  

•                   # ext4 needs absolute path name of file
  

•                   echo /${2}.w
  

•                   ;;
  

•           *)
  

•                   echo ${2}.w
  

•                   ;;
  

•   esac
  

+}

+# 1st parameter is image file +# 2nd parameter is file system type - fat/ext4 +# 3rd parameter is name of small file +# 4th parameter is name of big file +# 5th parameter is fs/nonfs/sb - to dictate generic fs commands or +# otherwise or sb hostfs +# 6th parameter is the directory path for the files. Its "" for generic +# fs and ext4/fat and full patch for sb hostfs +# UBOOT is set in env +function test_image() {

•   addr="0x01000008"
  

•   length="0x00100000"
  

•   case "$2" in
  

•           fat)
  

•           PREFIX="fat"
  

•           WRITE="write"
  

•           ;;
  

•           ext4)
  

•           PREFIX="ext4"
  

•           WRITE="write"
  

•           ;;
  

•           *)
  

•           echo "Unhandled filesystem $2. Exiting!"
  

•           exit
  

•           ;;
  

•   esac
  

•   case "$5" in
  

•           fs)
  

•           PREFIX=""
  

•           WRITE="save"
  

•           SUFFIX=" 0:0"
  

•           ;;
  

•           nonfs)
  

•           SUFFIX=" 0:0"
  

•           ;;
  

•           sb)
  

•           PREFIX="sb "
  

•           WRITE="save"
  

•           SUFFIX="fs -"
  

•           ;;
  

•           *)
  

•           echo "Unhandled mode $5. Exiting!"
  

•           exit
  

•           ;;
  

•   esac
  

•   if [ -z "$6" ]; then
  

•           FILE_WRITE=`fname_for_write $2 $3`
  

•           FILE_SMALL=$3
  

•           FILE_BIG=$4
  

•   else
  

•           FILE_WRITE=$6/`fname_for_write $2 $3`
  

•           FILE_SMALL=$6/$3
  

•           FILE_BIG=$6/$4
  

•   fi
  

•   # In u-boot commands, <interface> stands for host or hostfs
  

•   # hostfs maps to the host fs.
  

•   # host maps to the "sb bind" that we do
  

•   $UBOOT << EOF
  

+sb=$5 +setenv bind 'if test "$sb" != sb; then sb bind 0 "$1"; fi' +run bind +# Test Case 1 - ls +${PREFIX}ls host${SUFFIX} $6 +# +# We want ${PREFIX}size host 0:0 $3 for host commands and +# sb size hostfs - $3 for hostfs commands. +# 1MB is 0x0010 0000 +# Test Case 2 - size of small file +${PREFIX}size host${SUFFIX} $FILE_SMALL +printenv filesize +setenv filesize

+# 2.5GB (1024*1024*2500) is 0x9C40 0000 +# Test Case 3 - size of big file +${PREFIX}size host${SUFFIX} $FILE_BIG +printenv filesize +setenv filesize

+# Notes about load operation +# If I use 0x01000000 I get DMA misaligned error message +# Last two parameters are size and offset.

+# Test Case 4a - Read full 1MB of small file +${PREFIX}load host${SUFFIX} $addr $FILE_SMALL +printenv filesize +# Test Case 4b - Read full 1MB of small file +md5sum $addr $filesize +setenv filesize

+# Test Case 5a - First 1MB of big file +${PREFIX}load host${SUFFIX} $addr $FILE_BIG $length 0x0 +printenv filesize +# Test Case 5b - First 1MB of big file +md5sum $addr $filesize +setenv filesize

+# fails for ext as no offset support +# Test Case 6a - Last 1MB of big file +${PREFIX}load host${SUFFIX} $addr $FILE_BIG $length 0x9C300000 +printenv filesize +# Test Case 6b - Last 1MB of big file +md5sum $addr $filesize +setenv filesize

+# fails for ext as no offset support +# Test Case 7a - One from the last 1MB chunk of 2GB +${PREFIX}load host${SUFFIX} $addr $FILE_BIG $length 0x7FF00000 +printenv filesize +# Test Case 7b - One from the last 1MB chunk of 2GB +md5sum $addr $filesize +setenv filesize

+# fails for ext as no offset support +# Test Case 8a - One from the start 1MB chunk from 2GB +${PREFIX}load host${SUFFIX} $addr $FILE_BIG $length 0x80000000 +printenv filesize +# Test Case 8b - One from the start 1MB chunk from 2GB +md5sum $addr $filesize +setenv filesize

+# fails for ext as no offset support +# Test Case 9a - One 1MB chunk crossing the 2GB boundary +${PREFIX}load host${SUFFIX} $addr $FILE_BIG $length 0x7FF80000 +printenv filesize +# Test Case 9b - One 1MB chunk crossing the 2GB boundary +md5sum $addr $filesize +setenv filesize

+# Generic failure case +# Test Case 10 - 2MB chunk from the last 1MB of big file +${PREFIX}load host${SUFFIX} $addr $FILE_BIG 0x00200000 0x9C300000 +printenv filesize +#

+# Read 1MB from small file +${PREFIX}load host${SUFFIX} $addr $FILE_SMALL +# Write it back to test the writes +# Test Case 11a - Check that the write succeeded +${PREFIX}${WRITE} host${SUFFIX} $addr $FILE_WRITE $filesize +mw.b $addr 00 100 +${PREFIX}load host${SUFFIX} $addr $FILE_WRITE +# Test Case 11b - Check md5 of written to is same as the one read from +md5sum $addr $filesize +setenv filesize +# +reset

+EOF +}

+# 1st argument is the name of the image file. +# 2nd argument is the file where we generate the md5s of the files +# generated with the appropriate start and length that we use to test. +# It creates the necessary files in the image to test. +# $GB2p5 is the path of the big file (2.5 GB) +# $MB1 is the path of the small file (1 MB) +# $MOUNT_DIR is the path we can use to mount the image file. +function create_files() {

•   # Mount the image so we can populate it.
  

•   mkdir -p "$MOUNT_DIR"
  

•   sudo mount -o loop,rw "$1" "$MOUNT_DIR"
  

•   # Create big file in this image.
  

•   # Note that we work only on the start 1MB, couple MBs in the 2GB range
  

•   # and the last 1 MB of the huge 2.5GB file.
  

•   # So, just put random values only in those areas.
  

•   if [ ! -f "${GB2p5}" ]; then
  

•           sudo dd if=/dev/urandom of="${GB2p5}" bs=1M count=1 \
  

•                   &> /dev/null
  

•           sudo dd if=/dev/urandom of="${GB2p5}" bs=1M count=2 seek=2047 \
  

•                   &> /dev/null
  

•           sudo dd if=/dev/urandom of="${GB2p5}" bs=1M count=1 seek=2499 \
  

•                   &> /dev/null
  

•   fi
  

•   # Create a small file in this image.
  

•   if [ ! -f "${MB1}" ]; then
  

•           sudo dd if=/dev/urandom of="${MB1}" bs=1M count=1 \
  

•                   &> /dev/null
  

•   fi
  

•   # Delete the small file which possibly is written as part of a
  

•   # previous test.
  

•   sudo rm -f "${MB1}.w"
  

•   # Generate the md5sums of reads that we will test against small file
  

•   dd if="${MB1}" bs=1M skip=0 count=1 2> /dev/null | md5sum > "$2"
  

•   # Generate the md5sums of reads that we will test against big file
  

•   # One from beginning of file.
  

•   dd if="${GB2p5}" bs=1M skip=0 count=1 \
  

•           2> /dev/null | md5sum >> "$2"
  

•   # One from end of file.
  

•   dd if="${GB2p5}" bs=1M skip=2499 count=1 \
  

•           2> /dev/null | md5sum >> "$2"
  

•   # One from the last 1MB chunk of 2GB
  

•   dd if="${GB2p5}" bs=1M skip=2047 count=1 \
  

•           2> /dev/null | md5sum >> "$2"
  

•   # One from the start 1MB chunk from 2GB
  

•   dd if="${GB2p5}" bs=1M skip=2048 count=1 \
  

•           2> /dev/null | md5sum >> "$2"
  

•   # One 1MB chunk crossing the 2GB boundary
  

•   dd if="${GB2p5}" bs=512K skip=4095 count=2 \
  

•           2> /dev/null | md5sum >> "$2"
  

•   sync
  

•   sudo umount "$MOUNT_DIR"
  

•   rmdir "$MOUNT_DIR"
  

+}

+# 1st parameter is the text to print +# if $? is 0 its a pass, else a fail +# As a side effect it shall update env variable PASS and FAIL +function pass_fail() {

•   if [ $? -eq 0 ]; then
  

•           echo pass - "$1"
  

•           PASS=$((PASS + 1))
  

•   else
  

•           echo FAIL - "$1"
  

•           FAIL=$((FAIL + 1))
  

•   fi
  

+}

+# 1st parameter is the string which leads to an md5 generation +# 2nd parameter is the file we grep, for that string +# 3rd parameter is the name of the file which has md5s in it +# 4th parameter is the line # in the md5 file that we match it against +# This function checks if the md5 of the file in the sandbox matches +# that calculated while generating the file +# 5th parameter is the string to print with the result +check_md5() {

•   # md5sum in u-boot has output of form:
  

•   # md5 for 01000008 ... 01100007 ==> <md5>
  

•   # the 7th field is the actual md5
  

•   md5_src=`grep -A3 "$1" "$2" | grep "md5 for"`
  

•   md5_src=($md5_src)
  

•   md5_src=${md5_src[6]}
  

•   # The md5 list, each line is of the form:
  

•   # - <md5>
  

•   # the 2nd field is the actual md5
  

•   md5_dst=`sed -n $4p $3`
  

•   md5_dst=($md5_dst)
  

•   md5_dst=${md5_dst[0]}
  

•   # For a pass they should match.
  

•   [ "$md5_src" = "$md5_dst" ]
  

•   pass_fail "$5"
  

+}

+# 1st parameter is the name of the output file to check +# 2nd parameter is the name of the file containing the md5 expected +# 3rd parameter is the name of the small file +# 4th parameter is the name of the big file +# 5th paramter is the name of the written file +# This function checks the output file for correct results. +function check_results() {

•   echo "** Start $1"
  

•   PASS=0
  

•   FAIL=0
  

•   # Check if the ls is showing correct results for 2.5 gb file
  

•   grep -A6 "Test Case 1 " "$1" | egrep -iq "2621440000 *$4"
  

•   pass_fail "TC1: ls of $4"
  

•   # Check if the ls is showing correct results for 1 mb file
  

•   grep -A6 "Test Case 1 " "$1" | egrep -iq "1048576 *$3"
  

•   pass_fail "TC1: ls of $3"
  

•   # Check size command on 1MB.file
  

•   egrep -A3 "Test Case 2 " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC2: size of $3"
  

•   # Check size command on 2.5GB.file
  

•   egrep -A3 "Test Case 3 " "$1" | grep -q "filesize=9c400000"
  

•   pass_fail "TC3: size of $4"
  

•   # Check read full mb of 1MB.file
  

•   grep -A6 "Test Case 4a " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC4: load of $3 size"
  

•   check_md5 "Test Case 4b " "$1" "$2" 1 "TC4: load from $3"
  

•   # Check first mb of 2.5GB.file
  

•   grep -A6 "Test Case 5a " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC5: load of 1st MB from $4 size"
  

•   check_md5 "Test Case 5b " "$1" "$2" 2 "TC5: load of 1st MB from $4"
  

•   # Check last mb of 2.5GB.file
  

•   grep -A6 "Test Case 6a " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC6: load of last MB from $4 size"
  

•   check_md5 "Test Case 6b " "$1" "$2" 3 "TC6: load of last MB from $4"
  

•   # Check last 1mb chunk of 2gb from 2.5GB file
  

•   grep -A6 "Test Case 7a " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC7: load of last 1mb chunk of 2GB from $4 size"
  

•   check_md5 "Test Case 7b " "$1" "$2" 4 \
  

•           "TC7: load of last 1mb chunk of 2GB from $4"
  

•   # Check first 1mb chunk after 2gb from 2.5GB file
  

•   grep -A6 "Test Case 8a " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC8: load 1st MB chunk after 2GB from $4 size"
  

•   check_md5 "Test Case 8b " "$1" "$2" 5 \
  

•           "TC8: load 1st MB chunk after 2GB from $4"
  

•   # Check 1mb chunk crossing the 2gb boundary from 2.5GB file
  

•   grep -A6 "Test Case 9a " "$1" | grep -q "filesize=100000"
  

•   pass_fail "TC9: load 1MB chunk crossing 2GB boundary from $4 size"
  

•   check_md5 "Test Case 9b " "$1" "$2" 6 \
  

•           "TC9: load 1MB chunk crossing 2GB boundary from $4"
  

•   # Check 2mb chunk from the last 1MB of 2.5GB file - generic failure case
  

•   grep -A6 "Test Case 10 " "$1" | grep -q 'Error: "filesize" not defined'
  

•   pass_fail "TC10: load 2MB from the last 1MB of $4 - generic fail case"
  

•   # Check 1mb chunk write
  

•   grep -A3 "Test Case 11a " "$1" | \
  

•           egrep -q '1048576 bytes written|update journal'
  

•   pass_fail "TC11: 1MB write to $5 - write succeeded"
  

•   check_md5 "Test Case 11b " "$1" "$2" 1 \
  

•           "TC11: 1MB write to $5 - content verified"
  

•   echo "** End $1"
  

+}

+# ******************** +# * End of functions * +# ********************

+check_clean "$1" +check_prereq +compile_sandbox +prepare_env

+# Track TOTAL_FAIL and TOTAL_PASS +TOTAL_FAIL=0 +TOTAL_PASS=0

+# In each loop, for a given file system image, we test both the +# fs command, like load/size/write, the file system specific command +# like: ext4load/ext4size/ext4write and the sb load/ls/save commands. +for fs in ext4 fat; do

•   echo "Creating $fs image if not already present."
  

•   IMAGE=${IMG}.${fs}.img
  

•   MD5_FILE_FS="${MD5_FILE}.${fs}"
  

•   create_image $IMAGE $fs
  

•   # sb commands test
  

•   echo "Creating files in $fs image if not already present."
  

•   create_files $IMAGE $MD5_FILE_FS
  

•   # Lets mount the image and test sb hostfs commands
  

•   mkdir -p "$MOUNT_DIR"
  

•   if [ "$fs" = "fat" ]; then
  

•           uid="uid=`id -u`"
  

•   else
  

•           uid=""
  

•   fi
  

•   sudo mount -o loop,rw,$uid "$IMAGE" "$MOUNT_DIR"
  

•   sudo chmod 777 "$MOUNT_DIR"
  

•   OUT_FILE="${OUT}.sb.${fs}.out"
  

•   test_image $IMAGE $fs $SMALL_FILE $BIG_FILE sb `pwd`/$MOUNT_DIR \
  

•           > ${OUT_FILE}
  

•   sudo umount "$MOUNT_DIR"
  

•   rmdir "$MOUNT_DIR"
  

•   check_results $OUT_FILE $MD5_FILE_FS $SMALL_FILE $BIG_FILE \
  

•           $WRITE_FILE
  

•   TOTAL_FAIL=$((TOTAL_FAIL + FAIL))
  

•   TOTAL_PASS=$((TOTAL_PASS + PASS))
  

•   echo "Summary: PASS: $PASS FAIL: $FAIL"
  

•   echo "--------------------------------------------"
  

•   echo "Creating files in $fs image if not already present."
  

•   create_files $IMAGE $MD5_FILE_FS
  

•   OUT_FILE="${OUT}.${fs}.out"
  

•   test_image $IMAGE $fs $SMALL_FILE $BIG_FILE nonfs "" \
  

•           > $OUT_FILE
  

•   check_results $OUT_FILE $MD5_FILE_FS $SMALL_FILE $BIG_FILE \
  

•           $WRITE_FILE
  

•   TOTAL_FAIL=$((TOTAL_FAIL + FAIL))
  

•   TOTAL_PASS=$((TOTAL_PASS + PASS))
  

•   echo "Summary: PASS: $PASS FAIL: $FAIL"
  

•   echo "--------------------------------------------"
  

•   echo "Creating files in $fs image if not already present."
  

•   create_files $IMAGE $MD5_FILE_FS
  

•   OUT_FILE="${OUT}.fs.${fs}.out"
  

•   test_image $IMAGE $fs $SMALL_FILE $BIG_FILE fs "" \
  

•           > ${OUT_FILE}
  

•   check_results $OUT_FILE $MD5_FILE_FS $SMALL_FILE $BIG_FILE \
  

•           $WRITE_FILE
  

•   TOTAL_FAIL=$((TOTAL_FAIL + FAIL))
  

•   TOTAL_PASS=$((TOTAL_PASS + PASS))
  

•   echo "Summary: PASS: $PASS FAIL: $FAIL"
  

•   echo "--------------------------------------------"
  

You could perhaps put the common code in a function, and call it three times, as there is a lot of duplicate in these two instances.

...

+done

+echo "Total Summary: TOTAL PASS: $TOTAL_PASS TOTAL FAIL: $TOTAL_FAIL" +echo "--------------------------------------------" +if [ $TOTAL_FAIL -eq 0 ]; then

•   echo "PASSED"
  

•   exit 0
  

+else

•   echo "FAILED"
  

•   exit 1
  

+fi

1.9.1

Regards, Simon

On Mon 2014-11-03 18:49:58, Suriyan Ramasami wrote:

Change the interface for the generic FS functions to take in an extra parameter of type "loff_t *" to return the size. The return values of these funtions now serve as an indicator of error conditions alone.

Signed-off-by: Suriyan Ramasami suriyan.r@gmail.com

Changes in v6:

• Simon - Split this into a separate patch

Changes in v5:

• Simon - update fs.h with comments for fs_read/fs_write/fs_size

common/cmd_fs.c | 17 +++++++++++++ fs/fs.c | 77 ++++++++++++++++++++++++++++++++++----------------------- include/fs.h | 41 ++++++++++++++++++------------ 3 files changed, 88 insertions(+), 47 deletions(-)

diff --git a/common/cmd_fs.c b/common/cmd_fs.c index 6754340..f70cb8a 100644 --- a/common/cmd_fs.c +++ b/common/cmd_fs.c @@ -51,6 +51,23 @@ U_BOOT_CMD( " If 'pos' is 0 or omitted, the file is read from the start." )

+static int do_save_wrapper(cmd_tbl_t *cmdtp, int flag, int argc,

• 		char * const argv[])
  

+{

• return do_save(cmdtp, flag, argc, argv, FS_TYPE_ANY);

+}

+U_BOOT_CMD(

• save, 7, 0, do_save_wrapper,
• "save file to a filesystem",
• "<interface> [<dev[:part]> [<addr> [<filename> [bytes [pos]]]]]\n"

That's not correct, AFAICT.

int do_save(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], int fstype) { unsigned long addr; const char *filename; unsigned long bytes; unsigned long pos; int len; unsigned long time;

if (argc < 6 || argc > 7) return CMD_RET_USAGE;

More arguments are mandatory then you describe here.

• " - Save binary file 'filename' to partition 'part' on device\n"
• " type 'interface' instance 'dev' from addr 'addr' in memory.\n"
• " 'bytes' gives the size to save in bytes and is mandatory.\n"
• " 'pos' gives the file byte position to start writing to.\n"
• " If 'pos' is 0 or omitted, the file is written from the start."

Plus maybe note if it can extend existing files and create files if they don't exit would be nice here?

Hi,

On 3 November 2014 18:49, Suriyan Ramasami suriyan.r@gmail.com wrote:

Change the interface for the generic FS functions to take in an extra parameter of type "loff_t *" to return the size. The return values of these funtions now serve as an indicator of error conditions alone.

Signed-off-by: Suriyan Ramasami suriyan.r@gmail.com

This looks good apart from Pavel's comments and my two minor nits below.

---

Changes in v6:

• Simon - Split this into a separate patch

Changes in v5:

• Simon - update fs.h with comments for fs_read/fs_write/fs_size

common/cmd_fs.c | 17 +++++++++++++ fs/fs.c | 77 ++++++++++++++++++++++++++++++++++----------------------- include/fs.h | 41 ++++++++++++++++++------------ 3 files changed, 88 insertions(+), 47 deletions(-)

diff --git a/common/cmd_fs.c b/common/cmd_fs.c index 6754340..f70cb8a 100644 --- a/common/cmd_fs.c +++ b/common/cmd_fs.c @@ -51,6 +51,23 @@ U_BOOT_CMD( " If 'pos' is 0 or omitted, the file is read from the start." )

+static int do_save_wrapper(cmd_tbl_t *cmdtp, int flag, int argc,

•                           char * const argv[])
  

+{

•   return do_save(cmdtp, flag, argc, argv, FS_TYPE_ANY);
  

+}

+U_BOOT_CMD(

•   save,   7,      0,      do_save_wrapper,
  

•   "save file to a filesystem",
  

•   "<interface> [<dev[:part]> [<addr> [<filename> [bytes [pos]]]]]\n"
  

•   "    - Save binary file 'filename' to partition 'part' on device\n"
  

•   "      type 'interface' instance 'dev' from addr 'addr' in memory.\n"
  

•   "      'bytes' gives the size to save in bytes and is mandatory.\n"
  

•   "      'pos' gives the file byte position to start writing to.\n"
  

•   "      If 'pos' is 0 or omitted, the file is written from the start."
  

+)

static int do_ls_wrapper(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) { diff --git a/fs/fs.c b/fs/fs.c index dd680f3..0f5a1f4 100644 --- a/fs/fs.c +++ b/fs/fs.c @@ -46,19 +46,21 @@ static inline int fs_exists_unsupported(const char *filename) return 0; }

-static inline int fs_size_unsupported(const char *filename) +static inline int fs_size_unsupported(const char *filename, loff_t *size) { return -1; }

static inline int fs_read_unsupported(const char *filename, void *buf,

•                                 int offset, int len)
  

•                                 loff_t offset, loff_t len,
  

•                                 loff_t *actread)
  

{ return -1; }

static inline int fs_write_unsupported(const char *filename, void *buf,

•                                 int offset, int len)
  

•                                 loff_t offset, loff_t len,
  

•                                 loff_t *actwrite)
  

{ return -1; } @@ -82,9 +84,11 @@ struct fstype_info { disk_partition_t *fs_partition); int (*ls)(const char *dirname); int (*exists)(const char *filename);

•   int (*size)(const char *filename);
  

•   int (*read)(const char *filename, void *buf, int offset, int len);
  

•   int (*write)(const char *filename, void *buf, int offset, int len);
  

•   int (*size)(const char *filename, loff_t *size);
  

•   int (*read)(const char *filename, void *buf, loff_t offset,
  

•               loff_t len, loff_t *actread);
  

•   int (*write)(const char *filename, void *buf, loff_t offset,
  

•                loff_t len, loff_t *actwrite);
    void (*close)(void);
  

};

@@ -99,7 +103,11 @@ static struct fstype_info fstypes[] = { .exists = fat_exists, .size = fat_size, .read = fat_read_file, +#ifdef CONFIG_FAT_WRITE

•           .write = file_fat_write,
  

+#else .write = fs_write_unsupported, +#endif }, #endif #ifdef CONFIG_FS_EXT4 @@ -112,7 +120,11 @@ static struct fstype_info fstypes[] = { .exists = ext4fs_exists, .size = ext4fs_size, .read = ext4_read_file, +#ifdef CONFIG_CMD_EXT4_WRITE

•           .write = ext4_write_file,
  

+#else .write = fs_write_unsupported, +#endif }, #endif #ifdef CONFIG_SANDBOX @@ -233,20 +245,21 @@ int fs_exists(const char *filename) return ret; }

-int fs_size(const char *filename) +int fs_size(const char *filename, loff_t *size) { int ret;

    struct fstype_info *info = fs_get_info(fs_type);

•   ret = info->size(filename);
  

•   ret = info->size(filename, size);
  
    fs_close();
  
    return ret;
  

}

-int fs_read(const char *filename, ulong addr, int offset, int len) +int fs_read(const char *filename, ulong addr, loff_t offset, loff_t len,

•       loff_t *actread)
  

{ struct fstype_info *info = fs_get_info(fs_type); void *buf; @@ -257,11 +270,11 @@ int fs_read(const char *filename, ulong addr, int offset, int len) * means read the whole file. */ buf = map_sysmem(addr, len);

•   ret = info->read(filename, buf, offset, len);
  

•   ret = info->read(filename, buf, offset, len, actread);
    unmap_sysmem(buf);
  
    /* If we requested a specific number of bytes, check we got it */
  

•   if (ret >= 0 && len && ret != len) {
  

•   if (ret == 0 && len && *actread != len) {
            printf("** Unable to read file %s **\n", filename);
            ret = -1;
    }
  

@@ -270,17 +283,18 @@ int fs_read(const char *filename, ulong addr, int offset, int len) return ret; }

-int fs_write(const char *filename, ulong addr, int offset, int len) +int fs_write(const char *filename, ulong addr, loff_t offset, loff_t len,

•        loff_t *actwrite)
  

{ struct fstype_info *info = fs_get_info(fs_type); void *buf; int ret;

    buf = map_sysmem(addr, len);

•   ret = info->write(filename, buf, offset, len);
  

•   ret = info->write(filename, buf, offset, len, actwrite);
    unmap_sysmem(buf);
  
  

•   if (ret >= 0 && ret != len) {
  

•   if (ret < 0 && len != *actwrite) {
            printf("** Unable to write file %s **\n", filename);
            ret = -1;
    }
  

@@ -292,7 +306,7 @@ int fs_write(const char *filename, ulong addr, int offset, int len) int do_size(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], int fstype) {

•   int size;
  

•   loff_t size;
  
    if (argc != 4)
            return CMD_RET_USAGE;
  

@@ -300,8 +314,7 @@ int do_size(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], if (fs_set_blk_dev(argv[1], argv[2], fstype)) return 1;

•   size = fs_size(argv[3]);
  

•   if (size < 0)
  

•   if (fs_size(argv[3], &size) < 0)
            return CMD_RET_FAILURE;
  
    setenv_hex("filesize", size);
  

@@ -315,9 +328,10 @@ int do_load(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], unsigned long addr; const char *addr_str; const char *filename;

•   unsigned long bytes;
  

•   unsigned long pos;
  

•   int len_read;
  

•   loff_t bytes;
  

•   loff_t pos;
  

•   loff_t len_read;
  

•   int ret;
    unsigned long time;
    char *ep;
  
  

@@ -359,12 +373,12 @@ int do_load(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], pos = 0;

    time = get_timer(0);

•   len_read = fs_read(filename, addr, pos, bytes);
  

•   ret = fs_read(filename, addr, pos, bytes, &len_read);
    time = get_timer(time);
  

•   if (len_read <= 0)
  

•   if (ret < 0)
            return 1;
  
  

•   printf("%d bytes read in %lu ms", len_read, time);
  

•   printf("%llu bytes read in %lu ms", len_read, time);
    if (time > 0) {
            puts(" (");
            print_size(len_read / time * 1000, "/s");
  

@@ -408,9 +422,10 @@ int do_save(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], { unsigned long addr; const char *filename;

•   unsigned long bytes;
  

•   unsigned long pos;
  

•   int len;
  

•   loff_t bytes;
  

•   loff_t pos;
  

•   loff_t len;
  

•   int ret;
    unsigned long time;
  
    if (argc < 6 || argc > 7)
  

@@ -419,8 +434,8 @@ int do_save(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], if (fs_set_blk_dev(argv[1], argv[2], fstype)) return 1;

•   filename = argv[3];
  

•   addr = simple_strtoul(argv[4], NULL, 16);
  

•   addr = simple_strtoul(argv[3], NULL, 16);
  

•   filename = argv[4];
    bytes = simple_strtoul(argv[5], NULL, 16);
    if (argc >= 7)
            pos = simple_strtoul(argv[6], NULL, 16);
  

@@ -428,12 +443,12 @@ int do_save(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[], pos = 0;

    time = get_timer(0);

•   len = fs_write(filename, addr, pos, bytes);
  

•   ret = fs_write(filename, addr, pos, bytes, &len);
    time = get_timer(time);
  

•   if (len <= 0)
  

•   if (ret < 0)
            return 1;
  
  

•   printf("%d bytes written in %lu ms", len, time);
  

•   printf("%llu bytes written in %lu ms", len, time);
    if (time > 0) {
            puts(" (");
            print_size(len / time * 1000, "/s");
  

diff --git a/include/fs.h b/include/fs.h index 06a45f2..6bd17c8 100644 --- a/include/fs.h +++ b/include/fs.h @@ -51,32 +51,41 @@ int fs_ls(const char *dirname); int fs_exists(const char *filename);

/*

• • Determine a file's size

• • fs_size - Determine a file's size

• • Returns the file's size in bytes, or a negative value if it doesn't exist.

• • @filename: Name of the file
• • @size: Size of file
• • @return 0 if ok with valid *size, negative on error
  */

-int fs_size(const char *filename); +int fs_size(const char *filename, loff_t *size);

/*

• • Read file "filename" from the partition previously set by fs_set_blk_dev(),
• • to address "addr", starting at byte offset "offset", and reading "len"
• • bytes. "offset" may be 0 to read from the start of the file. "len" may be
• • 0 to read the entire file. Note that not all filesystem types support
• • either/both offset!=0 or len!=0.

• • fs_read - Read file from the partition previously set by fs_set_blk_dev()
• • Note that not all filesystem types support either/both offset!=0 or len!=0.

• • Returns number of bytes read on success. Returns <= 0 on error.

• • @filename: Name of file to read from
• • @addr: The address to read into
• • @offset: The offset in file to read from
• • @len: The number of bytes to read. Maybe 0 to read entire file
• • @actread: The actual number of bytes read

Returns the actual number...

• • @return 0 if ok with valid *actread, -1 on error conditions
  */

-int fs_read(const char *filename, ulong addr, int offset, int len); +int fs_read(const char *filename, ulong addr, loff_t offset, loff_t len,

•       loff_t *actread);
  
  

/*

• • Write file "filename" to the partition previously set by fs_set_blk_dev(),
• • from address "addr", starting at byte offset "offset", and writing "len"
• • bytes. "offset" may be 0 to write to the start of the file. Note that not
• • all filesystem types support offset!=0.

• • fs_write - Write file to the partition previously set by fs_set_blk_dev()
• • Note that not all filesystem types support offset!=0.

• • Returns number of bytes read on success. Returns <= 0 on error.

• • @filename: Name of file to read from
• • @addr: The address to read into
• • @offset: The offset in file to read from. Maybe 0 to write to start of file
• • @len: The number of bytes to write
• • @actwrite: The actual number of bytes written

Returns the actual number...

• • @return 0 if ok with valid *actwrite, -1 on error conditions
  */

-int fs_write(const char *filename, ulong addr, int offset, int len); +int fs_write(const char *filename, ulong addr, loff_t offset, loff_t len,

•        loff_t *actwrite);
  
  

/*

• Common implementation for various filesystem commands, optionally limited

-- 1.9.1

Regards, Simon

On 3 November 2014 18:50, Suriyan Ramasami suriyan.r@gmail.com wrote:

Enable md5sum to check the MD5 of the read and written files to check their contents for validity. Use map_sysmem() to map buffer in a sandbox environment.

Signed-off-by: Suriyan Ramasami suriyan.r@gmail.com

Acked-by: Simon Glass sjg@chromium.org

---

Changes in v6:

• Simon - Split this into a separate patch

Changes in v5: None

common/cmd_md5sum.c | 12 ++++++++++-- include/configs/sandbox.h | 2 ++ 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/common/cmd_md5sum.c b/common/cmd_md5sum.c index 3ac8cc4..d22ace5 100644 --- a/common/cmd_md5sum.c +++ b/common/cmd_md5sum.c @@ -11,6 +11,7 @@ #include <common.h> #include <command.h> #include <u-boot/md5.h> +#include <asm/io.h>

/*

• Store the resulting sum to an address or variable

@@ -79,6 +80,7 @@ int do_md5sum(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) int verify = 0; int ac; char * const *av;

•   void *buf;
  
    if (argc < 3)
            return CMD_RET_USAGE;
  

@@ -96,7 +98,9 @@ int do_md5sum(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) addr = simple_strtoul(*av++, NULL, 16); len = simple_strtoul(*av++, NULL, 16);

•   md5_wd((unsigned char *) addr, len, output, CHUNKSZ_MD5);
  

•   buf = map_sysmem(addr, len);
  

•   md5_wd(buf, len, output, CHUNKSZ_MD5);
  

•   unmap_sysmem(buf);
  
    if (!verify) {
            printf("md5 for %08lx ... %08lx ==> ", addr, addr + len - 1);
  

@@ -135,6 +139,7 @@ static int do_md5sum(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) unsigned long addr, len; unsigned int i; u8 output[16];

•   void *buf;
  
    if (argc < 3)
            return CMD_RET_USAGE;
  

@@ -142,7 +147,10 @@ static int do_md5sum(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[]) addr = simple_strtoul(argv[1], NULL, 16); len = simple_strtoul(argv[2], NULL, 16);

•   md5_wd((unsigned char *) addr, len, output, CHUNKSZ_MD5);
  

•   buf = map_sysmem(addr, len);
  

•   md5_wd(buf, len, output, CHUNKSZ_MD5);
  

•   unmap_sysmem(buf);
  

•   printf("md5 for %08lx ... %08lx ==> ", addr, addr + len - 1);
    for (i = 0; i < 16; i++)
            printf("%02x", output[i]);
  

diff --git a/include/configs/sandbox.h b/include/configs/sandbox.h index ee4b244..2b03841 100644 --- a/include/configs/sandbox.h +++ b/include/configs/sandbox.h @@ -48,6 +48,7 @@ #define CONFIG_ANDROID_BOOT_IMAGE

#define CONFIG_FS_FAT +#define CONFIG_FAT_WRITE #define CONFIG_FS_EXT4 #define CONFIG_EXT4_WRITE #define CONFIG_CMD_FAT @@ -57,6 +58,7 @@ #define CONFIG_DOS_PARTITION #define CONFIG_HOST_MAX_DEVICES 4 #define CONFIG_CMD_FS_GENERIC +#define CONFIG_CMD_MD5SUM

#define CONFIG_SYS_VSNPRINTF

-- 1.9.1