Re: [U-Boot] [PATCH 8/8] test/py: add DFU test

Hi Stephen,

From: Stephen Warren swarren@nvidia.com

Add a test of DFU functionality to the Python test suite. The test starts DFU in U-Boot, waits for USB device enumeration on the host, executes dfu-util multiple times to test various transfer sizes, many of which trigger USB driver edge cases, and finally aborts the DFU command in U-Boot.

This test mirrors the functionality previously available via the shell scripts in test/dfu, and hence those are removed too.

Cc: Lukasz Majewski l.majewski@majess.pl Signed-off-by: Stephen Warren swarren@nvidia.com

---

test/dfu/README | 44 ------- test/dfu/dfu_gadget_test.sh | 108 ---------------- test/dfu/dfu_gadget_test_init.sh | 45 ------- test/py/tests/test_dfu.py | 257 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+), 197 deletions(-) delete mode 100644 test/dfu/README delete mode 100755 test/dfu/dfu_gadget_test.sh delete mode 100755 test/dfu/dfu_gadget_test_init.sh create mode 100644 test/py/tests/test_dfu.py

diff --git a/test/dfu/README b/test/dfu/README deleted file mode 100644 index 408d5594219a..000000000000 --- a/test/dfu/README +++ /dev/null @@ -1,44 +0,0 @@ -DFU TEST CASE DESCRIPTION:

-The prerequisites for running this script are assured by -dfu_gadget_test_init.sh, which is automatically invoked by dfu_gadget_test.sh. -In this file user is able to generate their own set of test files by altering -the default set of TEST_FILES_SIZES variable. -The dfu_gadget_test_init.sh would generate test images only if they are not -already generated.

-On the target device, environment variable "dfu_alt_info" must contain at -least:

• dfu_test.bin fat 0 6;dfudummy.bin fat 0 6

-Depending on your device, you may need to replace "fat" with -"ext4", and "6" with the relevant partition number. For reference please -consult the config file for TRATS/TRATS2 devices -(../../include/configs/trats{2}.h)

-One can use fat, ext4 or any other supported file system supported by U-Boot. -These can be created by exporting storage devices via UMS (ums 0 mmc 0) and -using standard tools on host (like mkfs.ext4).

-Example usage: -1. On the target:

• setenv dfu_alt_info dfu_test.bin fat 0 6;dfudummy.bin fat 0 6
• dfu 0 mmc 0

-2. On the host:

• test/dfu/dfu_gadget_test.sh X Y [test file name] [usb device

vendor:product]

• e.g. test/dfu/dfu_gadget_test.sh 0 1
• or
• e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img
• or
• e.g. test/dfu/dfu_gadget_test.sh 0 1 0451:d022
• or
• e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img 0451:d022

-... where X and Y are dfu_test.bin's and dfudummy.bin's alt setting numbers. -They can be obtained from dfu-util -l or $dfu_alt_info. -It is also possible to pass optional [test file name] to force the script to -test one particular file. -If many DFU devices are connected, it may be useful to filter on USB -vendor/product ID (0451:d022). -One can get them by running "lsusb" command on a host PC. diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh deleted file mode 100755 index 9c7942257b44..000000000000 --- a/test/dfu/dfu_gadget_test.sh +++ /dev/null @@ -1,108 +0,0 @@ -#! /bin/bash

-# Copyright (C) 2014 Samsung Electronics -# Lukasz Majewski l.majewski@samsung.com -# -# Script fixes, enhancements and testing: -# Stephen Warren swarren@nvidia.com -# -# DFU operation test script -# -# SPDX-License-Identifier: GPL-2.0+

-set -e # any command return if not equal to zero -clear

-COLOUR_RED="\33[31m" -COLOUR_GREEN="\33[32m" -COLOUR_DEFAULT="\33[0m"

-DIR=./ -SUFFIX=img -RCV_DIR=rcv/ -LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`

-cd `dirname $0` -./dfu_gadget_test_init.sh

-cleanup () {

• rm -rf $DIR$RCV_DIR

-}

-die () {

• printf " $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
• cleanup
• exit 1

-}

-calculate_md5sum () {

• MD5SUM=`md5sum $1`
• MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
• echo "md5sum:"$MD5SUM

-}

-dfu_test_file () {

• printf "$COLOUR_GREEN

========================================================================================= $COLOUR_DEFAULT\n"

• printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
• dfu-util $USB_DEV -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1

|| die $? -

• echo -n "TX: "
• calculate_md5sum $1
• MD5_TX=$MD5SUM
• dfu-util $USB_DEV -D ${DIR}/dfudummy.bin -a

$TARGET_ALT_SETTING_B >> $LOG_FILE 2>&1 || die $? -

• N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
• dfu-util $USB_DEV -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE

2>&1 || die $? -

• echo -n "RX: "
• calculate_md5sum $N_FILE
• MD5_RX=$MD5SUM
• if [ "$MD5_TX" == "$MD5_RX" ]; then
• printf " $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
• else
• printf " $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
• cleanup
• exit 1
• fi

-}

-printf "$COLOUR_GREEN========================================================================================= $COLOUR_DEFAULT\n" -echo "DFU EP0 transmission test program" -echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver" -echo "@ -> TRATS2 # dfu 0 mmc 0" -cleanup -mkdir -p $DIR$RCV_DIR -touch $LOG_FILE

-if [ $# -eq 0 ] -then

• printf " $COLOUR_RED Please pass alt setting number!!

$COLOUR_DEFAULT \n"

• exit 0

-fi

-TARGET_ALT_SETTING=$1 -TARGET_ALT_SETTING_B=$2

-file=$3 -[[ $3 == *':'* ]] && USB_DEV="-d $3" && file="" -[ $# -eq 4 ] && USB_DEV="-d $4"

-if [ -n "$file" ] -then

• dfu_test_file $file

-else

• for f in $DIR*.$SUFFIX
• do

•    dfu_test_file $f
  

• done

-fi

-cleanup

-exit 0 diff --git a/test/dfu/dfu_gadget_test_init.sh b/test/dfu/dfu_gadget_test_init.sh deleted file mode 100755 index 640628eecb7a..000000000000 --- a/test/dfu/dfu_gadget_test_init.sh +++ /dev/null @@ -1,45 +0,0 @@ -#! /bin/bash

-# Copyright (C) 2014 Samsung Electronics -# Lukasz Majewski l.majewski@samsung.com -# -# Script fixes, enhancements and testing: -# Stephen Warren swarren@nvidia.com -# -# Script for test files generation -# -# SPDX-License-Identifier: GPL-2.0+

-set -e # any command return if not equal to zero -clear

-COLOUR_RED="\33[31m" -COLOUR_GREEN="\33[32m" -COLOUR_DEFAULT="\33[0m"

-LOG_DIR="./log"

-if [ $# -eq 0 ]; then

• TEST_FILES_SIZES="63 64 65 127 128 129 4095 4096 4097 959 960

961 1048575 1048576 8M" -else

• TEST_FILES_SIZES=$@

-fi

-printf "Init script for generating data necessary for DFU test script" - -if [ ! -d $LOG_DIR ]; then

• `mkdir $LOG_DIR`

-fi

-for size in $TEST_FILES_SIZES -do

• FILE="./dat_$size.img"
• if [ ! -f $FILE ]; then
• dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1

...

/dev/null 2>&1 || exit $?

• fi

-done -dd if=/dev/urandom of="./dfudummy.bin" bs=1024 count=1 > /dev/null 2>&1 || exit $? - -printf "$COLOUR_GREEN OK $COLOUR_DEFAULT \n"

-exit 0 diff --git a/test/py/tests/test_dfu.py b/test/py/tests/test_dfu.py new file mode 100644 index 000000000000..e86ea9a1887c --- /dev/null +++ b/test/py/tests/test_dfu.py @@ -0,0 +1,257 @@ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0

+# Test U-Boot's "ums" command. At present, this test only ensures that a UMS +# device can be enumerated by the host/test machine. In the future, this test +# should be enhanced to validate disk IO.

Please update this comment to regard dfu, not ums.

+import os +import os.path +import pytest +import u_boot_utils

+''' +Note: This test relies on:

+a) boardenv_* to contain configuration values to define which USB ports are +available for testing. Without this, this test will be automatically skipped. +For example:

+env__usb_dev_ports = (

• {

•    "tgt_usb_ctlr": "0",
  

•    "host_usb_dev_node": "/dev/usbdev-p2371-2180",
  

•    # This parameter is optional /if/ you only have a single
  

board

•    # attached to your host at a time.
  

•    "host_usb_port_path": "3-13",
  

• },

+)

+env__dfu_configs = (

• # eMMC, partition 1
• {

•    "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
  

•    "cmd_params": "mmc 0",
  

• },

+) +b) udev rules to set permissions on devices nodes, so that sudo is not +required. For example:

+ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="666" + +(You may wish to change the group ID instead of setting the permissions wide +open. All that matters is that the user ID running the test can access the +device.) +'''

+# The set of file sizes to test. These values trigger various edge-cases such +# as one less than, equal to, and one greater than typical USB max packet +# sizes, and similar boundary conditions. +test_sizes = (

• 64 - 1,
• 64,
• 64 + 1,
• 128 - 1,
• 128,
• 128 + 1,
• 960 - 1,
• 960,
• 960 + 1,
• 4096 - 1,
• 4096,
• 4096 + 1,
• 1024 * 1024 - 1,
• 1024 * 1024,
• 8 * 1024 * 1024,

+)

+first_usb_dev_port = None

+@pytest.mark.buildconfigspec('cmd_dfu') +def test_dfu(u_boot_console, env__usb_dev_port, env__dfu_config):

• '''Test DFU functionality, using numerous file sizes.
• Args:

•    u_boot_console: A U-Boot console connection.
  

•    env__usb_dev_port: The single USB device-mode port
  

specification on

•        which to run the test.
  

•    env__dfu_config: The single DFU (memory region)
  

configuration on which

•        to run the test.
  

• Returns:

•    Nothing.
  

• '''
• def start_dfu():

•    '''Start U-Boot's dfu shell command.
  

•    This also waits for the host-side USB enumeration process to
  

complete. +

•    Args:
  

•        None.
  

•    Returns:
  

•        Nothing.
  

•    '''
  

•    u_boot_console.log.action(
  

•        'Starting long-running U-Boot dfu shell command')
  

•    cmd = 'setenv dfu_alt_info "%s"' %
  

env__dfu_config['alt_info']

•    u_boot_console.run_command(cmd)
  

•    cmd = 'dfu 0 ' + env__dfu_config['cmd_params']
  

•    u_boot_console.run_command(cmd, wait_for_prompt=False)
  

•    u_boot_console.log.action('Waiting for DFU USB device to
  

appear')

•    fh = u_boot_utils.wait_until_open_succeeds(
  

•        env__usb_dev_port['host_usb_dev_node'])
  

•    fh.close()
  

• def stop_dfu(ignore_errors):

•    '''Stop U-Boot's dfu shell command from executing.
  

•    This also waits for the host-side USB de-enumeration process
  

to

•    complete.
  

•    Args:
  

•        ignore_errors: Ignore any errors. This is useful if an
  

error has

•            already been detected, and the code is performing
  

best-effort

•            cleanup. In this case, we do not want to mask the
  

original

•            error by "honoring" any new errors.
  

•    Returns:
  

•        Nothing.
  

•    '''
  

•    try:
  

•        u_boot_console.log.action(
  

•            'Stopping long-running U-Boot dfu shell command')
  

•        u_boot_console.ctrlc()
  

•        u_boot_console.log.action(
  

•            'Waiting for DFU USB device to disappear')
  

•        u_boot_utils.wait_until_file_open_fails(
  

•            env__usb_dev_port['host_usb_dev_node'],
  

ignore_errors)

•    except:
  

•        if not ignore_errors:
  

•            raise
  

• def run_dfu_util(alt_setting, fn, up_dn_load_arg):

•    '''Invoke dfu-util on the host.
  

•    Args:
  

•        alt_setting: The DFU "alternate setting" identifier to
  

interact

•            with.
  

•        fn: The host-side file name to transfer.
  

•        up_dn_load_arg: '-U' or '-D' depending on whether a DFU
  

upload or

•            download operation should be performed.
  

•    Returns:
  

•        Nothing.
  

•    '''
  

•    cmd = ['dfu-util', '-a', str(alt_setting), up_dn_load_arg,
  

fn]

•    if 'host_usb_port_path' in env__usb_dev_port:
  

•        cmd += ['-p', env__usb_dev_port['host_usb_port_path']]
  

•    u_boot_utils.run_and_log(u_boot_console, cmd)
  

•    u_boot_console.wait_for('Ctrl+C to exit ...')
  

• def dfu_write(alt_setting, fn):

•    '''Write a file to the target board using DFU.
  

•    Args:
  

•        alt_setting: The DFU "alternate setting" identifier to
  

interact

•            with.
  

•        fn: The host-side file name to transfer.
  

•    Returns:
  

•        Nothing.
  

•    '''
  

•    run_dfu_util(alt_setting, fn, '-D')
  

• def dfu_read(alt_setting, fn):

•    '''Read a file from the target board using DFU.
  

•    Args:
  

•        alt_setting: The DFU "alternate setting" identifier to
  

interact

•            with.
  

•        fn: The host-side file name to transfer.
  

•    Returns:
  

•        Nothing.
  

•    '''
  

•    # dfu-util fails reads/uploads if the host file already
  

exists

•    if os.path.exists(fn):
  

•        os.remove(fn)
  

•    run_dfu_util(alt_setting, fn, '-U')
  

• def dfu_write_read_check(size):

•    '''Test DFU transfers of a specific size of data
  

•    This function first writes data to the board then reads it
  

back and

•    compares the written and read back data. Measures are taken
  

to avoid

•    certain types of false positives.
  

•    Args:
  

•        size: The data size to test.
  

•    Returns:
  

•        Nothing.
  

•    '''
  

•    test_f = u_boot_utils.PersistentRandomFile(u_boot_console,
  

•        'dfu_%d.bin' % size, size)
  

•    readback_fn = u_boot_console.config.result_dir +
  

'/dfu_readback.bin' +

•    u_boot_console.log.action('Writing test data to DFU primary
  

' +

•        'altsetting')
  

•    dfu_write(0, test_f.abs_fn)
  

•    u_boot_console.log.action('Writing dummy data to DFU
  

secondary ' +

•        'altsetting to clear DFU buffers')
  

•    dfu_write(1, dummy_f.abs_fn)
  

•    u_boot_console.log.action('Reading DFU primary altsetting
  

for ' +

•        'comparison')
  

•    dfu_read(0, readback_fn)
  

•    u_boot_console.log.action('Comparing written and read data')
  

•    written_hash = test_f.content_hash
  

•    read_back_hash = u_boot_utils.md5sum_file(readback_fn, size)
  

•    assert(written_hash == read_back_hash)
  

• # This test may be executed against multiple USB ports. The test

takes a

• # long time, so we don't want to do the whole thing each time.

Instead,

• # execute the full test on the first USB port, and perform a

very limited

• # test on other ports. In the limited case, we solely validate

that the

• # host PC can enumerate the U-Boot USB device.
• global first_usb_dev_port
• if not first_usb_dev_port:

•    first_usb_dev_port = env__usb_dev_port
  

• if env__usb_dev_port == first_usb_dev_port:

•    sizes = test_sizes
  

• else:

•    sizes = []
  

• dummy_f = u_boot_utils.PersistentRandomFile(u_boot_console,

•    'dfu_dummy.bin', 1024)
  

• ignore_cleanup_errors = True
• try:

•    start_dfu()
  

•    u_boot_console.log.action(
  

•        'Overwriting DFU primary altsetting with dummy data')
  

•    dfu_write(0, dummy_f.abs_fn)
  

•    for size in sizes:
  

•        with u_boot_console.log.section("Data size %d" % size):
  

•            dfu_write_read_check(size)
  

•            # Make the status of each sub-test obvious. If the
  

test didn't

•            # pass, an exception was thrown so this code isn't
  

executed.

•            u_boot_console.log.status_pass('OK')
  

•    ignore_cleanup_errors = False
  

• finally:

•    stop_dfu(ignore_cleanup_errors)
  

Acked-by: Lukasz Majewski l.majewski@samsung.com

Great work Stephen, Thanks !