From: Stephen Warren swarren@nvidia.com
Add a test of DFU functionality to the Python test suite. The test starts DFU in U-Boot, waits for USB device enumeration on the host, executes dfu-util multiple times to test various transfer sizes, many of which trigger USB driver edge cases, and finally aborts the DFU command in U-Boot.
This test mirrors the functionality previously available via the shell scripts in test/dfu, and hence those are removed too.
Cc: Lukasz Majewski l.majewski@majess.pl Signed-off-by: Stephen Warren swarren@nvidia.com --- test/dfu/README | 44 ------- test/dfu/dfu_gadget_test.sh | 108 ---------------- test/dfu/dfu_gadget_test_init.sh | 45 ------- test/py/tests/test_dfu.py | 257 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+), 197 deletions(-) delete mode 100644 test/dfu/README delete mode 100755 test/dfu/dfu_gadget_test.sh delete mode 100755 test/dfu/dfu_gadget_test_init.sh create mode 100644 test/py/tests/test_dfu.py
diff --git a/test/dfu/README b/test/dfu/README deleted file mode 100644 index 408d5594219a..000000000000 --- a/test/dfu/README +++ /dev/null @@ -1,44 +0,0 @@ -DFU TEST CASE DESCRIPTION: - -The prerequisites for running this script are assured by -dfu_gadget_test_init.sh, which is automatically invoked by dfu_gadget_test.sh. -In this file user is able to generate their own set of test files by altering -the default set of TEST_FILES_SIZES variable. -The dfu_gadget_test_init.sh would generate test images only if they are not -already generated. - -On the target device, environment variable "dfu_alt_info" must contain at -least: - - dfu_test.bin fat 0 6;dfudummy.bin fat 0 6 - -Depending on your device, you may need to replace "fat" with -"ext4", and "6" with the relevant partition number. For reference please -consult the config file for TRATS/TRATS2 devices -(../../include/configs/trats{2}.h) - -One can use fat, ext4 or any other supported file system supported by U-Boot. -These can be created by exporting storage devices via UMS (ums 0 mmc 0) and -using standard tools on host (like mkfs.ext4). - -Example usage: -1. On the target: - setenv dfu_alt_info dfu_test.bin fat 0 6;dfudummy.bin fat 0 6 - dfu 0 mmc 0 -2. On the host: - test/dfu/dfu_gadget_test.sh X Y [test file name] [usb device vendor:product] - e.g. test/dfu/dfu_gadget_test.sh 0 1 - or - e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img - or - e.g. test/dfu/dfu_gadget_test.sh 0 1 0451:d022 - or - e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img 0451:d022 - -... where X and Y are dfu_test.bin's and dfudummy.bin's alt setting numbers. -They can be obtained from dfu-util -l or $dfu_alt_info. -It is also possible to pass optional [test file name] to force the script to -test one particular file. -If many DFU devices are connected, it may be useful to filter on USB -vendor/product ID (0451:d022). -One can get them by running "lsusb" command on a host PC. diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh deleted file mode 100755 index 9c7942257b44..000000000000 --- a/test/dfu/dfu_gadget_test.sh +++ /dev/null @@ -1,108 +0,0 @@ -#! /bin/bash - -# Copyright (C) 2014 Samsung Electronics -# Lukasz Majewski l.majewski@samsung.com -# -# Script fixes, enhancements and testing: -# Stephen Warren swarren@nvidia.com -# -# DFU operation test script -# -# SPDX-License-Identifier: GPL-2.0+ - -set -e # any command return if not equal to zero -clear - -COLOUR_RED="\33[31m" -COLOUR_GREEN="\33[32m" -COLOUR_DEFAULT="\33[0m" - -DIR=./ -SUFFIX=img -RCV_DIR=rcv/ -LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S` - -cd `dirname $0` -./dfu_gadget_test_init.sh - -cleanup () { - rm -rf $DIR$RCV_DIR -} - -die () { - printf " $COLOUR_RED FAILED $COLOUR_DEFAULT \n" - cleanup - exit 1 -} - -calculate_md5sum () { - MD5SUM=`md5sum $1` - MD5SUM=`echo $MD5SUM | cut -d ' ' -f1` - echo "md5sum:"$MD5SUM -} - -dfu_test_file () { - printf "$COLOUR_GREEN ========================================================================================= $COLOUR_DEFAULT\n" - printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1 - - dfu-util $USB_DEV -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $? - - echo -n "TX: " - calculate_md5sum $1 - - MD5_TX=$MD5SUM - - dfu-util $USB_DEV -D ${DIR}/dfudummy.bin -a $TARGET_ALT_SETTING_B >> $LOG_FILE 2>&1 || die $? - - N_FILE=$DIR$RCV_DIR${1:2}"_rcv" - - dfu-util $USB_DEV -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $? - - echo -n "RX: " - calculate_md5sum $N_FILE - MD5_RX=$MD5SUM - - if [ "$MD5_TX" == "$MD5_RX" ]; then - printf " $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n" - else - printf " $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n" - cleanup - exit 1 - fi - -} - -printf "$COLOUR_GREEN========================================================================================= $COLOUR_DEFAULT\n" -echo "DFU EP0 transmission test program" -echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver" -echo "@ -> TRATS2 # dfu 0 mmc 0" -cleanup -mkdir -p $DIR$RCV_DIR -touch $LOG_FILE - -if [ $# -eq 0 ] -then - printf " $COLOUR_RED Please pass alt setting number!! $COLOUR_DEFAULT \n" - exit 0 -fi - -TARGET_ALT_SETTING=$1 -TARGET_ALT_SETTING_B=$2 - -file=$3 -[[ $3 == *':'* ]] && USB_DEV="-d $3" && file="" -[ $# -eq 4 ] && USB_DEV="-d $4" - -if [ -n "$file" ] -then - dfu_test_file $file -else - for f in $DIR*.$SUFFIX - do - dfu_test_file $f - done -fi - -cleanup - -exit 0 diff --git a/test/dfu/dfu_gadget_test_init.sh b/test/dfu/dfu_gadget_test_init.sh deleted file mode 100755 index 640628eecb7a..000000000000 --- a/test/dfu/dfu_gadget_test_init.sh +++ /dev/null @@ -1,45 +0,0 @@ -#! /bin/bash - -# Copyright (C) 2014 Samsung Electronics -# Lukasz Majewski l.majewski@samsung.com -# -# Script fixes, enhancements and testing: -# Stephen Warren swarren@nvidia.com -# -# Script for test files generation -# -# SPDX-License-Identifier: GPL-2.0+ - -set -e # any command return if not equal to zero -clear - -COLOUR_RED="\33[31m" -COLOUR_GREEN="\33[32m" -COLOUR_DEFAULT="\33[0m" - -LOG_DIR="./log" - -if [ $# -eq 0 ]; then - TEST_FILES_SIZES="63 64 65 127 128 129 4095 4096 4097 959 960 961 1048575 1048576 8M" -else - TEST_FILES_SIZES=$@ -fi - -printf "Init script for generating data necessary for DFU test script" - -if [ ! -d $LOG_DIR ]; then - `mkdir $LOG_DIR` -fi - -for size in $TEST_FILES_SIZES -do - FILE="./dat_$size.img" - if [ ! -f $FILE ]; then - dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1 > /dev/null 2>&1 || exit $? - fi -done -dd if=/dev/urandom of="./dfudummy.bin" bs=1024 count=1 > /dev/null 2>&1 || exit $? - -printf "$COLOUR_GREEN OK $COLOUR_DEFAULT \n" - -exit 0 diff --git a/test/py/tests/test_dfu.py b/test/py/tests/test_dfu.py new file mode 100644 index 000000000000..e86ea9a1887c --- /dev/null +++ b/test/py/tests/test_dfu.py @@ -0,0 +1,257 @@ +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Test U-Boot's "ums" command. At present, this test only ensures that a UMS +# device can be enumerated by the host/test machine. In the future, this test +# should be enhanced to validate disk IO. + +import os +import os.path +import pytest +import u_boot_utils + +''' +Note: This test relies on: + +a) boardenv_* to contain configuration values to define which USB ports are +available for testing. Without this, this test will be automatically skipped. +For example: + +env__usb_dev_ports = ( + { + "tgt_usb_ctlr": "0", + "host_usb_dev_node": "/dev/usbdev-p2371-2180", + # This parameter is optional /if/ you only have a single board + # attached to your host at a time. + "host_usb_port_path": "3-13", + }, +) + +env__dfu_configs = ( + # eMMC, partition 1 + { + "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1", + "cmd_params": "mmc 0", + }, +) +b) udev rules to set permissions on devices nodes, so that sudo is not +required. For example: + +ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="666" + +(You may wish to change the group ID instead of setting the permissions wide +open. All that matters is that the user ID running the test can access the +device.) +''' + +# The set of file sizes to test. These values trigger various edge-cases such +# as one less than, equal to, and one greater than typical USB max packet +# sizes, and similar boundary conditions. +test_sizes = ( + 64 - 1, + 64, + 64 + 1, + 128 - 1, + 128, + 128 + 1, + 960 - 1, + 960, + 960 + 1, + 4096 - 1, + 4096, + 4096 + 1, + 1024 * 1024 - 1, + 1024 * 1024, + 8 * 1024 * 1024, +) + +first_usb_dev_port = None + +@pytest.mark.buildconfigspec('cmd_dfu') +def test_dfu(u_boot_console, env__usb_dev_port, env__dfu_config): + '''Test DFU functionality, using numerous file sizes. + + Args: + u_boot_console: A U-Boot console connection. + env__usb_dev_port: The single USB device-mode port specification on + which to run the test. + env__dfu_config: The single DFU (memory region) configuration on which + to run the test. + + Returns: + Nothing. + ''' + + def start_dfu(): + '''Start U-Boot's dfu shell command. + + This also waits for the host-side USB enumeration process to complete. + + Args: + None. + + Returns: + Nothing. + ''' + + u_boot_console.log.action( + 'Starting long-running U-Boot dfu shell command') + + cmd = 'setenv dfu_alt_info "%s"' % env__dfu_config['alt_info'] + u_boot_console.run_command(cmd) + + cmd = 'dfu 0 ' + env__dfu_config['cmd_params'] + u_boot_console.run_command(cmd, wait_for_prompt=False) + u_boot_console.log.action('Waiting for DFU USB device to appear') + fh = u_boot_utils.wait_until_open_succeeds( + env__usb_dev_port['host_usb_dev_node']) + fh.close() + + def stop_dfu(ignore_errors): + '''Stop U-Boot's dfu shell command from executing. + + This also waits for the host-side USB de-enumeration process to + complete. + + Args: + ignore_errors: Ignore any errors. This is useful if an error has + already been detected, and the code is performing best-effort + cleanup. In this case, we do not want to mask the original + error by "honoring" any new errors. + + Returns: + Nothing. + ''' + + try: + u_boot_console.log.action( + 'Stopping long-running U-Boot dfu shell command') + u_boot_console.ctrlc() + u_boot_console.log.action( + 'Waiting for DFU USB device to disappear') + u_boot_utils.wait_until_file_open_fails( + env__usb_dev_port['host_usb_dev_node'], ignore_errors) + except: + if not ignore_errors: + raise + + def run_dfu_util(alt_setting, fn, up_dn_load_arg): + '''Invoke dfu-util on the host. + + Args: + alt_setting: The DFU "alternate setting" identifier to interact + with. + fn: The host-side file name to transfer. + up_dn_load_arg: '-U' or '-D' depending on whether a DFU upload or + download operation should be performed. + + Returns: + Nothing. + ''' + + cmd = ['dfu-util', '-a', str(alt_setting), up_dn_load_arg, fn] + if 'host_usb_port_path' in env__usb_dev_port: + cmd += ['-p', env__usb_dev_port['host_usb_port_path']] + u_boot_utils.run_and_log(u_boot_console, cmd) + u_boot_console.wait_for('Ctrl+C to exit ...') + + def dfu_write(alt_setting, fn): + '''Write a file to the target board using DFU. + + Args: + alt_setting: The DFU "alternate setting" identifier to interact + with. + fn: The host-side file name to transfer. + + Returns: + Nothing. + ''' + + run_dfu_util(alt_setting, fn, '-D') + + def dfu_read(alt_setting, fn): + '''Read a file from the target board using DFU. + + Args: + alt_setting: The DFU "alternate setting" identifier to interact + with. + fn: The host-side file name to transfer. + + Returns: + Nothing. + ''' + + # dfu-util fails reads/uploads if the host file already exists + if os.path.exists(fn): + os.remove(fn) + run_dfu_util(alt_setting, fn, '-U') + + def dfu_write_read_check(size): + '''Test DFU transfers of a specific size of data + + This function first writes data to the board then reads it back and + compares the written and read back data. Measures are taken to avoid + certain types of false positives. + + Args: + size: The data size to test. + + Returns: + Nothing. + ''' + + test_f = u_boot_utils.PersistentRandomFile(u_boot_console, + 'dfu_%d.bin' % size, size) + readback_fn = u_boot_console.config.result_dir + '/dfu_readback.bin' + + u_boot_console.log.action('Writing test data to DFU primary ' + + 'altsetting') + dfu_write(0, test_f.abs_fn) + + u_boot_console.log.action('Writing dummy data to DFU secondary ' + + 'altsetting to clear DFU buffers') + dfu_write(1, dummy_f.abs_fn) + + u_boot_console.log.action('Reading DFU primary altsetting for ' + + 'comparison') + dfu_read(0, readback_fn) + + u_boot_console.log.action('Comparing written and read data') + written_hash = test_f.content_hash + read_back_hash = u_boot_utils.md5sum_file(readback_fn, size) + assert(written_hash == read_back_hash) + + # This test may be executed against multiple USB ports. The test takes a + # long time, so we don't want to do the whole thing each time. Instead, + # execute the full test on the first USB port, and perform a very limited + # test on other ports. In the limited case, we solely validate that the + # host PC can enumerate the U-Boot USB device. + global first_usb_dev_port + if not first_usb_dev_port: + first_usb_dev_port = env__usb_dev_port + if env__usb_dev_port == first_usb_dev_port: + sizes = test_sizes + else: + sizes = [] + + dummy_f = u_boot_utils.PersistentRandomFile(u_boot_console, + 'dfu_dummy.bin', 1024) + + ignore_cleanup_errors = True + try: + start_dfu() + + u_boot_console.log.action( + 'Overwriting DFU primary altsetting with dummy data') + dfu_write(0, dummy_f.abs_fn) + + for size in sizes: + with u_boot_console.log.section("Data size %d" % size): + dfu_write_read_check(size) + # Make the status of each sub-test obvious. If the test didn't + # pass, an exception was thrown so this code isn't executed. + u_boot_console.log.status_pass('OK') + ignore_cleanup_errors = False + finally: + stop_dfu(ignore_cleanup_errors)