January 2016 - U-Boot

[U-Boot] [PATCH V3 1/7] test/py: Implement pytest infrastructure
by Stephen Warren 11 Jan '16

11 Jan '16

This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are: - Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect. - There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C. - It is reasonably simple to interact with U-Boot in this way. A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too. The hook scripts, relay control utilities, and udev rules I use for my own HW setup are published at https://github.com/swarren/uboot-test-hooks. See README.md for more details! Signed-off-by: Stephen Warren <swarren(a)wwwdotorg.org> Signed-off-by: Stephen Warren <swarren(a)nvidia.com> Tested-by: Michal Simek <michal.simek(a)xilinx.com> Tested-by: Simon Glass <sjg(a)chromium.org> --- v3: - Rework HTML log generation so that TAB characters render as expected. Suggested by Michal Simek. - Move test scripts into a sub-directory. Suggested by Michal Simek. - s/uboot/u[-_]boot/g. Suggested by Simon Glass. - s/"/'/g. Suggested by Simon Glass. - Typo fixes. - Add more documentation. Suggested by Simon Glass. - Make "notes" in the log file be <pre> so that their formatting is preserved. This is useful for large notes such as exception dumps. v2: - Many fixes and tweaks have been squashed in. Separated out some of the tests into separate commits, and added some more tests. --- test/py/.gitignore | 1 + test/py/README.md | 300 ++++++++++++++++++++++++++++++ test/py/conftest.py | 335 ++++++++++++++++++++++++++++++++++ test/py/multiplexed_log.css | 88 +++++++++ test/py/multiplexed_log.py | 335 ++++++++++++++++++++++++++++++++++ test/py/pytest.ini | 11 ++ test/py/test.py | 32 ++++ test/py/tests/test_000_version.py | 20 ++ test/py/tests/test_help.py | 9 + test/py/tests/test_unknown_cmd.py | 14 ++ test/py/u_boot_console_base.py | 260 ++++++++++++++++++++++++++ test/py/u_boot_console_exec_attach.py | 51 ++++++ test/py/u_boot_console_sandbox.py | 48 +++++ test/py/u_boot_spawn.py | 123 +++++++++++++ 14 files changed, 1627 insertions(+) create mode 100644 test/py/.gitignore create mode 100644 test/py/README.md create mode 100644 test/py/conftest.py create mode 100644 test/py/multiplexed_log.css create mode 100644 test/py/multiplexed_log.py create mode 100644 test/py/pytest.ini create mode 100755 test/py/test.py create mode 100644 test/py/tests/test_000_version.py create mode 100644 test/py/tests/test_help.py create mode 100644 test/py/tests/test_unknown_cmd.py create mode 100644 test/py/u_boot_console_base.py create mode 100644 test/py/u_boot_console_exec_attach.py create mode 100644 test/py/u_boot_console_sandbox.py create mode 100644 test/py/u_boot_spawn.py diff --git a/test/py/.gitignore b/test/py/.gitignore new file mode 100644 index 000000000000..0d20b6487c61 --- /dev/null +++ b/test/py/.gitignore @@ -0,0 +1 @@ +*.pyc diff --git a/test/py/README.md b/test/py/README.md new file mode 100644 index 000000000000..7c9bb4f04c24 --- /dev/null +++ b/test/py/README.md @@ -0,0 +1,300 @@ +# U-Boot pytest suite + +## Introduction + +This tool aims to test U-Boot by executing U-Boot shell commands using the +console interface. A single top-level script exists to execute or attach to the +U-Boot console, run the entire script of tests against it, and summarize the +results. Advantages of this approach are: + +- Testing is performed in the same way a user or script would interact with + U-Boot; there can be no disconnect. +- There is no need to write or embed test-related code into U-Boot itself. + It is asserted that writing test-related code in Python is simpler and more + flexible that writing it all in C. +- It is reasonably simple to interact with U-Boot in this way. + +## Requirements + +The test suite is implemented using pytest. Interaction with the U-Boot console +involves executing some binary and interacting with its stdin/stdout. You will +need to implement various "hook" scripts that are called by the test suite at +the appropriate time. + +On Debian or Debian-like distributions, the following packages are required. +Similar package names should exist in other distributions. + +| Package | Version tested (Ubuntu 14.04) | +| -------------- | ----------------------------- | +| python | 2.7.5-5ubuntu3 | +| python-pytest | 2.5.1-1 | + +The test script supports either: + +- Executing a sandbox port of U-Boot on the local machine as a sub-process, + and interacting with it over stdin/stdout. +- Executing an external "hook" scripts to flash a U-Boot binary onto a + physical board, attach to the board's console stream, and reset the board. + Further details are described later. + +### Using `virtualenv` to provide requirements + +Older distributions (e.g. Ubuntu 10.04) may not provide all the required +packages, or may provide versions that are too old to run the test suite. One +can use the Python `virtualenv` script to locally install more up-to-date +versions of the required packages without interfering with the OS installation. +For example: + +```bash +$ cd /path/to/u-boot +$ sudo apt-get install python python-virtualenv +$ virtualenv venv +$ . ./venv/bin/activate +$ pip install pytest +``` + +## Testing sandbox + +To run the testsuite on the sandbox port (U-Boot built as a native user-space +application), simply execute: + +``` +./test/py/test.py --bd sandbox --build +``` + +The `--bd` option tells the test suite which board type is being tested. This +lets the test suite know which features the board has, and hence exactly what +can be tested. + +The `--build` option tells U-Boot to compile U-Boot. Alternatively, you may +omit this option and build U-Boot yourself, in whatever way you choose, before +running the test script. + +The test script will attach to U-Boot, execute all valid tests for the board, +then print a summary of the test process. A complete log of the test session +will be written to `${build_dir}/test-log.html`. This is best viewed in a web +browser, but may be read directly as plain text, perhaps with the aid of the +`html2text` utility. + +## Command-line options + +- `--board-type`, `--bd`, `-B` set the type of the board to be tested. For + example, `sandbox` or `seaboard`. +- `--board-identity`, `--id` set the identity of the board to be tested. + This allows differentiation between multiple instances of the same type of + physical board that are attached to the same host machine. This parameter is + not interpreted by the test script in any way, but rather is simply passed + to the hook scripts described below, and may be used in any site-specific + way deemed necessary. +- `--build` indicates that the test script should compile U-Boot itself + before running the tests. If using this option, make sure that any + environment variables required by the build process are already set, such as + `$CROSS_COMPILE`. +- `--build-dir` sets the directory containing the compiled U-Boot binaries. + If omitted, this is `${source_dir}/build-${board_type}`. +- `--result-dir` sets the directory to write results, such as log files, + into. If omitted, the build directory is used. +- `--persistent-data-dir` sets the directory used to store persistent test + data. This is test data that may be re-used across test runs, such as file- + system images. + +`pytest` also implements a number of its own command-line options. Please see +`pytest` documentation for complete details. Execute `py.test --version` for +a brief summary. Note that U-Boot's test.py script passes all command-line +arguments directly to `pytest` for processing. + +## Testing real hardware + +The tools and techniques used to interact with real hardware will vary +radically between different host and target systems, and the whims of the user. +For this reason, the test suite does not attempt to directly interact with real +hardware in any way. Rather, it executes a standardized set of "hook" scripts +via `$PATH`. These scripts implement certain actions on behalf of the test +suite. This keeps the test suite simple and isolated from system variances +unrelated to U-Boot features. + +### Hook scripts + +#### Environment variables + +The following environment variables are set when running hook scripts: + +- `UBOOT_BOARD_TYPE` the board type being tested. +- `UBOOT_BOARD_IDENTITY` the board identity being tested, or `na` if none was + specified. +- `UBOOT_SOURCE_DIR` the U-Boot source directory. +- `UBOOT_TEST_PY_DIR` the full path to `test/py/` in the source directory. +- `UBOOT_BUILD_DIR` the U-Boot build directory. +- `UBOOT_RESULT_DIR` the test result directory. +- `UBOOT_PERSISTENT_DATA_DIR` the test peristent data directory. + +#### `u-boot-test-console` + +This script provides access to the U-Boot console. The script's stdin/stdout +should be connected to the board's console. This process should continue to run +indefinitely, until killed. The test suite will run this script in parallel +with all other hooks. + +This script may be implemented e.g. by exec()ing `cu`, `conmux`, etc. + +If you are able to run U-Boot under a hardware simulator such as qemu, then +you would likely spawn that simulator from this script. However, note that +`u-boot-test-reset` may be called multiple times per test script run, and must +cause U-Boot to start execution from scratch each time. Hopefully your +simulator includes a virtual reset button! If not, you can launch the +simulator from `u-boot-test-reset` instead, while arranging for this console +process to always communicate with the current simulator instance. + +#### `u-boot-test-flash` + +Prior to running the test suite against a board, some arrangement must be made +so that the board executes the particular U-Boot binary to be tested. Often, +this involves writing the U-Boot binary to the board's flash ROM. The test +suite calls this hook script for that purpose. + +This script should perform the entire flashing process synchronously; the +script should only exit once flashing is complete, and a board reset will +cause the newly flashed U-Boot binary to be executed. + +It is conceivable that this script will do nothing. This might be useful in +the following cases: + +- Some other process has already written the desired U-Boot binary into the + board's flash prior to running the test suite. +- The board allows U-Boot to be downloaded directly into RAM, and executed + from there. Use of this feature will reduce wear on the board's flash, so + may be preferable if available, and if cold boot testing of U-Boot is not + required. If this feature is used, the `u-boot-test-reset` script should + peform this download, since the board could conceivably be reset multiple + times in a single test run. + +It is up to the user to determine if those situations exist, and to code this +hook script appropriately. + +This script will typically be implemented by calling out to some SoC- or +board-specific vendor flashing utility. + +#### `u-boot-test-reset` + +Whenever the test suite needs to reset the target board, this script is +executed. This is guaranteed to happen at least once, prior to executing the +first test function. If any test fails, the test infra-structure will execute +this script again to restore U-Boot to an operational state before running the +next test function. + +This script will likely be implemented by communicating with some form of +relay or electronic switch attached to the board's reset signal. + +The semantics of this script require that when it is executed, U-Boot will +start running from scratch. If the U-Boot binary to be tested has been written +to flash, pulsing the board's reset signal is likely all this script need do. +However, in some scenarios, this script may perform other actions. For +example, it may call out to some SoC- or board-specific vendor utility in order +to download the U-Boot binary directly into RAM and execute it. This would +avoid the need for `u-boot-test-flash` to actually write U-Boot to flash, thus +saving wear on the flash chip(s). + +### Board-type-specific configuration + +Each board has a different configuration and behaviour. Many of these +differences can be automatically detected by parsing the `.config` file in the +build directory. However, some differences can't yet be handled automatically. + +For each board, an optional Python module `u_boot_board_${board_type}` may exist +to provide board-specific information to the test script. Any global value +defined in these modules is available for use by any test function. The data +contained in these scripts must be purely derived from U-Boot source code. +Hence, these configuration files are part of the U-Boot source tree too. + +### Execution environment configuration + +Each user's hardware setup may enable testing different subsets of the features +implemented by a particular board's configuration of U-Boot. For example, a +U-Boot configuration may support USB device mode and USB Mass Storage, but this +can only be tested if a USB cable is connected between the board and the host +machine running the test script. + +For each board, optional Python modules `u_boot_boardenv_${board_type}` and +`u_boot_boardenv_${board_type}_${board_identity}` may exist to provide +board-specific and board-identity-specific information to the test script. Any +global value defined in these modules is available for use by any test +function. The data contained in these is specific to a particular user's +hardware configuration. Hence, these configuration files are not part of the +U-Boot source tree, and should be installed outside of the source tree. Users +should set `$PYTHONPATH` prior to running the test script to allow these +modules to be loaded. + +### Board module parameter usage + +The test scripts rely on the following variables being defined by the board +module: + +- None at present. + +### U-Boot `.config` feature usage + +The test scripts rely on various U-Boot `.config` features, either directly in +order to test those features, or indirectly in order to query information from +the running U-Boot instance in order to test other features. + +One example is that testing of the `md` command requires knowledge of a RAM +address to use for the test. This data is parsed from the output of the +`bdinfo` command, and hence relies on CONFIG_CMD_BDI being enabled. + +For a complete list of dependencies, please search the test scripts for +instances of: + +- `buildconfig.get(...` +- `@pytest.mark.buildconfigspec(...` + +### Complete invocation example + +Assuming that you have installed the hook scripts into $HOME/ubtest/bin, and +any required environment configuration Python modules into $HOME/ubtest/py, +then you would likely invoke the test script as follows: + +If U-Boot has already been built: + +```bash +PATH=$HOME/ubtest/bin:$PATH \ + PYTHONPATH=${HOME}/ubtest/py:${PYTHONPATH} \ + ./test/py/test.py --bd seaboard +``` + +If you want the test script to compile U-Boot for you too, then you likely +need to set `$CROSS_COMPILE` to allow this, and invoke the test script as +follow: + +```bash +CROSS_COMPILE=arm-none-eabi- \ + PATH=$HOME/ubtest/bin:$PATH \ + PYTHONPATH=${HOME}/ubtest/py:${PYTHONPATH} \ + ./test/py/test.py --bd seaboard --build +``` + +## Writing tests + +Please refer to the pytest documentation for details of writing pytest tests. +Details specific to the U-Boot test suite are described below. + +A test fixture named `u_boot_console` should be used by each test function. This +provides the means to interact with the U-Boot console, and retrieve board and +environment configuration information. + +The function `u_boot_console.run_command()` executes a shell command on the +U-Boot console, and returns all output from that command. This allows +validation or interpretation of the command output. This function validates +that certain strings are not seen on the U-Boot console. These include shell +error messages and the U-Boot sign-on message (in order to detect unexpected +board resets). See the source of `u_boot_console_base.py` for a complete list of +"bad" strings. Some test scenarios are expected to trigger these strings. Use +`u_boot_console.disable_check()` to temporarily disable checking for specific +strings. See `test_unknown_cmd.py` for an example. + +Board- and board-environment configuration values may be accessed as sub-fields +of the `u_boot_console.config` object, for example +`u_boot_console.config.ram_base`. + +Build configuration values (from `.config`) may be accessed via the dictionary +`u_boot_console.config.buildconfig`, with keys equal to the Kconfig variable +names. diff --git a/test/py/conftest.py b/test/py/conftest.py new file mode 100644 index 000000000000..3b905ed86f1c --- /dev/null +++ b/test/py/conftest.py @@ -0,0 +1,335 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Implementation of pytest run-time hook functions. These are invoked by +# pytest at certain points during operation, e.g. startup, for each executed +# test, at shutdown etc. These hooks perform functions such as: +# - Parsing custom command-line options. +# - Pullilng in user-specified board configuration. +# - Creating the U-Boot console test fixture. +# - Creating the HTML log file. +# - Monitoring each test's results. +# - Implementing custom pytest markers. + +import atexit +import errno +import os +import os.path +import pexpect +import pytest +from _pytest.runner import runtestprotocol +import ConfigParser +import StringIO +import sys + +# Globals: The HTML log file, and the connection to the U-Boot console. +log = None +console = None + +def mkdir_p(path): + '''Create a directory path, including parent directories, ignoring errors + due to already extant directories.''' + + try: + os.makedirs(path) + except OSError as exc: + if exc.errno == errno.EEXIST and os.path.isdir(path): + pass + else: + raise + +def pytest_addoption(parser): + '''pytest hook: Add custom command-line options to the cmdline parser.''' + + parser.addoption('--build-dir', default=None, + help='U-Boot build directory (O=)') + parser.addoption('--result-dir', default=None, + help='U-Boot test result/tmp directory') + parser.addoption('--persistent-data-dir', default=None, + help='U-Boot test persistent generated data directory') + parser.addoption('--board-type', '--bd', '-B', default='sandbox', + help='U-Boot board type') + parser.addoption('--board-identity', '--id', default='na', + help='U-Boot board identity/instance') + parser.addoption('--build', default=False, action='store_true', + help='Compile U-Boot before running tests') + +def pytest_configure(config): + '''pytest hook: Perform custom initialization at startup time.''' + + global log + global console + global ubconfig + + test_py_dir = os.path.dirname(os.path.abspath(__file__)) + source_dir = os.path.dirname(os.path.dirname(test_py_dir)) + + board_type = config.getoption('board_type') + board_type_filename = board_type.replace('-', '_') + + board_identity = config.getoption('board_identity') + board_identity_filename = board_identity.replace('-', '_') + + build_dir = config.getoption('build_dir') + if not build_dir: + build_dir = source_dir + '/build-' + board_type + mkdir_p(build_dir) + + result_dir = config.getoption('result_dir') + if not result_dir: + result_dir = build_dir + mkdir_p(result_dir) + + persistent_data_dir = config.getoption('persistent_data_dir') + if not persistent_data_dir: + persistent_data_dir = build_dir + '/persistent-data' + mkdir_p(persistent_data_dir) + + import multiplexed_log + log = multiplexed_log.Logfile(result_dir + '/test-log.html') + + if config.getoption('build'): + if build_dir != source_dir: + o_opt = 'O=%s' % build_dir + else: + o_opt = '' + cmds = ( + ['make', o_opt, '-s', board_type + '_defconfig'], + ['make', o_opt, '-s', '-j8'], + ) + runner = log.get_runner('make', sys.stdout) + for cmd in cmds: + runner.run(cmd, cwd=source_dir) + runner.close() + + class ArbitraryAttributeContainer(object): + pass + + ubconfig = ArbitraryAttributeContainer() + ubconfig.brd = dict() + ubconfig.env = dict() + + modules = [ + (ubconfig.brd, 'u_boot_board_' + board_type_filename), + (ubconfig.env, 'u_boot_boardenv_' + board_type_filename), + (ubconfig.env, 'u_boot_boardenv_' + board_type_filename + '_' + + board_identity_filename), + ] + for (dict_to_fill, module_name) in modules: + try: + module = __import__(module_name) + except ImportError: + continue + dict_to_fill.update(module.__dict__) + + ubconfig.buildconfig = dict() + + for conf_file in ('.config', 'include/autoconf.mk'): + dot_config = build_dir + '/' + conf_file + if not os.path.exists(dot_config): + raise Exception(conf_file + ' does not exist; ' + + 'try passing --build option?') + + with open(dot_config, 'rt') as f: + ini_str = '[root]\n' + f.read() + ini_sio = StringIO.StringIO(ini_str) + parser = ConfigParser.RawConfigParser() + parser.readfp(ini_sio) + ubconfig.buildconfig.update(parser.items('root')) + + ubconfig.test_py_dir = test_py_dir + ubconfig.source_dir = source_dir + ubconfig.build_dir = build_dir + ubconfig.result_dir = result_dir + ubconfig.persistent_data_dir = persistent_data_dir + ubconfig.board_type = board_type + ubconfig.board_identity = board_identity + + env_vars = ( + 'board_type', + 'board_identity', + 'source_dir', + 'test_py_dir', + 'build_dir', + 'result_dir', + 'persistent_data_dir', + ) + for v in env_vars: + os.environ['U_BOOT_' + v.upper()] = getattr(ubconfig, v) + + if board_type == 'sandbox': + import u_boot_console_sandbox + console = u_boot_console_sandbox.ConsoleSandbox(log, ubconfig) + else: + import u_boot_console_exec_attach + console = u_boot_console_exec_attach.ConsoleExecAttach(log, ubconfig) + +def pytest_generate_tests(metafunc): + '''pytest hook: parameterize test functions based on custom rules. + + If a test function takes parameter(s) (fixture names) of the form brd__xxx + or env__xxx, the brd and env configuration dictionaries are consulted to + find the list of values to use for those parameters, and the test is + parametrized so that it runs once for each combination of values.''' + + subconfigs = { + 'brd': console.config.brd, + 'env': console.config.env, + } + for fn in metafunc.fixturenames: + parts = fn.split('__') + if len(parts) < 2: + continue + if parts[0] not in subconfigs: + continue + subconfig = subconfigs[parts[0]] + vals = [] + val = subconfig.get(fn, []) + # If that exact name is a key in the data source: + if val: + # ... use the dict value as a single parameter value. + vals = (val, ) + else: + # ... otherwise, see if there's a key that contains a list of + # values to use instead. + vals = subconfig.get(fn + 's', []) + metafunc.parametrize(fn, vals) + +(a)pytest.fixture(scope='session') +def u_boot_console(request): + '''Returns the value of the u_boot_console test function parameter + (fixture).''' + + return console + +tests_not_run = set() +tests_failed = set() +tests_skipped = set() +tests_passed = set() + +def pytest_itemcollected(item): + '''pytest hook: Called once for each test invocation found during + collection. This enables our custom result analysis code to see the list + of all tests that should eventually be run.''' + + tests_not_run.add(item.name) + +def cleanup(): + '''Clean up all global state. Executed (via atexit) once the entire test + process is complete. This includes logging the status of all tests, and + the identity of any failed or skipped tests.''' + + if console: + console.close() + if log: + log.status_pass('%d passed' % len(tests_passed)) + if tests_skipped: + log.status_skipped('%d skipped' % len(tests_skipped)) + for test in tests_skipped: + log.status_skipped('... ' + test) + if tests_failed: + log.status_fail('%d failed' % len(tests_failed)) + for test in tests_failed: + log.status_fail('... ' + test) + if tests_not_run: + log.status_fail('%d not run' % len(tests_not_run)) + for test in tests_not_run: + log.status_fail('... ' + test) + log.close() +atexit.register(cleanup) + +def setup_boardspec(item): + '''Process any 'boardspec' marker for a test. Such a marker lists the set + of board types that a test does/doesn't support. If tests are being + executed on an unsupported board, the test is marked to be skipped.''' + + mark = item.get_marker('boardspec') + if not mark: + return + required_boards = [] + for board in mark.args: + if board.startswith('!'): + if ubconfig.board_type == board[1:]: + pytest.skip('board not supported') + return + else: + required_boards.append(board) + if required_boards and ubconfig.board_type not in required_boards: + pytest.skip('board not supported') + +def setup_buildconfigspec(item): + '''Process any 'buildconfigspec' marker for a test. Such a marker lists + some U-Boot configuration feature that the test requires. If tests are + being executed on an U-Boot build that doesn't have the required feature, + the test is marked to be skipped.''' + + mark = item.get_marker('buildconfigspec') + if not mark: + return + for option in mark.args: + if not ubconfig.buildconfig.get('config_' + option.lower(), None): + pytest.skip('.config feature not enabled') + +def pytest_runtest_setup(item): + '''pytest hook: Called once for each test to perform any custom + configuration. This hook is used to skip the test if certain conditions + apply.''' + + log.start_section(item.name) + setup_boardspec(item) + setup_buildconfigspec(item) + +def pytest_runtest_protocol(item, nextitem): + '''pytest hook: Called to execute a test. This hook wraps the standard + pytest runtestprotocol() function in order to acquire visibility into, and + record, each test function's result.''' + + reports = runtestprotocol(item, nextitem=nextitem) + failed = None + skipped = None + for report in reports: + if report.outcome == 'failed': + failed = report + break + if report.outcome == 'skipped': + if not skipped: + skipped = report + + if failed: + tests_failed.add(item.name) + elif skipped: + tests_skipped.add(item.name) + else: + tests_passed.add(item.name) + tests_not_run.remove(item.name) + + try: + if failed: + msg = 'FAILED:\n' + str(failed.longrepr) + log.status_fail(msg) + elif skipped: + msg = 'SKIPPED:\n' + str(skipped.longrepr) + log.status_skipped(msg) + else: + log.status_pass('OK') + except: + # If something went wrong with logging, it's better to let the test + # process continue, which may report other exceptions that triggered + # the logging issue (e.g. console.log wasn't created). Hence, just + # squash the exception. If the test setup failed due to e.g. syntax + # error somewhere else, this won't be seen. However, once that issue + # is fixed, if this exception still exists, it will then be logged as + # part of the test's stdout. + import traceback + print 'Exception occurred while logging runtest status:' + traceback.print_exc() + # FIXME: Can we force a test failure here? + + log.end_section(item.name) + + if failed: + console.cleanup_spawn() + + return reports diff --git a/test/py/multiplexed_log.css b/test/py/multiplexed_log.css new file mode 100644 index 000000000000..50f7b9092983 --- /dev/null +++ b/test/py/multiplexed_log.css @@ -0,0 +1,88 @@ +/* + * Copyright (c) 2015 Stephen Warren + * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. + * + * SPDX-License-Identifier: GPL-2.0 + */ + +/* + * This provides pretty formatting of the HTML log file, e.g. + * - colored bars beside/above log sections for easily parsed delineation. + * - color highlighting of various messages. + */ + +body { + background-color: black; + color: #ffffff; +} + +pre { + margin-top: 0px; + margin-bottom: 0px; +} + +.implicit { + color: #808080; +} + +.section { + border-style: solid; + border-color: #303030; + border-width: 0px 0px 0px 5px; + padding-left: 5px +} + +.section-header { + background-color: #303030; + margin-left: -5px; + margin-top: 5px; +} + +.section-trailer { + display: none; +} + +.stream { + border-style: solid; + border-color: #303030; + border-width: 0px 0px 0px 5px; + padding-left: 5px +} + +.stream-header { + background-color: #303030; + margin-left: -5px; + margin-top: 5px; +} + +.stream-trailer { + display: none; +} + +.error { + color: #ff0000 +} + +.warning { + color: #ffff00 +} + +.info { + color: #808080 +} + +.action { + color: #8080ff +} + +.status-pass { + color: #00ff00 +} + +.status-skipped { + color: #ffff00 +} + +.status-fail { + color: #ff0000 +} diff --git a/test/py/multiplexed_log.py b/test/py/multiplexed_log.py new file mode 100644 index 000000000000..0b8f5cfa4e1b --- /dev/null +++ b/test/py/multiplexed_log.py @@ -0,0 +1,335 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Generate an HTML-formatted log file containing multiple streams of data, +# each represented in a well-delineated/-structured fashion. + +import cgi +import os.path +import shutil +import subprocess + +mod_dir = os.path.dirname(os.path.abspath(__file__)) + +class LogfileStream(object): + '''A file-like object used to write a single logical stream of data into + a multiplexed log file. Objects of this type should be created by factory + functions in the Logfile class rather than directly.''' + + def __init__(self, logfile, name, chained_file): + '''Initialize a new object. + + logfile: The Logfile object to log to. + name: The name of this log stream. + chained_file: The file-like object to which all stream data should be + logged to in addition to logfile. Can be None.''' + + self.logfile = logfile + self.name = name + self.chained_file = chained_file + + def close(self): + '''Dummy function so that this class is "file-like".''' + + pass + + def write(self, data, implicit=False): + '''Write data to the log stream.''' + + self.logfile.write(self, data, implicit) + if self.chained_file: + self.chained_file.write(data) + + def flush(self): + '''Flush the log stream, to ensure correct log interleaving.''' + + self.logfile.flush() + if self.chained_file: + self.chained_file.flush() + +class RunAndLog(object): + '''A utility object used to execute sub-processes and log their output to + a multiplexed log file. Objects of this type should be created by factory + functions in the Logfile class rather than directly.''' + + def __init__(self, logfile, name, chained_file): + '''Initialize a new object. + + logfile: The Logfile object to log to. + name: The name of this log stream or sub-process. + chained_file: The file-like object to which all stream data should be + logged to in addition to logfile. Can be None.''' + + self.logfile = logfile + self.name = name + self.chained_file = chained_file + + def close(self): + '''Clean up any resources managed by this object.''' + pass + + def run(self, cmd, cwd=None): + '''run a command as a sub-process, and log the results. + + cmd: The command to execute. + cwd: The directory to run the command in. Can be None to use the + current directory.''' + + msg = "+" + " ".join(cmd) + "\n" + if self.chained_file: + self.chained_file.write(msg) + self.logfile.write(self, msg) + + try: + p = subprocess.Popen(cmd, cwd=cwd, + stdin=None, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + (output, stderr) = p.communicate() + status = p.returncode + except subprocess.CalledProcessError as cpe: + output = cpe.output + status = cpe.returncode + self.logfile.write(self, output) + if status: + if self.chained_file: + self.chained_file.write(output) + raise Exception("command failed; exit code " + str(status)) + +class SectionCtxMgr(object): + '''A context manager for Python's "with" statement, which allows a certain + portion of test code to be logged to a separate section of the log file. + Objects of this type should be created by factory functions in the Logfile + class rather than directly.''' + + def __init__(self, log, marker): + '''Initialize a new object. + + log: The Logfile object to log to. + marker: The name of the nested log section.''' + + self.log = log + self.marker = marker + + def __enter__(self): + self.log.start_section(self.marker) + + def __exit__(self, extype, value, traceback): + self.log.end_section(self.marker) + +class Logfile(object): + '''Generates an HTML-formatted log file containing multiple streams of + data, each represented in a well-delineated/-structured fashion.''' + + def __init__(self, fn): + '''Initialize a new object. + + fn: The filename to write to.''' + + self.f = open(fn, "wt") + self.last_stream = None + self.blocks = [] + self.cur_evt = 1 + shutil.copy(mod_dir + "/multiplexed_log.css", os.path.dirname(fn)) + self.f.write("""\ +<html> +<head> +<link rel="stylesheet" type="text/css" href="multiplexed_log.css"> +</head> +<body> +<tt> +""") + + def close(self): + '''Close the log file. After calling this function, no more data may + be written to the log.''' + + self.f.write("""\ +</tt> +</body> +</html> +""") + self.f.close() + + # The set of characters that should be represented as hexadecimal codes in + # the log file. + _nonprint = ("^%" + "".join(chr(c) for c in range(0, 32) if c not in (9, 10)) + + "".join(chr(c) for c in range(127, 256))) + + def _escape(self, data): + '''Render raw log data into a format suitable for inclusion in an HTML + document. This includes HTML-escaping certain characters, and + translating control characters to a hexadecimal representation. + + data: The raw string data to be escaped.''' + + data = data.replace(chr(13), "") + data = "".join((c in self._nonprint) and ("%%%02x" % ord(c)) or + c for c in data) + data = cgi.escape(data) + return data + + def _terminate_stream(self): + '''Write HTML to the log file to terminate the current stream's data.''' + + self.cur_evt += 1 + if not self.last_stream: + return + self.f.write("</pre>\n") + self.f.write("<div class=\"stream-trailer\" id=\"" + + self.last_stream.name + "\">End stream: " + + self.last_stream.name + "</div>\n") + self.f.write("</div>\n") + self.last_stream = None + + def _note(self, note_type, msg): + '''Write HTML to the log file that represents some form of note or + one-off message. + + note_type: The type of note. This must be a value supported by the + accompanying multiplexed_log.css. + msg: The note/message to log.''' + + self._terminate_stream() + self.f.write("<div class=\"" + note_type + "\">\n<pre>") + self.f.write(self._escape(msg)) + self.f.write("\n</pre></div>\n") + + def start_section(self, marker): + '''Write HTML to the log file to mark the start of a new nested + section. + + marker: The name of the section that is starting.''' + + self._terminate_stream() + self.blocks.append(marker) + blk_path = "/".join(self.blocks) + self.f.write("<div class=\"section\" id=\"" + blk_path + "\">\n") + self.f.write("<div class=\"section-header\" id=\"" + blk_path + + "\">Section: " + blk_path + "</div>\n") + + def end_section(self, marker): + '''Write HTML to the log file to mark the end of a the current nested + section. + + marker: The name of the section that is ending.''' + + if (not self.blocks) or (marker != self.blocks[-1]): + raise Exception("Block nesting mismatch: \"%s\" \"%s\"" % + (marker, "/".join(self.blocks))) + self._terminate_stream() + blk_path = "/".join(self.blocks) + self.f.write("<div class=\"section-trailer\" id=\"section-trailer-" + + blk_path + "\">End section: " + blk_path + "</div>\n") + self.f.write("</div>\n") + self.blocks.pop() + + def section(self, marker): + '''Create a context manager for Python's "with" statement, which allows + a certain portion of test code to be logged to a separate section of + the log file. + + marker: The name of the nested section. + + Usage: + with log.section("somename"): + some test code''' + + return SectionCtxMgr(self, marker) + + def error(self, msg): + '''Write an error note to the log file. + + msg: A message describing the error.''' + + self._note("error", msg) + + def warning(self, msg): + '''Write an warning note to the log file. + + msg: A message describing the warning.''' + + self._note("warning", msg) + + def info(self, msg): + '''Write an informational note to the log file. + + msg: An information message.''' + + self._note("info", msg) + + def action(self, msg): + '''Write an action note to the log file. + + msg: A message describing the action that is being logged.''' + + self._note("action", msg) + + def status_pass(self, msg): + '''Write a note to the log file describing test(s) which passed. + + msg: A message describing passed test(s).''' + + self._note("status-pass", msg) + + def status_skipped(self, msg): + '''Write a note to the log file describing skipped test(s). + + msg: A message describing passed test(s).''' + + self._note("status-skipped", msg) + + def status_fail(self, msg): + '''Write a note to the log file describing failed test(s). + + msg: A message describing passed test(s).''' + + self._note("status-fail", msg) + + def get_stream(self, name, chained_file=None): + '''Create an object to log a single stream's data into the log file. + + name: The name of the stream. + chained_file: The file-like object to which all stream data should be + logged to in addition to this log. Can be None.''' + + return LogfileStream(self, name, chained_file) + + def get_runner(self, name, chained_file=None): + '''Create a utility object to execute sub-processes and log their + output to + + name: The name of this sub-process. + chained_file: The file-like object to which all stream data should be + logged to in addition to logfile. Can be None.''' + + return RunAndLog(self, name, chained_file) + + def write(self, stream, data, implicit=False): + '''Write stream data into the log file. This function should only be + used by instances of LogfileStream or RunAndLog. + + stream: The stream whose data is being logged. + data: The data to log. + implicit: Boolean indicating whether data actually appeared in the + stream, or was implicitly generated. A valid use-case is to repeat a + shell prompt at the start of each separate log section, so that the + log makes more sense.''' + + if stream != self.last_stream: + self._terminate_stream() + self.f.write("<div class=\"stream\" id=\"%s\">\n" % stream.name) + self.f.write("<div class=\"stream-header\" id=\"" + stream.name + + "\">Stream: " + stream.name + "</div>\n") + self.f.write("<pre>") + if implicit: + self.f.write("<span class=\"implicit\">") + self.f.write(self._escape(data)) + if implicit: + self.f.write("</span>") + self.last_stream = stream + + def flush(self): + '''Flush the log stream, to ensure correct log interleaving.''' + + self.f.flush() diff --git a/test/py/pytest.ini b/test/py/pytest.ini new file mode 100644 index 000000000000..67e514f42058 --- /dev/null +++ b/test/py/pytest.ini @@ -0,0 +1,11 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Static configuration data for pytest. pytest reads this at startup time. + +[pytest] +markers = + boardspec: U-Boot: Describes the set of boards a test can/can't run on. + buildconfigspec: U-Boot: Describes Kconfig/config-header constraints. diff --git a/test/py/test.py b/test/py/test.py new file mode 100755 index 000000000000..9c23898774ed --- /dev/null +++ b/test/py/test.py @@ -0,0 +1,32 @@ +#!/usr/bin/env python + +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Wrapper script to invoke pytest with the directory name that contains the +# U-Boot tests. + +import os +import os.path +import sys + +# Get rid of argv[0] +sys.argv.pop(0) + +# argv; py.test test_directory_name user-supplied-arguments +args = ["py.test", os.path.dirname(__file__) + "/tests"] +args.extend(sys.argv) + +try: + os.execvp("py.test", args) +except: + # Log full details of any exception for detailed analysis + import traceback + traceback.print_exc() + # Hint to the user that they likely simply haven't installed the required + # dependencies. + print >>sys.stderr, """ +exec(py.test) failed; perhaps you are missing some dependencies? +See test/py/README.md for the list.""" diff --git a/test/py/tests/test_000_version.py b/test/py/tests/test_000_version.py new file mode 100644 index 000000000000..0fddaf4fb6d5 --- /dev/null +++ b/test/py/tests/test_000_version.py @@ -0,0 +1,20 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# pytest runs tests the order of their module path, which is related to the +# filename containing the test. This file is named such that it is sorted +# first, simply as a very basic sanity check of the functionality of the U-Boot +# command prompt. + +def test_version(u_boot_console): + '''Test that the "version" command prints the U-Boot version.''' + + # "version" prints the U-Boot sign-on message. This is usually considered + # an error, so that any unexpected reboot causes an error. Here, this + # error detection is disabled since the sign-on message is expected. + with u_boot_console.disable_check('main_signon'): + response = u_boot_console.run_command('version') + # Ensure "version" printed what we expected. + u_boot_console.validate_main_signon_in_text(response) diff --git a/test/py/tests/test_help.py b/test/py/tests/test_help.py new file mode 100644 index 000000000000..894f3b5f1700 --- /dev/null +++ b/test/py/tests/test_help.py @@ -0,0 +1,9 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +def test_help(u_boot_console): + '''Test that the "help" command can be executed.''' + + u_boot_console.run_command('help') diff --git a/test/py/tests/test_unknown_cmd.py b/test/py/tests/test_unknown_cmd.py new file mode 100644 index 000000000000..2de93e0026ff --- /dev/null +++ b/test/py/tests/test_unknown_cmd.py @@ -0,0 +1,14 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +def test_unknown_command(u_boot_console): + '''Test that executing an unknown command causes U-Boot to print an + error.''' + + # The "unknown command" error is actively expected here, + # so error detection for it is disabled. + with u_boot_console.disable_check('unknown_command'): + response = u_boot_console.run_command('non_existent_cmd') + assert('Unknown command \'non_existent_cmd\' - try \'help\'' in response) diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py new file mode 100644 index 000000000000..9f3696947d6d --- /dev/null +++ b/test/py/u_boot_console_base.py @@ -0,0 +1,260 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Common logic to interact with U-Boot via the console. This class provides +# the interface that tests use to execute U-Boot shell commands and wait for +# their results. Sub-classes exist to perform board-type-specific setup +# operations, such as spawning a sub-process for Sandbox, or attaching to the +# serial console of real hardware. + +import multiplexed_log +import os +import pytest +import re +import sys + +# Regexes for text we expect U-Boot to send to the console. +pattern_u_boot_spl_signon = re.compile('(U-Boot SPL \\d{4}\\.\\d{2}-[^\r\n]*)') +pattern_u_boot_main_signon = re.compile('(U-Boot \\d{4}\\.\\d{2}-[^\r\n]*)') +pattern_stop_autoboot_prompt = re.compile('Hit any key to stop autoboot: ') +pattern_unknown_command = re.compile('Unknown command \'.*\' - try \'help\'') +pattern_error_notification = re.compile('## Error: ') + +class ConsoleDisableCheck(object): + '''Context manager (for Python's with statement) that temporarily disables + the specified console output error check. This is useful when deliberately + executing a command that is known to trigger one of the error checks, in + order to test that the error condition is actually raised. This class is + used internally by ConsoleBase::disable_check(); it is not intended for + direct usage.''' + + def __init__(self, console, check_type): + self.console = console + self.check_type = check_type + + def __enter__(self): + self.console.disable_check_count[self.check_type] += 1 + + def __exit__(self, extype, value, traceback): + self.console.disable_check_count[self.check_type] -= 1 + +class ConsoleBase(object): + '''The interface through which test functions interact with the U-Boot + console. This primarily involves executing shell commands, capturing their + results, and checking for common error conditions. Some common utilities + are also provided too.''' + + def __init__(self, log, config, max_fifo_fill): + '''Initialize a U-Boot console connection. Can only usefully be called + by sub-classes.''' + + self.log = log + self.config = config + self.max_fifo_fill = max_fifo_fill + + self.logstream = self.log.get_stream('console', sys.stdout) + + # Array slice removes leading/trailing quotes + self.prompt = self.config.buildconfig['config_sys_prompt'][1:-1] + self.prompt_escaped = re.escape(self.prompt) + self.p = None + self.disable_check_count = { + 'spl_signon': 0, + 'main_signon': 0, + 'unknown_command': 0, + 'error_notification': 0, + } + + self.at_prompt = False + self.at_prompt_logevt = None + self.ram_base = None + + def close(self): + '''Terminate the connection to the U-Boot console. Likely only useful + once all interaction with U-Boot is complete.''' + + if self.p: + self.p.close() + self.logstream.close() + + def run_command(self, cmd, wait_for_echo=True, send_nl=True, + wait_for_prompt=True): + '''Execute a command via the U-Boot console. The command is always sent + to U-Boot. + + U-Boot echoes any command back to its output, and this function + typically waits for that to occur. The wait can be disabled by setting + wait_for_echo=False, which is useful e.g. when sending CTRL-C to + interrupt a long-running command such as "ums". + + Command execution is typically triggered by sending a newline + character. This can be disabled by setting send_nl=False, which is + also useful when sending CTRL-C. + + This function typically waits for the command to finish executing, and + returns the console output that it generated. This can be disabled by + setting wait_for_prompt=False, which is useful when invoking a long- + running command such as "ums".''' + + self.ensure_spawned() + + if self.at_prompt and \ + self.at_prompt_logevt != self.logstream.logfile.cur_evt: + self.logstream.write(self.prompt, implicit=True) + + bad_patterns = [] + bad_pattern_ids = [] + if (self.disable_check_count['spl_signon'] == 0 and + self.u_boot_spl_signon): + bad_patterns.append(self.u_boot_spl_signon_escaped) + bad_pattern_ids.append('SPL signon') + if self.disable_check_count['main_signon'] == 0: + bad_patterns.append(self.u_boot_main_signon_escaped) + bad_pattern_ids.append('U-Boot main signon') + if self.disable_check_count['unknown_command'] == 0: + bad_patterns.append(pattern_unknown_command) + bad_pattern_ids.append('Unknown command') + if self.disable_check_count['error_notification'] == 0: + bad_patterns.append(pattern_error_notification) + bad_pattern_ids.append('Error notification') + try: + self.at_prompt = False + if send_nl: + cmd += '\n' + while cmd: + # Limit max outstanding data, so UART FIFOs don't overflow + chunk = cmd[:self.max_fifo_fill] + cmd = cmd[self.max_fifo_fill:] + self.p.send(chunk) + if not wait_for_echo: + continue + chunk = re.escape(chunk) + chunk = chunk.replace('\\\n', '[\r\n]') + m = self.p.expect([chunk] + bad_patterns) + if m != 0: + self.at_prompt = False + raise Exception('Bad pattern found on console: ' + + bad_pattern_ids[m - 1]) + if not wait_for_prompt: + return + m = self.p.expect([self.prompt_escaped] + bad_patterns) + if m != 0: + self.at_prompt = False + raise Exception('Bad pattern found on console: ' + + bad_pattern_ids[m - 1]) + self.at_prompt = True + self.at_prompt_logevt = self.logstream.logfile.cur_evt + # Only strip \r\n; space/TAB might be significant if testing + # indentation. + return self.p.before.strip('\r\n') + except Exception as ex: + self.log.error(str(ex)) + self.cleanup_spawn() + raise + + def ctrlc(self): + '''Send a CTRL-C character to U-Boot. This is useful in order to + stop execution of long-running synchronous commands such as "ums".''' + + self.run_command(chr(3), wait_for_echo=False, send_nl=False) + + def ensure_spawned(self): + '''Ensure that this console object is attached to a correctly operating + U-Boot instance. This may require spawning a new Sandbox process or + resetting target hardware, as defined by the implementation sub-class. + + This is an internal function and should not be called directly.''' + + if self.p: + return + try: + self.at_prompt = False + self.log.action('Starting U-Boot') + self.p = self.get_spawn() + # Real targets can take a long time to scroll large amounts of + # text if LCD is enabled. This value may need tweaking in the + # future, possibly per-test to be optimal. This works for 'help' + # on board 'seaboard'. + self.p.timeout = 30000 + self.p.logfile_read = self.logstream + if self.config.buildconfig.get('CONFIG_SPL', False) == 'y': + self.p.expect([pattern_u_boot_spl_signon]) + self.u_boot_spl_signon = self.p.after + self.u_boot_spl_signon_escaped = re.escape(self.p.after) + else: + self.u_boot_spl_signon = None + self.p.expect([pattern_u_boot_main_signon]) + self.u_boot_main_signon = self.p.after + self.u_boot_main_signon_escaped = re.escape(self.p.after) + while True: + match = self.p.expect([self.prompt_escaped, + pattern_stop_autoboot_prompt]) + if match == 1: + self.p.send(chr(3)) # CTRL-C + continue + break + self.at_prompt = True + self.at_prompt_logevt = self.logstream.logfile.cur_evt + except Exception as ex: + self.log.error(str(ex)) + self.cleanup_spawn() + raise + + def cleanup_spawn(self): + '''Shut down all interaction with the U-Boot instance. This is used + when an error is detected prior to re-establishing a connection with a + fresh U-Boot instance. + + This is an internal function and should not be called directly.''' + + try: + if self.p: + self.p.close() + except: + pass + self.p = None + + def validate_main_signon_in_text(self, text): + '''Validate that a command's console output includes the expected + U-Boot signon message. This is primarily useful for validating the + "version" command without duplicating the signon text regex in a test + function.''' + + assert(self.u_boot_main_signon in text) + + def disable_check(self, check_type): + '''Create a new Python context manager (for use with the "with" + statement) which temporarily disables a particular console output error + check. Valid values for check_type may be found in + self.disable_check_count above.''' + + return ConsoleDisableCheck(self, check_type) + + def find_ram_base(self): + '''Probe the running U-Boot to determine the address of the first bank + of RAM. This is useful for tests that test reading/writing RAM, or + load/save files that aren't associated with some standard address + typically represented in an environment variable such as + ${kernel_addr_r}. The value is cached so that it only needs to be + actively read once.''' + + if self.config.buildconfig.get('config_cmd_bdi', 'n') != 'y': + pytest.skip('bdinfo command not supported') + if self.ram_base == -1: + pytest.skip('Previously failed to find RAM bank start') + if self.ram_base is not None: + return self.ram_base + + with self.log.section('find_ram_base'): + response = self.run_command('bdinfo') + for l in response.split('\n'): + if '-> start' in l: + self.ram_base = int(l.split('=')[1].strip(), 16) + break + if self.ram_base is None: + self.ram_base = -1 + raise Exception('Failed to find RAM bank start in `bdinfo`') + + return self.ram_base diff --git a/test/py/u_boot_console_exec_attach.py b/test/py/u_boot_console_exec_attach.py new file mode 100644 index 000000000000..9fcc32c5f73b --- /dev/null +++ b/test/py/u_boot_console_exec_attach.py @@ -0,0 +1,51 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Logic to interact with U-Boot running on real hardware, typically via a +# physical serial port. + +from ubspawn import Spawn +from u_boot_console_base import ConsoleBase + +class ConsoleExecAttach(ConsoleBase): + '''Represents a physical connection to a U-Boot console, typically via a + serial port. This implementation executes a sub-process to attach to the + console, expecting that the stdin/out of the sub-process will be forwarded + to/from the physical hardware. This approach isolates the test infra- + structure from the user-/installation-specific details of how to + communicate with, and the identity of, serial ports etc.''' + + def __init__(self, log, config): + '''Initialize a U-Boot console connection. + + log: A multiplexed_log::Logfile instance. + config: A "configuration" object as defined in conftest.py.''' + + # The max_fifo_fill value might need tweaking per-board/-SoC? + # 1 would be safe anywhere, but is very slow (a pexpect issue?). + # 16 is a common FIFO size. + # HW flow control would mean this could be infinite. + super(ConsoleExecAttach, self).__init__(log, config, max_fifo_fill=16) + + self.log.action('Flashing U-Boot') + cmd = ['u-boot-test-flash', config.board_type, config.board_identity] + runner = self.log.get_runner(cmd[0]) + runner.run(cmd) + runner.close() + + def get_spawn(self): + '''Create and return a new "spawn" object that is attached to a + freshly reset board running U-Boot.''' + + args = [self.config.board_type, self.config.board_identity] + s = Spawn(['u-boot-test-console'] + args) + + self.log.action('Resetting board') + cmd = ['u-boot-test-reset'] + args + runner = self.log.get_runner(cmd[0]) + runner.run(cmd) + runner.close() + + return s diff --git a/test/py/u_boot_console_sandbox.py b/test/py/u_boot_console_sandbox.py new file mode 100644 index 000000000000..0c0343935167 --- /dev/null +++ b/test/py/u_boot_console_sandbox.py @@ -0,0 +1,48 @@ +# Copyright (c) 2015 Stephen Warren +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Logic to interact with the sandbox port of U-Boot, running as a sub-process. + +import time +from ubspawn import Spawn +from u_boot_console_base import ConsoleBase + +class ConsoleSandbox(ConsoleBase): + '''Represents a connection to a sandbox U-Boot console, executed as a sub- + process.''' + + def __init__(self, log, config): + '''Initialize a U-Boot console connection. + + log: A multiplexed_log::Logfile instance. + config: A "configuration" object as defined in conftest.py.''' + + super(ConsoleSandbox, self).__init__(log, config, max_fifo_fill=1024) + + def get_spawn(self): + '''Create and return a new "spawn" object that is attached to a + freshly invoked U-Boot sandbox process.''' + + return Spawn([self.config.build_dir + '/u-boot']) + + def kill(self, sig): + '''Send a specific Unix signal to the sandbox process.''' + + self.ensure_spawned() + self.log.action('kill %d' % sig) + self.p.kill(sig) + + def validate_exited(self): + '''Validate that the sandbox process has exited.''' + + p = self.p + self.p = None + for i in xrange(100): + ret = not p.isalive() + if ret: + break + time.sleep(0.1) + p.close() + return ret diff --git a/test/py/u_boot_spawn.py b/test/py/u_boot_spawn.py new file mode 100644 index 000000000000..12e417c841ed --- /dev/null +++ b/test/py/u_boot_spawn.py @@ -0,0 +1,123 @@ +# Copyright (c) 2015-2016, NVIDIA CORPORATION. All rights reserved. +# +# SPDX-License-Identifier: GPL-2.0 + +# Logic to spawn a sub-process and interact with its stdio. + +import os +import re +import pty +import select +import time + +class Timeout(Exception): + '''An exception sub-class that indicates that a timeout occurred.''' + pass + +class Spawn(object): + '''Represents the stdio of a freshly created sub-process. Commands may be + sent to the process, and responses waited for.''' + + def __init__(self, args): + '''Spawn (fork/exec) the sub-process and initialize interaction with + it. + + args: array of processs arguments. argv[0] is the command to execute.''' + + self.waited = False + self.buf = '' + self.logfile_read = None + self.before = '' + self.after = '' + self.timeout = None + + (self.pid, self.fd) = pty.fork() + if self.pid == 0: + try: + os.execvp(args[0], args) + except: + print 'CHILD EXECEPTION:' + import traceback + traceback.print_exc() + finally: + os._exit(255) + + self.poll = select.poll() + self.poll.register(self.fd, select.POLLIN | select.POLLPRI | select.POLLERR | select.POLLHUP | select.POLLNVAL) + + def kill(self, sig): + '''Send unix signal "sig" to the child process.''' + + os.kill(self.pid, sig) + + def isalive(self): + '''Determine whether the child process is still running.''' + + if self.waited: + return False + + w = os.waitpid(self.pid, os.WNOHANG) + if w[0] == 0: + return True + + self.waited = True + return False + + def send(self, data): + '''Send data to the sub-process's stdin.''' + + os.write(self.fd, data) + + def expect(self, patterns): + '''Wait for the sub-process to emit one of a set of expected patterns, + or for a timeout to occur. + + patterns: a list of strings or regex objects that we expect to see in + the sub-process' stdout.''' + + for pi in xrange(len(patterns)): + if type(patterns[pi]) == type(''): + patterns[pi] = re.compile(patterns[pi]) + + try: + while True: + earliest_m = None + earliest_pi = None + for pi in xrange(len(patterns)): + pattern = patterns[pi] + m = pattern.search(self.buf) + if not m: + continue + if earliest_m and m.start() > earliest_m.start(): + continue + earliest_m = m + earliest_pi = pi + if earliest_m: + pos = earliest_m.start() + posafter = earliest_m.end() + 1 + self.before = self.buf[:pos] + self.after = self.buf[pos:posafter] + self.buf = self.buf[posafter:] + return earliest_pi + events = self.poll.poll(self.timeout) + if not events: + raise Timeout() + c = os.read(self.fd, 1024) + if not c: + raise EOFError() + if self.logfile_read: + self.logfile_read.write(c) + self.buf += c + finally: + if self.logfile_read: + self.logfile_read.flush() + + def close(self): + '''Close the stdio connection to the sub-process, and wait until the + sub-process is no longer running.''' + + os.close(self.fd) + for i in xrange(100): + if not self.isalive(): + break + time.sleep(0.1) -- 2.6.4

3 19

[U-Boot] curious if XTRN_DECLARE_GLOBAL_DATA_PTR is still of value
by Robert P. J. Day 11 Jan '16

11 Jan '16

perusing the code related to the global data ptr and noticed in common/board_f.c: /* * Pointer to initial global data area * * Here we initialize it if needed. */ #ifdef XTRN_DECLARE_GLOBAL_DATA_PTR #undef XTRN_DECLARE_GLOBAL_DATA_PTR #define XTRN_DECLARE_GLOBAL_DATA_PTR /* empty = allocate here */ DECLARE_GLOBAL_DATA_PTR = (gd_t *) (CONFIG_SYS_INIT_GD_ADDR); #else DECLARE_GLOBAL_DATA_PTR; #endif i was curious to see if anything took advantage of that and the only other place in the code that referred to that was in arch/powerpc/include/asm/global_data.h: #if 1 #define DECLARE_GLOBAL_DATA_PTR register volatile gd_t *gd asm ("r2") #else /* We could use plain global data, but the resulting code is bigger */ #define XTRN_DECLARE_GLOBAL_DATA_PTR extern #define DECLARE_GLOBAL_DATA_PTR XTRN_DECLARE_GLOBAL_DATA_PTR \ gd_t *gd #endif and given the "#if 1", it's pretty clear powerpc doesn't care about that, and that file common/board_f.c is the only place i can see in the entire code base that refers to CONFIG_SYS_INIT_GD_ADDR. so are people still taking advantage of this in some way? rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ========================================================================

2 1

[U-Boot] [PATCH v3 00/10] sf: Add flash_ids inside Core
by Jagan Teki 11 Jan '16

11 Jan '16

This series handle flash_ids stuff inside spi_flash core. Changes for v3: - Updated SNOR_F_SST_WRITE at sf_probe.c - two patches, wrt sanbox newly added Changes for v2: - Remove SECT_4K flag for s25fl128s Jagan Teki (10): sf: Add spi_flash_ids local to core sf: Rename SST_WR to SST_WRITE sf: Remove SPI_NOR_DUAL|QUAD_READ sf: Use sf_params in sandbox sf: Remove SECT_32K sf: Add SECT_4K_PMC sf: Use CONFIG_SPI_FLASH_USE_4K_SECTORS sf: Fix few naming convention for read modes sf: sandbox: Move spi_flash_params_table sf: Drop sf_params.c drivers/mtd/spi/Makefile | 2 +- drivers/mtd/spi/sandbox.c | 155 +++++++++++++++ drivers/mtd/spi/sf_internal.h | 29 +-- drivers/mtd/spi/sf_params.c | 146 -------------- drivers/mtd/spi/sf_probe.c | 2 +- drivers/mtd/spi/spi_flash.c | 446 +++++++++++++++++++++++++++++++++++------- include/linux/err.h | 5 + 7 files changed, 538 insertions(+), 247 deletions(-) delete mode 100644 drivers/mtd/spi/sf_params.c -- 1.9.1

3 13

[U-Boot] [Patch V4 4/4] dm: env_sf: fix saveenv() to use driver model
by Gong Qianyu 11 Jan '16

11 Jan '16

From: Gong Qianyu <Qianyu.Gong(a)freescale.com> It might be missed when converting spi_flash_probe() in cmd_sf.c. This commit refers to fbb099183e3a53f77a975964cdf2e73d11e565af. Signed-off-by: Gong Qianyu <Qianyu.Gong(a)freescale.com> --- V4: - Use CONFIG_ENV_* instead of CONFIG_SF_*. - Remove the variables and call the macros directly. - Use set_default_env instead of print info. V3: - Remove redundant operations for saveenv() V2: - New patch. common/env_sf.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/common/env_sf.c b/common/env_sf.c index 9409831..892e6cb 100644 --- a/common/env_sf.c +++ b/common/env_sf.c @@ -16,6 +16,7 @@ #include <spi_flash.h> #include <search.h> #include <errno.h> +#include <dm/device-internal.h> #ifndef CONFIG_ENV_SPI_BUS # define CONFIG_ENV_SPI_BUS 0 @@ -51,6 +52,19 @@ int saveenv(void) char *saved_buffer = NULL, flag = OBSOLETE_FLAG; u32 saved_size, saved_offset, sector = 1; int ret; +#ifdef CONFIG_DM_SPI_FLASH + struct udevice *new; + + ret = spi_flash_probe_bus_cs(CONFIG_ENV_SPI_BUS, CONFIG_ENV_SPI_CS, + CONFIG_ENV_SPI_MAX_HZ, + CONFIG_ENV_SPI_MODE, &new); + if (ret) { + set_default_env("!spi_flash_probe_bus_cs() failed"); + return 1; + } + + env_flash = dev_get_uclass_priv(new); +#else if (!env_flash) { env_flash = spi_flash_probe(CONFIG_ENV_SPI_BUS, @@ -61,6 +75,7 @@ int saveenv(void) return 1; } } +#endif ret = env_export(&env_new); if (ret) @@ -227,6 +242,19 @@ int saveenv(void) char *saved_buffer = NULL; int ret = 1; env_t env_new; +#ifdef CONFIG_DM_SPI_FLASH + struct udevice *new; + + ret = spi_flash_probe_bus_cs(CONFIG_ENV_SPI_BUS, CONFIG_ENV_SPI_CS, + CONFIG_ENV_SPI_MAX_HZ, + CONFIG_ENV_SPI_MODE, &new); + if (ret) { + set_default_env("!spi_flash_probe_bus_cs() failed"); + return 1; + } + + env_flash = dev_get_uclass_priv(new); +#else if (!env_flash) { env_flash = spi_flash_probe(CONFIG_ENV_SPI_BUS, @@ -237,6 +265,7 @@ int saveenv(void) return 1; } } +#endif /* Is the sector larger than the env (i.e. embedded) */ if (CONFIG_ENV_SECT_SIZE > CONFIG_ENV_SIZE) { -- 2.1.0.27.g96db324

2 1

[U-Boot] sf: About sandbox flash driver
by Jagan Teki 11 Jan '16

11 Jan '16

Hi Simon, Few question wrt sandbox flash driver - drivers/mtd/spi/sandbox.c 1) Does this driver used for creating flash emulation environment? or something else? 2) Can't it use spi_flash.c core for all spi flash operations, how different is this? Please let me know your inputs. thanks! -- Jagan.

2 3

[U-Boot] [PATCH] net: phy: ensure Gigabit features are masked off if requested
by Alexey Brodkin 11 Jan '16

11 Jan '16

From: Florian Fainelli <f.fainelli(a)gmail.com> When a Gigabit PHY device is connected to a 10/100Mbits capable Ethernet MAC, the driver will restrict the phydev->supported modes to mask off Gigabit. If the Gigabit PHY comes out of reset with the Gigabit features set by default in MII_CTRL1000, it will keep advertising these feature, so by the time we call genphy_config_advert(), the condition on phydev->supported having the Gigabit features on is false, and we do not update MII_CTRL1000 with updated values, and we keep advertising Gigabit features, eventually configuring the PHY for Gigabit whilst the Ethernet MAC does not support that. This patches fixes the problem by ensuring that the Gigabit feature bits are always cleared in MII_CTRL1000, if the PHY happens to be a Gigabit PHY, and then, if Gigabit features are supported, setting those and updating MII_CTRL1000 accordingly. This is a copy of patch from Linux kernel, see http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=5… Signed-off-by: Florian Fainelli <f.fainelli(a)gmail.com> Signed-off-by: Alexey Brodkin <abrodkin(a)synopsys.com> Cc: Joe Hershberger <joe.hershberger(a)ni.com> --- drivers/net/phy/phy.c | 43 ++++++++++++++++++++++++++----------------- 1 file changed, 26 insertions(+), 17 deletions(-) diff --git a/drivers/net/phy/phy.c b/drivers/net/phy/phy.c index 51b5746..084276f 100644 --- a/drivers/net/phy/phy.c +++ b/drivers/net/phy/phy.c @@ -38,11 +38,10 @@ DECLARE_GLOBAL_DATA_PTR; static int genphy_config_advert(struct phy_device *phydev) { u32 advertise; - int oldadv, adv; + int oldadv, adv, bmsr; int err, changed = 0; - /* Only allow advertising what - * this PHY supports */ + /* Only allow advertising what this PHY supports */ phydev->advertising &= phydev->supported; advertise = phydev->advertising; @@ -79,29 +78,39 @@ static int genphy_config_advert(struct phy_device *phydev) changed = 1; } + bmsr = phy_read(phydev, MDIO_DEVAD_NONE, MII_BMSR); + if (bmsr < 0) + return bmsr; + + /* Per 802.3-2008, Section 22.2.4.2.16 Extended status all + * 1000Mbits/sec capable PHYs shall have the BMSR_ESTATEN bit set to a + * logical 1. + */ + if (!(bmsr & BMSR_ESTATEN)) + return changed; + /* Configure gigabit if it's supported */ - if (phydev->supported & (SUPPORTED_1000baseT_Half | - SUPPORTED_1000baseT_Full)) { - oldadv = adv = phy_read(phydev, MDIO_DEVAD_NONE, MII_CTRL1000); + oldadv = adv = phy_read(phydev, MDIO_DEVAD_NONE, MII_CTRL1000); + + if (adv < 0) + return adv; - if (adv < 0) - return adv; + adv &= ~(ADVERTISE_1000FULL | ADVERTISE_1000HALF); - adv &= ~(ADVERTISE_1000FULL | ADVERTISE_1000HALF); + if (phydev->supported & (SUPPORTED_1000baseT_Half | + SUPPORTED_1000baseT_Full)) { if (advertise & SUPPORTED_1000baseT_Half) adv |= ADVERTISE_1000HALF; if (advertise & SUPPORTED_1000baseT_Full) adv |= ADVERTISE_1000FULL; + } - if (adv != oldadv) { - err = phy_write(phydev, MDIO_DEVAD_NONE, MII_CTRL1000, - adv); + if (adv != oldadv) + changed = 1; - if (err < 0) - return err; - changed = 1; - } - } + err = phy_write(phydev, MDIO_DEVAD_NONE, MII_CTRL1000, adv); + if (err < 0) + return err; return changed; } -- 2.4.3

1 1

[U-Boot] [PATCH] net: phy: breakdown PHY_*_FEATURES defines
by Alexey Brodkin 11 Jan '16

11 Jan '16

From: Florian Fainelli <f.fainelli(a)gmail.com> Breakdown the PHY_*_FEATURES into per speed defines such that we can easily re-use them individually. This is a copy of patch from Linux kernel, see http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e… Signed-off-by: Florian Fainelli <f.fainelli(a)gmail.com> Signed-off-by: Alexey Brodkin <abrodkin(a)synopsys.com>i Cc: Joe Hershberger <joe.hershberger(a)ni.com> --- include/phy.h | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/include/phy.h b/include/phy.h index 66cf61b..b793e90 100644 --- a/include/phy.h +++ b/include/phy.h @@ -17,18 +17,26 @@ #define PHY_MAX_ADDR 32 -#define PHY_BASIC_FEATURES (SUPPORTED_10baseT_Half | \ - SUPPORTED_10baseT_Full | \ - SUPPORTED_100baseT_Half | \ - SUPPORTED_100baseT_Full | \ - SUPPORTED_Autoneg | \ +#define PHY_DEFAULT_FEATURES (SUPPORTED_Autoneg | \ SUPPORTED_TP | \ SUPPORTED_MII) -#define PHY_GBIT_FEATURES (PHY_BASIC_FEATURES | \ - SUPPORTED_1000baseT_Half | \ +#define PHY_10BT_FEATURES (SUPPORTED_10baseT_Half | \ + SUPPORTED_10baseT_Full) + +#define PHY_100BT_FEATURES (SUPPORTED_100baseT_Half | \ + SUPPORTED_100baseT_Full) + +#define PHY_1000BT_FEATURES (SUPPORTED_1000baseT_Half | \ SUPPORTED_1000baseT_Full) +#define PHY_BASIC_FEATURES (PHY_10BT_FEATURES | \ + PHY_100BT_FEATURES | \ + PHY_DEFAULT_FEATURES) + +#define PHY_GBIT_FEATURES (PHY_BASIC_FEATURES | \ + PHY_1000BT_FEATURES) + #define PHY_10G_FEATURES (PHY_GBIT_FEATURES | \ SUPPORTED_10000baseT_Full) -- 2.4.3

1 1

[U-Boot] [PATCH] net/designware: do explicit port selection for 1Gb mode
by Alexey Brodkin 11 Jan '16

11 Jan '16

Current implementation only sets "port select" bit for non-1Gb mode. That works fine if GMAC has just exited reset state but we may as well change connection mode in runtime. Then we'll need to reprogram GMAC for that new mode of operation and if previous mode was 10 or 100 Mb and new one is 1 Gb we'll need to reset port mode bit. Signed-off-by: Alexey Brodkin <abrodkin(a)synopsys.com> Cc: Bin Meng <bmeng.cn(a)gmail.com> Cc: Joe Hershberger <joe.hershberger(a)ni.com> Cc: Sonic Zhang <sonic.zhang(a)analog.com> cc: Simon Glass <sjg(a)chromium.org> --- drivers/net/designware.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/designware.c b/drivers/net/designware.c index a6c39c3..58c6d36 100644 --- a/drivers/net/designware.c +++ b/drivers/net/designware.c @@ -200,6 +200,8 @@ static void dw_adjust_link(struct eth_mac_regs *mac_p, if (phydev->speed != 1000) conf |= MII_PORTSELECT; + else + conf &= ~MII_PORTSELECT; if (phydev->speed == 100) conf |= FES_100; -- 2.4.3

2 2

[U-Boot] [PATCH] include/net.h: add max_speed member in struct eth_pdata
by Alexey Brodkin 11 Jan '16

11 Jan '16

This will be used for getting max speed mode of Ethernet interface that a particular MAC supports from Device Tree blob and later being used for phy configuration. Signed-off-by: Alexey Brodkin <abrodkin(a)synopsys.com> Cc: Joe Hershberger <joe.hershberger(a)ni.com> --- include/net.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/include/net.h b/include/net.h index ebed29a..7dbba09 100644 --- a/include/net.h +++ b/include/net.h @@ -86,11 +86,13 @@ enum eth_state_t { * @iobase: The base address of the hardware registers * @enetaddr: The Ethernet MAC address that is loaded from EEPROM or env * @phy_interface: PHY interface to use - see PHY_INTERFACE_MODE_... + * @max_speed: Maximum speed of Ethernet connection supported by MAC */ struct eth_pdata { phys_addr_t iobase; unsigned char enetaddr[6]; int phy_interface; + int max_speed; }; enum eth_recv_flags { -- 2.4.3

1 1

[U-Boot] Accessing a file on a NAND JFFS2 partition from u-boot
by Marco 11 Jan '16

11 Jan '16

Hello,I need to retrieve I have some settings from a file stored in one of the JFFS2 partitions, from within u-boot.Can someone please give me a hint on how to configure u-boot to do so and which command should I then use? thanksmarco

1 0