This series makes a further attempt to get closer to having all tests run in parallel. It introduces a new 'make pcheck' option which runs tests in parallel, skipping those that are not compatible.
A number of fixes are included for existing tests. The vboot test is updated to only run a single scenario in 'quick' mode.
This makes use of pytest's parallel-testing features. The resulting times (including incremental building with LTO) on a 16-core machine are as follows:
make pcheck - 1 minute 6 seconds make qcheck - 3 minutes make check - 5 minutes 15 seconds
Note that this is not a fair comparison, since 'make pcheck' omits a number of tests, even more than 'make qcheck'.
Simon Glass (15): test: Fix test_pinmux to run in parallel test: Fix bootm_test_subst_var() running independently test/py: Allow tests to be marked single-threaded only test: Make test_bind_unbind_with_uclass() single-threaded test: Update FIT tests to run in parallel test: Make test_sqfs_ls() single-threaded test: Make test_efi_bootmgr() single-threaded test: Mark all but the first vboot test as slow test: Mark test_gpt tests as slow test: Make test_gpio_read() independent test: Refactor arg parsing for the run script test/py: Move U-Boot building into a function test/py: Support --build when running tests in parallel dtoc: Drop sys.exit() in test_fdt Makefile: Add a pcheck option to run tests in parallel
Makefile | 8 +- doc/develop/py_testing.rst | 37 ++++--- doc/develop/testing.rst | 6 +- test/bootm.c | 3 +- test/py/conftest.py | 104 +++++++++++++----- test/py/pytest.ini | 1 + test/py/requirements.txt | 2 + test/py/tests/test_bind.py | 1 + .../test_efi_bootmgr/test_efi_bootmgr.py | 1 + test/py/tests/test_fit_ecdsa.py | 4 +- test/py/tests/test_fit_hashes.py | 5 +- .../test_fs/test_squashfs/test_sqfs_ls.py | 1 + test/py/tests/test_gpio.py | 1 + test/py/tests/test_gpt.py | 3 + test/py/tests/test_pinmux.py | 1 + test/py/tests/test_vboot.py | 6 +- test/run | 51 +++++++-- tools/dtoc/test_fdt.py | 1 - 18 files changed, 173 insertions(+), 63 deletions(-)