On Wed, Mar 03, 2021 at 02:37:51PM -0500, Tom Rini wrote:
On Wed, Feb 03, 2021 at 05:44:06AM -0700, Simon Glass wrote:
At present U-Boot has two broad sets of tests in the C code: driver model tests which do a lot of pre-/post-init and command tests which do not.
This separation makes it slightly harder to write a test, since there are two different test-state structures and different rules for running the two different test types. At present these rules are determined by where the test is (actually its prefix).
All unit tests can be run from the command line with the 'ut' command. Since SPL does not have commands, it currently calls the test runner directly and offers no control of which tests are run.
This seems like a good time to refactor the tests into a unified test runner, allowing U-Boot proper and SPL to use the same path, perhaps with some different conditions along the way.
This series sets up a unified runner called ut_run_list(), which runs a set of tests from a linker_list. Driver model tests are distinguished by a new UT_TESTF_DM flag so that the necessary init and cleanup can still be done.
The runner is modified to support running SPL tests that are not solely for driver model. An example test for FIT loading is added as a demonstration.
In addition, some documentation is added to explain how to write tests.
This series is available at u-boot-dm/test-working
Changes in v3:
- Add new patch to re-enable test_ofplatdata
- Reword the SPL tests section for clarity
- Use test_set_state() throughout test-main.c instead of direct assignment
- Update the pytest collector as well
Changes in v2:
- Use correct rst format for 'Ad-hoc tests' section
- Expand docs on how each type of test is marked
- Put the docs in tests_sandbox since it is more related to sandbox
- Put in a mention of tests_sandbox in the main testing docs
- Add a note that SPL tests can in fact be run individualy
- Document how to run all C tests with 'ut all'
- Fix 'get list' typo
- Fix conditions so non-DM SPL tests are actually run
- Allow for prefix to be NULL, to match function comment
- Add new patches to cover running an SPL test
There was a small problem applying one or two of these patches that I fixed up. But then when testing locally (Ubuntu 18.04, but virtualenv setup and populated with test/py/requirements.txt) like 200 tests failed to run for Sandbox. Please check this and that CI runs.
Er, something has gone odd for me locally, please disregard.