Hi Heinrich,
On Mon, 8 Feb 2021 at 00:25, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 2/8/21 5:05 AM, Simon Glass wrote:
Tests are supposed to be independent. With driver model tests, the environment is reset before each test, which ensures that.
With Python tests there is no reset of the board between tests, since we want to run all the tests as quickly as possible and without needing the external scripts running constantly.
In principle the Python tests can be independent if they each put the world back the way they found it, but it turns out that some are not. This means that some tests cannot be run unless another test is run first. It also means that tests cannot be run in parallel, e.g. on sandbox.
This series fixes some of them. Those that remain:
test_gpt_swap_partitions - not sure? test_pinmux_status - not sure? test_sqfs_load - cannot be run more than once! test_bind_unbind_with_uclass - relies on previous testThe last one would be much better done as a C test, so it doesn't have to deal with the changing driver tree. There isn't a lot of value in running the test on a real board, since sandbox should find any bugs in driver model or the 'bind' command.
If the above can be resolved we can enable parallel tests. On my test machine (32 threads) it reduces the time from 38 seconds to 7.5s
To use this feature: pip3 install pytest-xdist
test/py/test.py -B sandbox --build-dir /tmp/xx -q -k 'not slow' -n32Thanks for looking into parallelization.
What I am missing in this series is a patch for doc/develop/py_testing.rst describing how parallelization of Python tests is controlled.
I don't think we can mention that yet, as it doesn't actually work.
I have seen that test/py/tests/test_fs/test_basic.py test_fs1() is always failing on my machine because the test file 2.5GB.file is truncated. It is not truncated if I add some waiting time.
Could this be caused by parallelization?
Not at present since we don't use it. I seldom run those tests. I wonder whether they work once and then not again, i.e. something needs to be reset at the start of that test?
Also I have not even tried to parallelise those tests. For me they use too much memory.
Package 'python3-pytest-xdist' is not installed on my system.
OK. Then the -n flag is not available.
Regards, Simon