Hi Heinrich,
On Thu, 13 Oct 2022 at 10:19, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 10/13/22 14:28, Simon Glass wrote:
At present we normally write tests either in Python or in C. But most Python tests end up doing a lot of checks which would be better done in C. Checks done in C are orders of magnitude faster and it is possible to get full access to U-Boot's internal workings, rather than just relying on the command line.
The model is to have a Python test set up some things and then use C code (in a unit test) to check that they were done correctly. But we don't want those checks to happen as part of normal test running, since each C unit tests is dependent on the associate Python tests, so cannot run without it.
To acheive this, add a new UT_TESTF_MANUAL flag to use with the C 'check' tests, so that they can be skipped by default when the 'ut' command is used. Require that tests have a name ending with '_norun', so that pytest
Why do want to use a naming convention?
See the line immediately above. The pytests don't have access to the flags as they are compiled into the executable.
What will we do when we want more flags like e.g. "slow"?
We already have that in the pytest side. This 'manual' thing is unusual in that both pytest and C test need to have it.
Adding more fields to struct unit_test would be more future prove.
Well a flag is more compact and suits the purpose. See above for why we need to add something to the name.
Regards, Simon
[..]