Hi Mattijs,
On Thu, 16 Jan 2025 at 02:47, Mattijs Korpershoek mkorpershoek@baylibre.com wrote:
Hi Simon,
Thank you for the series.
On mer., janv. 15, 2025 at 06:30, Simon Glass sjg@chromium.org wrote:
The current method of running unit tests relies on subcommands of the ut command. Only the code in each subcommand knows how to find the tests related to that subcomand.
This is not ideal and we now have quite a few subcommands which do nothing but locate the relevant tests in a linker list, then call a common function to run them.
This series adds a list of test suites, so that these subcommands can be removed.
An issue with 'ut all' is that it doesn't record how many tests failed overall, so it is necessary to examine copious amounts of output to look for failures. This series adds a new 'total' feature allow recording the total number of failed tests.
To help with 'ut all' a new pytest is created which runs it (as well as 'ut info') and makes sure that all is well. Due to the 'ut all' failures this does not pass, so the test is disabled for now. It is here because it provides security against misnaming a test suite and causing it not to run.
I wanted to test this, but it seems it does not apply on both master or next:
With master: 178f6ecb21fe ("Merge patch series "bootstd: Support recording images"") With next: d6da3dbaef57 ("Merge patch series "cmd: Add support for optee commands."")
When running: $ b4 shazam -s -l --check 20250115133114.590375-1-sjg@chromium.org
I see:
""" [...] snip
Total patches: 43
Applying: test: Drop unused suite prototypes Applying: test: Rename test_get_state() to ut_get_state() Applying: test: Add functions to init and uninit the test state Applying: test: Pass the test-state into ut_run_list() Applying: Improve support for linker lists in data structures Applying: test: Add newlines to hush-test messages Applying: test: Rename test suites to match their linker-list name Applying: test: Update ut info to show suites Applying: test/py: Add a test which runs all unit tests Applying: test: Drop the _test suffix on linker lists Applying: test: Introduce a better array of test suites Patch failed at 0011 test: Introduce a better array of test suites error: sha1 information is lacking or useless (test/cmd_ut.c). error: could not build fake ancestor hint: Use 'git am --show-current-patch=diff' to see the failed patch hint: When you have resolved this problem, run "git am --continue". hint: If you prefer to skip this patch, run "git am --skip" instead. hint: To restore the original branch and stop patching, run "git am --abort". """
On what base should this be applied if we want to test this? Sorry if this is a sily question, usually I get dependencies from the cover letter but I could not find any here.
Yes, it is supposed to be based on -next but there are two patches in -next but not in the tree I used:
90856d695f2 test: sandbox: fix link error with do_ut_bootm if BLOBLIST=n 6970eeb7885 test: sandbox: fix link error with do_ut_bloblist if BLOBLIST=n
The series is available at sjg.u-boot.org/u-boot/u-boot/-/tree/tes?ref_type=heads
I'll sort this out when I send v2, but will wait for some more feedback.
[..]
Regards, SImon