Hello Stephen,
Am 02.12.2015 um 23:18 schrieb Stephen Warren:
This tool aims to test U-Boot by executing U-Boot shell commands using the console interface. A single top-level script exists to execute or attach to the U-Boot console, run the entire script of tests against it, and summarize the results. Advantages of this approach are:
- Testing is performed in the same way a user or script would interact with U-Boot; there can be no disconnect.
- There is no need to write or embed test-related code into U-Boot itself. It is asserted that writing test-related code in Python is simpler and more flexible that writing it all in C.
- It is reasonably simple to interact with U-Boot in this way.
A few simple tests are provided as examples. Soon, we should convert as many as possible of the other tests in test/* and test/cmd_ut.c too.
In the future, I hope to publish (out-of-tree) the hook scripts, relay control utilities, and udev rules I will use for my own HW setup.
See README.md for more details!
Signed-off-by: Stephen Warren swarren@wwwdotorg.org Signed-off-by: Stephen Warren swarren@nvidia.com
Nice work!
I am working on another python approach, not only good for testing u-boot, also works with linux, or other console based tests, see:
[1] tbot https://github.com/hsdenx/tbot
With tbot it is possible to connect to a lab PC over ssh. So no need to have the hw physically accessible. Currently there is only one lab, the denx lab in munich (I live in hungary and tbot runs at my home),
It should be really easy to add other labs, if others want to join. see: https://github.com/hsdenx/tbot/blob/master/doc/howto_add_new_lab.txt
I set up nightly builds using buildbot [2] http://buildbot.net/ on my rasspyberry pi at my home (!low dsl speed, especially if my kids play games, so if you cannot see the webpage press F5 again ;-)
[3] http://xeidos.ddns.net/buildbot/waterfall
For every build there is a logfile, also shell log. The logfile is generated from tbot. It is possible to define different loglevel, as tbot uses "logging" python module. Each testcase can add logfile messages.
Testcases can call other testcases, so you can do very small testcases, and combinate them to bigger ones ...
As tbot defines board states, it is possible to switch between u-boot and linux tests, and tbot will switch automatically to the board state the testcase is defined for ... a testcase has only to call https://github.com/hsdenx/tbot/blob/master/src/common/tbotlib.py#L677
So it is for example easy to set a date in u-boot, switch to board state linux, check there the date, set another date in linux, and switch to u-boot and check there again the time ... (had such a testcase, but not for current version of tbot)
It would be great to have a lot of labs accessible with board(s) in it for u-boot/linux tests and make there automated nightly builds ...
Currently there are only basic tbot testcases, but some good examples, what is possible: - smartweb dfu test: https://github.com/hsdenx/tbot/blob/master/src/tc/tc_board_smartweb.py - compile u-boot source from current mainline - install it on the board - make dfu tests with dfu-util installed on the PC in the Lab
- ari_ubi https://github.com/hsdenx/tbot/blob/master/src/tc/tc_board_aristainetos2.py - compile current mainline source for the aristainetos2 board - install it on the board - make u-boot ubi/ubifs test on the nand and spi nor
- tqm5200s https://github.com/hsdenx/tbot/blob/master/src/tc/tc_board_tqm5200s_try_cur_... - compile current mainline source for the tqm5200s board - install it on the board with BDI - boot u-boot (currently fail, bugfix is posted on the ML)
- interesting testcase here: https://github.com/hsdenx/tbot/blob/master/src/tc/tc_board_git_bisect.py This testcase checks out a git tree, and start a "git bisect" with starting a testcase on the board for detecting if current version is good or bad. At the end, you get the info, which commit breaks ... so detected the fail for the tqm5200s, see: http://xeidos.ddns.net/buildbot/builders/tqm5200s/builds/3/steps/shell/logs/... If I find time I want to start this tc automatically in the nightly build if a testcase for a board fails. So we get for free every night breaking commits tested on real boards ...
- there are also linux test on [3] ...
- register check (used on the mcx board for pinmux on the ccu1 board for pinmux and gpmc register) https://github.com/hsdenx/tbot/blob/master/src/tc/tc_lx_check_reg_file.py checks the registers defined in a register file with current settings (use devmem2 for this task) register file example: https://github.com/hsdenx/tbot/blob/master/src/files/ccu1_pinmux_scm.reg create this file with testcase: https://github.com/hsdenx/tbot/blob/master/src/tc/tc_lx_create_reg_file.py
- https://github.com/hsdenx/tbot/blob/master/src/tc/tc_board_sirius_dds.py The sirius dds board had problems with ubi/ubifs and power cuts. This testcase exeutes 50 power cuts, if no error detected it succeeds, else fail. The above testcase does: - go into state u-boot - start linux with ubifs as rootfs line 40 (as it is an old testcase, this should be better "tb.set_board_state("linux")" - wait until Userspace APP SiriusApplicat is started - wait random seconds (3 -10) - power off the board - wait 3 seconds for powering really of the board - loop this 50 times if we have an error in this steps, testcase ends with error
For this testcase 56 lines needed (more as the half are comments)
Basic framework is running, time for adding more testcases is needed...
bye, Heiko