Hello Tom,
Am 19.02.2019 um 00:53 schrieb Tom Rini:
Hey all,
While I'm not the best at taking notes, and for next time I'll see how badly the record function slows down my connection, here's my notes / action items from the call:
Attendees: Myself, Philipp Tomsich, Michal Simek, Stefano Babic, Vignesh R, Jagan Teki, Lokesh Vutla, Matthias Brugger, Masahiro Yamada.
Talked about more automated testing, ala kernelci. I noted that we have travis-ci setup, and with test.py that runs on on qemu on a number of platforms. With respect to real hardware, I run mine on a handful of TI platforms and RPi 3 (I've tried sunxi before, but had power vampire problems, and imx6, but my platform wasn't supported in SPL at the time). Michal noted he has a setup for some of his platforms.
I also have/had a running testsetup with tbot, see: http://xeidos.ddns.net/tests/test_db_auslesen.php#987
But as you see, dead for newer tests, as I had no time to fixup the at91 boards for example, for which wdt is broken. Yeah, testing *is* a time consuming task, as you always find something, which does not work.
I just can vote for using tbot, added Harald Seiler to cc, as he rewrote tbot completly in python 3.6, see [1] and [2], feel free to take a look at it. I really like the interactive testcases for U-Boot and linux ... you start tbot and you are on the boards U-Boot or linux commandline ...
I start for example test/py with tbot, or on the U-Boot commandline the "ut all" command if enabled (or enable the ut command with tbot before compiling U-Boot). You do not need to have the boards you want to test near you, it could be around the world (in my case I run tbot and the above webpage on a raspberry pi in hungary, while the boards I test are in munich/germany)
The "problem" with tbot is, that it activly opens a ssh connection to a so called Lab Host see [3] and a lot of customers do not allow to use this for automated testing purposes ...
But I talked with Kevin Hillman some months ago, and he mentioned that it should be possible to run kernelci without LAVA, so may it is worth to look into kernelci and use it for "ubootci" purposes. Than we would have an API for reporting testresults, and can start for example tbot locally without the need of an ssh connection. It should be easy than to generate a testreport for kernelci with a tbot generator, see [4] for generator examples. I am sorry that I did not found time yet, for more investigations...
And customers are more willing to send testresults to us instead to allow ssh access for testing... hopefully.
bye, Heiko
[1] https://github.com/Rahix/tbot [2] https://rahix.de/tbot/getting-started.html [3] https://rahix.de/tbot/index.html [4] https://github.com/Rahix/tbot/tree/master/generators