On Wed, Aug 28, 2024 at 04:08:26PM -0600, Simon Glass wrote:
Labgrid provides access to a hardware lab in an automated way. It is possible to boot U-Boot on boards in the lab without physically touching them. It relies on relays, USB UARTs and SD muxes, among other things.
By way of background, about 4 years ago I wrong a thing called Labman[1] which allowed my lab of about 30 devices to be operated remotely, using tbot for the console and build integration. While it worked OK and I used it for many bisects, I didn't take it any further.
It turns out that there was already an existing program, called Labgrid, which I did not know about at time (thank you Tom for telling me). It is more rounded than Labman and has a number of advantages:
- does not need udev rules, mostly
- has several existing users who rely on it
- supports multiple machines exporting their devices
It lacks a 'lab check' feature and a few other things, but these can be remedied.
On and off over the past several weeks I have been experimenting with Labgrid. I have managed to create an initial U-Boot integration (this series) by adding various features to Labgrid[2] and the U-Boot test hooks.
I hope that this might inspire others to set up boards and run tests automatically, rather than relying on infrequent, manual test. Perhaps it may even be possible to have a number of labs available.
Included in the integration are a number of simple scripts which make it easy to connect to boards and run tests:
ub-int <target> Build and boot on a target, starting an interactive session
ub-cli <target> Build and boot on a target, ensure U-Boot starts and provide an interactive session from there
ub-smoke <target> Smoke test U-Boot to check that it boots to a prompt on a target
ub-bisect <target> Bisect a git tree to locate a failure on a particular target
ub-pyt <target> <testspec> Run U-Boot pytests on a target
Some of these help to provide the same tbot[4] workflow which I have relied on for several years, albeit much simpler versions.
The goal here is to create some sort of script which can collect patches from the mailing list, apply them and test them on a selection of boards. I suspect that script already exists, so please let me know what you suggest.
I hope you find this interesting and take a look!
[1] https://github.com/sjg20/u-boot/tree/lab6a [2] https://github.com/labgrid-project/labgrid/pull/1411 [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid [4] https://tbot.tools/index.html
Part of my concern here is still that this series contains:
- Generic test improvements and fixes:
test: Use a constant for the test timeout test: Pass stderr to stdout test: Avoid failing skipped tests test: Move the receive code into a function test: Separate out the exception handling test: Detect dead connections test: Tidy up remaining exceptions test: Improve handling of sending commands test: Fix mulptiplex_log typo test: Avoid double echo when starting up
- Labgrid specific listed as generic changes:
test: Release board after tests complete test: Try to shut down the lab console gracefully
- Your labgrid+builman implementation specific changes:
test: Allow signaling that U-Boot is ready test: Allow connecting to a running board test: Create a common function to get the config test: Introduce the concept of a role test: Introduce lab mode test: Add a section for closing the connection test: Support testing with two board-builds
Which makes this harder to review and clearly merge. For example, we already have hooks for power on / power off, and we should just call them as wow that would make everyones lab easier (I used to have a pre-script to turn on everything).