On Sat, Aug 12, 2023 at 09:14:45PM -0600, Simon Glass wrote:
Hi Tom,
I notice that the runners are not utilised much by the QEMU jobs, since we only run one at a time.
I wonder if we could improve this, perhaps by using a different tag for the QEMU ones and then having a machine that only runs those (and runs 40 in parallel)?
In general our use of the runners seems a bit primitive, since the main use of parallelism is in the world builds.
I'm honestly not sure. I think there's a few tweaks that we should do, like putting the opensbi and coreboot files in to the Dockerfile logic instead. And maybe seeing if just like we can have a docker registry cache, if we can setup local pypi cache too? I'm not otherwise sure what's taking 23 seconds or so of https://source.denx.de/u-boot/u-boot/-/jobs/673565#L34 since the build and run parts aren't much.
My first big worry about running 2 or 3 qemu jobs at the same time on a host is that any wins get from a shorter queue will be lost to buildman doing "make -j$(nproc)" 2 or 3 times at once and so we build slower.
My second big worry is that getting the right tags on runners will be a little tricky.
My third big worry (but this is something you can test easy enough at least) is that running the big sandbox tests, 2 or 3 times at once on the same host will get much slower. I think, but profiling would be helpful, that those get slow due to I/O and not CPU.