Please pull u-boot-dm

Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
---------------------------------------------------------------- misc fixes buildman refactoring (no functional change)
---------------------------------------------------------------- Eugen Hristev (1): dm: core: of_access: fix return value in of_property_match_string
Heinrich Schuchardt (1): cmd: fix loads, saves on sandbox
John Clark (1): bootstd: USB devtype detection for script boot
John Keeping (1): core: read: fix dev_read_addr_size()
Maxim Cournoyer (2): tools: Fix README file in pyproject.toml of u_boot_pylib. tools: Fix package discovery in pyproject.toml of u_boot_pylib.
Sergei Antonov (1): sandbox: fix a compilation error
Simon Glass (62): buildman: Fix verboose typo and add comment buildman: Provide an argument to the -R option buildman: Tidy up pylint warnings in main buildman: Convert camel case in control.py buildman: Fix most pylint warnings in control buildman: Move full-help processing to main buildman: Move series calculations into a separate function buildman: Move fetch-arch code into a separate function buildman: Add a test for the -A option buildman: Drop use of builder in show_actions() buildman: Move dry-run handling higher in do_buildman() buildman: Move board-selection code into a function buildman: Move more code to determine_series() buildman: Move Boards-object code into a function buildman: Move toolchain handling to a function buildman: Set up output_dir earlier buildman: Move output-file setup into one place buildman: Pass option values to get_action_summary() buildman: Pass option values to show_actions() buildman: Build option-adjusting into a function buildman: Move counting of commits into a function buildman: Move setting up the output dir into a function buildman: Move commit numbering into determine_series() buildman: Avoid too many returns in do_buildman() buildman: Move remaining builder properties to constructor buildman: Tweak commits and show_bloat buildman: Moving running of the builder into a function buildman: Drop some unnecessary variables buildman: Adjust show_toolchain_prefix() to not return buildman: Move checking for make into run_builder() buildman: Move getting the adjust_cfg into run_builder() buildman: Use get_alow_missing() directly to avoid var buildman: Create a function to get number of built commits buildman: Convert camel case in cmdline.py buildman: Correct most pylint warnings in cmdline buildman: Add a test for --boards buildman: Convert to argparse buildman: Convert camel case in bsettings.py buildman: Convert camel case in builder.py buildman: Split parser creation in two buildman: Convert camel case in builderthread.py buildman: Correct most pylint warnings in builderthread buildman: Export _get_output_dir() to avoid warnings buildman: Correct invalid use of out_dir variable buildman: Drop unnecessary assignment of config_out buildman: Start a function to set up the make arguments buildman: Move setting of toolchain arguments to _build_args() buildman: Move more things into _build_args() buildman: Convert config_out to string IO buildman: Move reconfigure code into its own function buildman: Move bulid code into its own function buildman: Move reading of the done file into a function buildman: Move code to remove old outputs buildman: Move code to decide output dirs buildman: Move checkout code to a separate function buildman: Create a function to handle config and build buildman: Avoid passing result into _read_done_file() buildman: Tidy up reporting of a toolchain error buildman: Tidy up some comments in builderthread buildman: Move copy_files() out ot BuilderThread class buildman: Add a way to print the architecture for a board buildman: Enable test coverage
.azure-pipelines.yml | 2 +- .gitlab-ci.yml | 2 +- arch/sandbox/include/asm/sdl.h | 23 +++ arch/sandbox/include/asm/test.h | 25 --- boot/bootmeth_script.c | 5 +- cmd/load.c | 16 +- drivers/core/of_access.c | 5 +- drivers/core/read.c | 5 +- drivers/reset/reset-rockchip.c | 2 +- include/dm/read.h | 12 +- test/dm/video.c | 1 + tools/buildman/bsettings.py | 14 +- tools/buildman/builder.py | 262 +++++++++++++++-------------- tools/buildman/builderthread.py | 652 ++++++++++++++++++++++++++++++++++++++++++++---------------------------- tools/buildman/buildman.rst | 9 +- tools/buildman/cmdline.py | 172 +++++++++++-------- tools/buildman/control.py | 778 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------- tools/buildman/func_test.py | 76 +++++++-- tools/buildman/main.py | 66 +++++--- tools/buildman/test.py | 28 ++-- tools/buildman/toolchain.py | 14 +- tools/moveconfig.py | 2 +- tools/u_boot_pylib/pyproject.toml | 6 +- tools/u_boot_pylib/test_util.py | 10 +- 24 files changed, 1360 insertions(+), 827 deletions(-)
Regards, Simon

On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option
Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller.
Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass sjg@chromium.org
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".

On Wed, Jul 12, 2023 at 03:29:39PM -0400, Tom Rini wrote:
On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller. Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".
And even harder NAK, this commit 075c6a5eb53f8fb8d375373293c92396f3b4a448 Author: Simon Glass sjg@chromium.org Date: Wed Jul 5 13:18:38 2023 +0100
buildman: Move board-selection code into a function
Create a new determine_boards() function to hold the code which selects which boards to build.
Signed-off-by: Simon Glass sjg@chromium.org
Breaks "buildman foo -x bar" and so all of the Azure jobs that exclude things don't exclude things at all anymore. For example: ./tools/buildman/buildman --dry-run -v freescale -x powerpc,m68k,aarch64,ls101,ls102,ls104,ls108,ls20,lx21 gives 229 boards and not 28.

Hi Tom,
On Wed, 12 Jul 2023 at 13:36, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 03:29:39PM -0400, Tom Rini wrote:
On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller. Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".
From the docs:
This file is no-longer needed by buildman but it is still generated in the working directory. This helps avoid a delay on every build, since scanning all the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force regeneration of the file - in that case buildman exits after writing the file with exit code 2 if there was an error in the maintainer files. To use the default filename, use a hyphen, i.e. `-R -`.
And even harder NAK, this commit 075c6a5eb53f8fb8d375373293c92396f3b4a448 Author: Simon Glass sjg@chromium.org Date: Wed Jul 5 13:18:38 2023 +0100
buildman: Move board-selection code into a function Create a new determine_boards() function to hold the code which selects which boards to build. Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks "buildman foo -x bar" and so all of the Azure jobs that exclude things don't exclude things at all anymore. For example: ./tools/buildman/buildman --dry-run -v freescale -x powerpc,m68k,aarch64,ls101,ls102,ls104,ls108,ls20,lx21 gives 229 boards and not 28.
I'll take a look. One of the reasons this series adds test-coverage measurement is to see where there are gaps in testing.
Regards, Simon

On Wed, Jul 12, 2023 at 02:09:19PM -0600, Simon Glass wrote:
Hi Tom,
On Wed, 12 Jul 2023 at 13:36, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 03:29:39PM -0400, Tom Rini wrote:
On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller. Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".
From the docs:
This file is no-longer needed by buildman but it is still generated in the working directory. This helps avoid a delay on every build, since scanning all the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force regeneration of the file - in that case buildman exits after writing the file with exit code 2 if there was an error in the maintainer files. To use the default filename, use a hyphen, i.e. `-R -`.
The old name needs to be the default if none is specified, to not break existing setups.
And even harder NAK, this commit 075c6a5eb53f8fb8d375373293c92396f3b4a448 Author: Simon Glass sjg@chromium.org Date: Wed Jul 5 13:18:38 2023 +0100
buildman: Move board-selection code into a function Create a new determine_boards() function to hold the code which selects which boards to build. Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks "buildman foo -x bar" and so all of the Azure jobs that exclude things don't exclude things at all anymore. For example: ./tools/buildman/buildman --dry-run -v freescale -x powerpc,m68k,aarch64,ls101,ls102,ls104,ls108,ls20,lx21 gives 229 boards and not 28.
I'll take a look. One of the reasons this series adds test-coverage measurement is to see where there are gaps in testing.
OK.

Hi Tom,
On Wed, 12 Jul 2023 at 14:14, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 02:09:19PM -0600, Simon Glass wrote:
Hi Tom,
On Wed, 12 Jul 2023 at 13:36, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 03:29:39PM -0400, Tom Rini wrote:
On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller. Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".
From the docs:
This file is no-longer needed by buildman but it is still generated in the working directory. This helps avoid a delay on every build, since scanning all the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force regeneration of the file - in that case buildman exits after writing the file with exit code 2 if there was an error in the maintainer files. To use the default filename, use a hyphen, i.e. `-R -`.
The old name needs to be the default if none is specified, to not break existing setups.
Well if none is specified you get an error. I realised after adding -R that it writes the file to a place that cannot be determined without parsing the ~/.buildman file!
And even harder NAK, this commit 075c6a5eb53f8fb8d375373293c92396f3b4a448 Author: Simon Glass sjg@chromium.org Date: Wed Jul 5 13:18:38 2023 +0100
buildman: Move board-selection code into a function Create a new determine_boards() function to hold the code which selects which boards to build. Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks "buildman foo -x bar" and so all of the Azure jobs that exclude things don't exclude things at all anymore. For example: ./tools/buildman/buildman --dry-run -v freescale -x powerpc,m68k,aarch64,ls101,ls102,ls104,ls108,ls20,lx21 gives 229 boards and not 28.
I'll take a look. One of the reasons this series adds test-coverage measurement is to see where there are gaps in testing.
OK.
-- Tom
Regards, Simon

On Wed, Jul 12, 2023 at 02:17:45PM -0600, Simon Glass wrote:
Hi Tom,
On Wed, 12 Jul 2023 at 14:14, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 02:09:19PM -0600, Simon Glass wrote:
Hi Tom,
On Wed, 12 Jul 2023 at 13:36, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 03:29:39PM -0400, Tom Rini wrote:
On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
Hi Tom.
https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872
The following changes since commit 8e21064cb3452950b09301baec06d86e37342471:
Merge tag 'efi-2023-07-rc7' of https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 13:27:32 -0400)
are available in the Git repository at:
git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a
for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487:
tools: Fix package discovery in pyproject.toml of u_boot_pylib. (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller. Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".
From the docs:
This file is no-longer needed by buildman but it is still generated in the working directory. This helps avoid a delay on every build, since scanning all the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force regeneration of the file - in that case buildman exits after writing the file with exit code 2 if there was an error in the maintainer files. To use the default filename, use a hyphen, i.e. `-R -`.
The old name needs to be the default if none is specified, to not break existing setups.
Well if none is specified you get an error. I realised after adding -R that it writes the file to a place that cannot be determined without parsing the ~/.buildman file!
I noticed it now produces an error when it broke the part of my build scripts to check for new unmaintained defconfig files. And I don't really like adding incompatible changes to this script as it then breaks when I need to go and back track on issues. So please figure out what needs to happen for -R to take an optional name, so that existing scripts aren't broken.

Hi Tom,
On Wed, 12 Jul 2023 at 14:31, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 02:17:45PM -0600, Simon Glass wrote:
Hi Tom,
On Wed, 12 Jul 2023 at 14:14, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 02:09:19PM -0600, Simon Glass wrote:
Hi Tom,
On Wed, 12 Jul 2023 at 13:36, Tom Rini trini@konsulko.com wrote:
On Wed, Jul 12, 2023 at 03:29:39PM -0400, Tom Rini wrote:
On Wed, Jul 12, 2023 at 09:51:13AM -0600, Simon Glass wrote:
> Hi Tom. > > https://source.denx.de/u-boot/custodians/u-boot-dm/-/pipelines/16872 > > > The following changes since commit 8e21064cb3452950b09301baec06d86e37342471: > > Merge tag 'efi-2023-07-rc7' of > https://source.denx.de/u-boot/custodians/u-boot-efi (2023-07-11 > 13:27:32 -0400) > > are available in the Git repository at: > > git://git.denx.de/u-boot-dm.git tags/dm-pull-12jul23a > > for you to fetch changes up to 45aa7ac492d90ddc2977ea6c9d161e7cc3333487: > > tools: Fix package discovery in pyproject.toml of u_boot_pylib. > (2023-07-12 09:48:19 -0600)
NAK, commit 5a801ad450303f028213c5da12017db5d3668cbc Author: Simon Glass sjg@chromium.org Date: Thu Jun 22 12:44:24 2023 +0100
buildman: Provide an argument to the -R option Allow writing the file to a selected location, since otherwise this is controlled by the buildman configuration, so cannot be determined by the caller. Fixes: https://source.denx.de/u-boot/u-boot/-/issues/17 Signed-off-by: Simon Glass <sjg@chromium.org>
Breaks buildman -R as it now requires a location to be specified instead of creating one where it's normally needed / used by default.
And I believe this matters in that I use "buildman -R" for "Did someone add a new defconfig and not update a MAINATINERS file?".
From the docs:
This file is no-longer needed by buildman but it is still generated in the working directory. This helps avoid a delay on every build, since scanning all the Kconfig files takes a few seconds. Use the `-R <filename>` flag to force regeneration of the file - in that case buildman exits after writing the file with exit code 2 if there was an error in the maintainer files. To use the default filename, use a hyphen, i.e. `-R -`.
The old name needs to be the default if none is specified, to not break existing setups.
Well if none is specified you get an error. I realised after adding -R that it writes the file to a place that cannot be determined without parsing the ~/.buildman file!
I noticed it now produces an error when it broke the part of my build scripts to check for new unmaintained defconfig files. And I don't really like adding incompatible changes to this script as it then breaks when I need to go and back track on issues. So please figure out what needs to happen for -R to take an optional name, so that existing scripts aren't broken.
As per irc I'll create a new option for this, so we don't have to use -R anymore.
Regards, Simon
participants (2)
-
Simon Glass
-
Tom Rini