[PATCH] buildman: Add a way to build a particular target
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org ---
tools/buildman/builder.py | 4 +++- tools/buildman/builderthread.py | 2 ++ tools/buildman/buildman.rst | 4 ++++ tools/buildman/cmdline.py | 3 +++ tools/buildman/control.py | 5 ++++- tools/buildman/func_test.py | 10 ++++++++++ 6 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/tools/buildman/builder.py b/tools/buildman/builder.py index cbf1345281b..380c4057613 100644 --- a/tools/buildman/builder.py +++ b/tools/buildman/builder.py @@ -265,7 +265,7 @@ class Builder: reproducible_builds=False, force_build=False, force_build_failures=False, force_reconfig=False, in_tree=False, force_config_on_failure=False, make_func=None, - dtc_skip=False): + dtc_skip=False, build_target=None): """Create a new Builder object
Args: @@ -315,6 +315,7 @@ class Builder: retrying a failed build make_func (function): Function to call to run 'make' dtc_skip (bool): True to skip building dtc and use the system one + build_target (str): Build target to use (None to use the default) """ self.toolchains = toolchains self.base_dir = base_dir @@ -363,6 +364,7 @@ class Builder: raise ValueError('Cannot find dtc') else: self.dtc = None + self.build_target = build_target
if not self.squash_config_y: self.config_filenames += EXTRA_CONFIG_FILENAMES diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index 29e6cf32af1..5c61d9c81bd 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -286,6 +286,8 @@ class BuilderThread(threading.Thread): """ if config_only: args.append('cfg') + elif self.builder.build_target: + args.append(self.builder.build_target) result = self.make(commit, brd, 'build', cwd, *args, env=env) cmd_list.append([self.builder.gnu_make] + args) if (result.return_code == 2 and diff --git a/tools/buildman/buildman.rst b/tools/buildman/buildman.rst index 924564b5700..a6a44d8f3ba 100644 --- a/tools/buildman/buildman.rst +++ b/tools/buildman/buildman.rst @@ -1329,6 +1329,10 @@ sometimes useful to have buildman wait until the others have finished. Use the --process-limit option for this: --process-limit 1 will allow only one buildman to process jobs at a time.
+To build a particular target, rather than the default U-Boot target, use the +`--target` option. This is unlikely to be useful unless you are building a +single board. + Build summary -------------
diff --git a/tools/buildman/cmdline.py b/tools/buildman/cmdline.py index 7573e5bdfe8..9236d6187cf 100644 --- a/tools/buildman/cmdline.py +++ b/tools/buildman/cmdline.py @@ -22,6 +22,7 @@ def add_upto_m(parser):
This is split out to avoid having too many statements in one function """ + # Available JqzZ parser.add_argument('-a', '--adjust-cfg', type=str, action='append', help='Adjust the Kconfig settings in .config before building') parser.add_argument('-A', '--print-prefix', action='store_true', @@ -153,6 +154,8 @@ def add_after_m(parser): parser.add_argument('-T', '--threads', type=int, default=None, help='Number of builder threads to use (0=single-thread)') + parser.add_argument('--target', type=str, + default=None, help='Build target to use') parser.add_argument('-u', '--show_unknown', action='store_true', default=False, help='Show boards with unknown build result') parser.add_argument('-U', '--show-environment', action='store_true', diff --git a/tools/buildman/control.py b/tools/buildman/control.py index 55d4d770c5c..fb7bcbc43bf 100644 --- a/tools/buildman/control.py +++ b/tools/buildman/control.py @@ -785,6 +785,9 @@ def do_buildman(args, toolchains=None, make_func=None, brds=None, args.verbose) return 0
+ if args.config_only and args.target: + raise ValueError('Cannot use --config-only with --target') + # Create a new builder with the selected args builder = Builder(toolchains, output_dir, git_dir, args.threads, args.jobs, checkout=True, @@ -810,7 +813,7 @@ def do_buildman(args, toolchains=None, make_func=None, brds=None, force_build_failures = args.force_build_failures, force_reconfig = args.force_reconfig, in_tree = args.in_tree, force_config_on_failure=not args.quick, make_func=make_func, - dtc_skip=args.dtc_skip) + dtc_skip=args.dtc_skip, build_target=args.target)
TEST_BUILDER = builder
diff --git a/tools/buildman/func_test.py b/tools/buildman/func_test.py index 4e12c671a3d..03a4536d69c 100644 --- a/tools/buildman/func_test.py +++ b/tools/buildman/func_test.py @@ -1152,3 +1152,13 @@ CONFIG_SOC="fred" 'board': 'ARM Board 0', 'config': 'config0', 'target': 'board0'}, []), res) + + def testTarget(self): + """Test that the --target flag works""" + lines = self.check_command('--target', 'u-boot.dtb')[0] + + # It should not affect the defconfig line + self.assertNotIn(b'u-boot.dtb', lines[0]) + + # It should appear at the end of the build line + self.assertEqual(b'u-boot.dtb', lines[1].split()[-1])
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
But yes, we could add more things to buildman, e.g. it could output the initial env as a file for later parsing.
Regards, Simon
On Tue, Dec 17, 2024 at 12:45:29PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
I don't recall if there's a good reason for imx8 not setting CONFIG_BUILD_TARGET to something, but that sounds like how to solve this problem. "make all" should work, normally.
But yes, we could add more things to buildman, e.g. it could output the initial env as a file for later parsing.
Well, your patch supports that, if I read it right. It just doesn't save it.
Hi Tom,
On Tue, 17 Dec 2024 at 12:53, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 12:45:29PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
I don't recall if there's a good reason for imx8 not setting CONFIG_BUILD_TARGET to something, but that sounds like how to solve this problem. "make all" should work, normally.
Even if it did, the build fails the first time. It just needs to be fixed/
But yes, we could add more things to buildman, e.g. it could output the initial env as a file for later parsing.
Well, your patch supports that, if I read it right. It just doesn't save it.
Yes, it would be pretty easy to provide a flag that builds that target, runs the executable and puts the environment in a file.
Regards, Simon
On Tue, Dec 17, 2024 at 04:42:30PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 12:53, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 12:45:29PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
I don't recall if there's a good reason for imx8 not setting CONFIG_BUILD_TARGET to something, but that sounds like how to solve this problem. "make all" should work, normally.
Even if it did, the build fails the first time. It just needs to be fixed/
Yes, there's both the problem of CONFIG_BUILD_TARGET not being set and then whatever issues there are with configuring access to the blobs especially when they exist and aren't faked via BINMAN_ALLOW_MISSING. IIRC Toradex does weekly builds of top of tree, via OpenEmbedded / Yocto so there's not some bitrot here to be fixed, just smoothing out integration.
But yes, we could add more things to buildman, e.g. it could output the initial env as a file for later parsing.
Well, your patch supports that, if I read it right. It just doesn't save it.
Yes, it would be pretty easy to provide a flag that builds that target, runs the executable and puts the environment in a file.
Er, "make u-boot-initial-env" does that. So "--target u-boot-initial-env" should suffice, if that was also saved.
Hi Tom,
On Tue, 17 Dec 2024 at 16:54, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 04:42:30PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 12:53, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 12:45:29PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
At present buildman only supports building the default target. Generally this is what is wanted, but in some cases boards erroneously have a different target for product extra files.
Add a --target option to help. Also add a comment indicating which letters are free for new options.
Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
I don't recall if there's a good reason for imx8 not setting CONFIG_BUILD_TARGET to something, but that sounds like how to solve this problem. "make all" should work, normally.
Even if it did, the build fails the first time. It just needs to be fixed/
Yes, there's both the problem of CONFIG_BUILD_TARGET not being set and then whatever issues there are with configuring access to the blobs especially when they exist and aren't faked via BINMAN_ALLOW_MISSING. IIRC Toradex does weekly builds of top of tree, via OpenEmbedded / Yocto so there's not some bitrot here to be fixed, just smoothing out integration.
To me, BUILD_TARGET is a pre-Binman construction and we should just use binman to make sure that the default target builds a bootable image.
It's good to hear that Toradex has CI going, but it looks like there is a workaround in there.
But yes, we could add more things to buildman, e.g. it could output the initial env as a file for later parsing.
Well, your patch supports that, if I read it right. It just doesn't save it.
Yes, it would be pretty easy to provide a flag that builds that target, runs the executable and puts the environment in a file.
Er, "make u-boot-initial-env" does that. So "--target u-boot-initial-env" should suffice, if that was also saved.
Oh, yes. Either I didn't know that or I forgot...
Regards, Simon
On Thu, Dec 19, 2024 at 08:07:22AM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 16:54, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 04:42:30PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 12:53, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 12:45:29PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote:
> At present buildman only supports building the default target. Generally > this is what is wanted, but in some cases boards erroneously have a > different target for product extra files. > > Add a --target option to help. Also add a comment indicating which > letters are free for new options. > > Signed-off-by: Simon Glass sjg@chromium.org
How do you use this? It would be nice to be able to do "--target u-boot-initial-env" for example for converting and verifying the conversion of boards to plain text environment. Except the list of artifacts that are kept isn't updated with that string as well. And FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
I don't recall if there's a good reason for imx8 not setting CONFIG_BUILD_TARGET to something, but that sounds like how to solve this problem. "make all" should work, normally.
Even if it did, the build fails the first time. It just needs to be fixed/
Yes, there's both the problem of CONFIG_BUILD_TARGET not being set and then whatever issues there are with configuring access to the blobs especially when they exist and aren't faked via BINMAN_ALLOW_MISSING. IIRC Toradex does weekly builds of top of tree, via OpenEmbedded / Yocto so there's not some bitrot here to be fixed, just smoothing out integration.
To me, BUILD_TARGET is a pre-Binman construction and we should just use binman to make sure that the default target builds a bootable image.
Well, no. Assuming that doc/board/toradex/colibri-imx8x.rst is correct then CONFIG_BUILD_TARGET should be set to "u-boot-dtb.imx" so that "make" just works like other platforms. If there's some other target actually in use, that should be what's put in CONFIG_BUILD_TARGETS and in turn binman will get invoked as needed.
Hi Tom,
On Thu, 19 Dec 2024 at 09:37, Tom Rini trini@konsulko.com wrote:
On Thu, Dec 19, 2024 at 08:07:22AM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 16:54, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 04:42:30PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 12:53, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 17, 2024 at 12:45:29PM -0700, Simon Glass wrote:
Hi Tom,
On Tue, 17 Dec 2024 at 07:20, Tom Rini trini@konsulko.com wrote: > > On Tue, Dec 17, 2024 at 06:26:16AM -0700, Simon Glass wrote: > > > At present buildman only supports building the default target. Generally > > this is what is wanted, but in some cases boards erroneously have a > > different target for product extra files. > > > > Add a --target option to help. Also add a comment indicating which > > letters are free for new options. > > > > Signed-off-by: Simon Glass sjg@chromium.org > > How do you use this? It would be nice to be able to do "--target > u-boot-initial-env" for example for converting and verifying the > conversion of boards to plain text environment. Except the list of > artifacts that are kept isn't updated with that string as well. And > FWIW, is this for a case not solved by CONFIG_BUILD_TARGET?
It's related to the problem where colibri-imx8 doesn't actually build an image with the default boot target.
I don't recall if there's a good reason for imx8 not setting CONFIG_BUILD_TARGET to something, but that sounds like how to solve this problem. "make all" should work, normally.
Even if it did, the build fails the first time. It just needs to be fixed/
Yes, there's both the problem of CONFIG_BUILD_TARGET not being set and then whatever issues there are with configuring access to the blobs especially when they exist and aren't faked via BINMAN_ALLOW_MISSING. IIRC Toradex does weekly builds of top of tree, via OpenEmbedded / Yocto so there's not some bitrot here to be fixed, just smoothing out integration.
To me, BUILD_TARGET is a pre-Binman construction and we should just use binman to make sure that the default target builds a bootable image.
Well, no. Assuming that doc/board/toradex/colibri-imx8x.rst is correct then CONFIG_BUILD_TARGET should be set to "u-boot-dtb.imx" so that "make" just works like other platforms. If there's some other target actually in use, that should be what's put in CONFIG_BUILD_TARGETS and in turn binman will get invoked as needed.
I suspect the real issue here is that it has not been converted to binman yet.
But adding BUILD_TARGET just causes the build to fail...I already reported that to Francesco.
Regards, Simon
On Tue, Dec 17, 2024 at 05:54:11PM -0600, Tom Rini wrote:
IIRC Toradex does weekly builds of top of tree, via OpenEmbedded / Yocto so there's not some bitrot here to be fixed, just smoothing out integration.
Correct, we do weekly builds from OE integrating the u-boot master tip and run those on all of our modules.
We catch U-Boot regressions that prevents either the build or booting our Linux OS (not much more than that, at the moment).
This https://lore.kernel.org/all/20241216114231.qpfwug3zfqkxn3d5@joaog-nb.corp.to... is the last one we reported and helped debugging, and thanks to the help of various people we have a fix patch pending https://lore.kernel.org/all/20241218090007.692815-1-ilias.apalodimas@linaro.....
Francesco
participants (3)
-
Francesco Dolcini -
Simon Glass -
Tom Rini