Hi Simon,

On Wed, 8 Jan 2014 09:54:47 -0700, Simon Glass sjg@chromium.org wrote:

Hi York,

On 4 January 2014 02:21, Wolfgang Denk wd@denx.de wrote:

...

Dear York,

In message 52C7424A.4090205@freescale.com you wrote:

...

I have some troubles to run MAKEALL with BUILD_NBUILDS. If I set

BUILD_NBUILDS

...

to 2 or greater, there is a good chance the total number of targets is

not an

...

integral multiple of BUILD_NBUILDS. It has two undesired results.

1. The status report has wrong number of passed builds.
2. This script throws out SIGTERM.

The second one is troubling me. I am using Jenkins to monitor and build automatically. I can trap the SIGTERM on some hosts but not all of them.

Can you shed some light on this?

I'm sorry, but I am not familiar with this BUILD_NBUILDS code at all. It was added by Andy Fleming, so maybe he can help.

Andy?

It might also be worth looking at tools/buildman, which automatically allocates one build thread per CPU.

Jumping in late, but my question is incidental and not urgent anyway.

Would using buildman make the multiple build / multiple CPU code in MAKEALL useless? I'm wondering whether we could apply the Unix philosophy here (1), let buildman alone deal with handling parallel builds, and remove code from MAKEALL.

(1) not the "unix is user-friendly, it's just picky about who its friends are" one; the "do one thing well" one.

Regards, Simon

Amicalement,

Hi,

On 12 February 2014 03:42, Masahiro Yamada yamada.m@jp.panasonic.com wrote:

Hi Albert,

...

...

It might also be worth looking at tools/buildman, which automatically allocates one build thread per CPU.

Jumping in late, but my question is incidental and not urgent anyway.

Would using buildman make the multiple build / multiple CPU code in MAKEALL useless? I'm wondering whether we could apply the Unix philosophy here (1), let buildman alone deal with handling parallel builds, and remove code from MAKEALL.

This is a good summary thank you, but I have a few comments as well.

I think parallel build feature is missing from buildman. So, buildman cannot check if -j option is working correctly.

It is not used by default, but there is a -j option. If you use -j, first make sure you reduce the number of threads to compensate. So for example you can use:

buildman -j 8 -T 1

to use -j8 with only a single thread on a 8-core machine. I typically find that building with -j1 and increasing the thread count is much faster. Another tip if you are not using your machine for anything else is to use -T with a number 20-30% larger than the number of CPUs you have.

Note: Please do not be confused by the difference of what "parallel" means.

[1] MAKEALL runs single "make" thread by default. (You can change this with BUILD_NBUILDS variable) One "make" thread runs multiple jobs by giving -j option. (You can change this with BUILD_NCPUS variable)

[2] buildman runs multiple "make" threads. Each make thread runs one job (that is, always -j 1 ).

Indeed, this is the default behaviour, but it can be changed with options.

Besides parallel build , I notice some differences between MAKEALL and buildman.

• MAKEALL runs "make mrproper" everty time before "make", but buildman doesn't. This means objects are remaining, that were generated by the previous commit. So, even if some build rules in makefiles get broken at an intermediate commit, buildman possibly cannot detect the error.

That is correct. To get the MAKEALL behaviour you need to give the -f option.

• MAKEALL can select in-tree-build or out-of-tree build by BUILD_DIR option, but buildman always does out-of-tree build.

That's right, in fact buildman does not support in-tree build at all unfortunately. However, given that every thread has its own git tree, it would actually be fairly easy to add this feature.

(Simon, please correct me if I am wrong.)

If you touch only C sources, buildman is enough (and faster).

But if you change make targets, it is highly recommended to check with MAKEALL. At least, I need both. (for Kbuild and Kconfig work)

I guess we cannot replace MAKEALL with buildman for now.

The intent was that buildman supported everything that MAKEALL does and more. I believe that is true except for the in-tree build you mention above. However, I think there might also be some small difference between the way the -f option is implemented in buildman, and the way MAKEALL works. It might be worth splitting the -f option into two:

- one to force rebuild of a board - another to force a reconfigure on every build

Then we could be sure. In summary, if we do that and support in-tree builds then I think buildman has all the features of MAKEALL.

BTW a few other options that buildman has, which people may not have noticed:

-S - shows increase/decrease of image sizes across commits -B - same but for individual functions -k - keep the build outputs -d - show detailed board information -S - allows building every second commit, or only first and last, etc.

Regards, Simon