Hi,
On 12 February 2014 03:42, Masahiro Yamada yamada.m@jp.panasonic.com wrote:
Hi Albert,
It might also be worth looking at tools/buildman, which automatically allocates one build thread per CPU.
Jumping in late, but my question is incidental and not urgent anyway.
Would using buildman make the multiple build / multiple CPU code in MAKEALL useless? I'm wondering whether we could apply the Unix philosophy here (1), let buildman alone deal with handling parallel builds, and remove code from MAKEALL.
This is a good summary thank you, but I have a few comments as well.
I think parallel build feature is missing from buildman. So, buildman cannot check if -j option is working correctly.
It is not used by default, but there is a -j option. If you use -j, first make sure you reduce the number of threads to compensate. So for example you can use:
buildman -j 8 -T 1
to use -j8 with only a single thread on a 8-core machine. I typically find that building with -j1 and increasing the thread count is much faster. Another tip if you are not using your machine for anything else is to use -T with a number 20-30% larger than the number of CPUs you have.
Note: Please do not be confused by the difference of what "parallel" means.
[1] MAKEALL runs single "make" thread by default. (You can change this with BUILD_NBUILDS variable) One "make" thread runs multiple jobs by giving -j option. (You can change this with BUILD_NCPUS variable)
[2] buildman runs multiple "make" threads. Each make thread runs one job (that is, always -j 1 ).
Indeed, this is the default behaviour, but it can be changed with options.
Besides parallel build , I notice some differences between MAKEALL and buildman.
- MAKEALL runs "make mrproper" everty time before "make", but buildman doesn't. This means objects are remaining, that were generated by the previous commit. So, even if some build rules in makefiles get broken at an intermediate commit, buildman possibly cannot detect the error.
That is correct. To get the MAKEALL behaviour you need to give the -f option.
- MAKEALL can select in-tree-build or out-of-tree build by BUILD_DIR option, but buildman always does out-of-tree build.
That's right, in fact buildman does not support in-tree build at all unfortunately. However, given that every thread has its own git tree, it would actually be fairly easy to add this feature.
(Simon, please correct me if I am wrong.)
If you touch only C sources, buildman is enough (and faster).
But if you change make targets, it is highly recommended to check with MAKEALL. At least, I need both. (for Kbuild and Kconfig work)
I guess we cannot replace MAKEALL with buildman for now.
The intent was that buildman supported everything that MAKEALL does and more. I believe that is true except for the in-tree build you mention above. However, I think there might also be some small difference between the way the -f option is implemented in buildman, and the way MAKEALL works. It might be worth splitting the -f option into two:
- one to force rebuild of a board - another to force a reconfigure on every build
Then we could be sure. In summary, if we do that and support in-tree builds then I think buildman has all the features of MAKEALL.
BTW a few other options that buildman has, which people may not have noticed:
-S - shows increase/decrease of image sizes across commits -B - same but for individual functions -k - keep the build outputs -d - show detailed board information -S - allows building every second commit, or only first and last, etc.
Regards, Simon