On Nov 20, 2011, at 12:11 AM, Simon Glass wrote:
Hi Kumar,
On Tue, Nov 8, 2011 at 5:48 AM, Kumar Gala galak@kernel.crashing.org wrote:
On Nov 3, 2011, at 11:22 AM, Simon Glass wrote:
Hi Andy,
On Thu, Nov 3, 2011 at 12:28 AM, Andy Fleming afleming@freescale.com wrote:
The MAKEALL script cleverly runs make with the appropriate options to use all of the cores on the system, but your average U-Boot build can't make much use of more than a few cores. If you happen to have a many-core server, your builds will leave most of the system idle.
In order to make full use of such a system, we need to build multiple targets in parallel, and this requires directing make output into multiple directories. We add a BUILD_NBUILDS variable, which allows users to specify how many builds to run in parallel. I've found that 16 is too many on my system (fork starts to fail). When BUILD_NBUILDS is set greater than 1, we redefine BUILD_DIR for each build to be ${BUILD_DIR}/${target}. Also, we make "./build" the default BUILD_DIR when BUILD_NBUILDS is greater than 1.
Once each build finishes, we run make clean on its directory, to reduce the footprint, and also remove all .depend files.
As a result, we are left with a build directory with all of the built targets still there for use, which means anyone who wanted to use MAKEALL as part of a test harness can now do so.
Signed-off-by: Andy Fleming afleming@freescale.com
Inspired by all the MAKEALL improvements, I decided to clean up my old one for parallel builds. I think this version addresses the concerns raised last time...
I see an issue if I have a build in working tree already. I think we might need a distclean before starting
I also it seems this does things in clumps of BUILD_NBUILDS. What I mean is, if BUILD_NBUILDS is 4, you do 4 and wait for all 4 to finish before starting the 5th job once the 1st is complete. Anything we can do about that to always try and have 4 builds going until the end?
- k
What is the issue you see? Given that it uses an empty output dir for each build, I can't see what could be wrong.
I think if I have an existing build in my source tree it screws up between that and the O= build.
I am seeing a strange issue which is probably my fault. In a whole MAKEALL maybe one build (always mx31 so far) fails with errors like this:
$ more logs/mx31pdk.ERR mv: cannot stat `/c/cosarm/buildall/u-boot/build/mx31pdk/include/autoconf.mk.tmp ': No such file or directory /c/cosarm/buildall/u-boot/include/timestamp.h:27: fatal error: generated/timesta mp_autogenerated.h: No such file or directory compilation terminated. make[1]: *** No rule to make target `/c/cosarm/buildall/u-boot/build/mx31pdk/arc h/arm/cpu/arm1136/.depend', needed by `_depend'. Stop. In file included from /c/cosarm/buildall/u-boot/include/version.h:27:0, from mkimage.c:26: /c/cosarm/buildall/u-boot/include/timestamp.h:27:47: fatal error: generated/time stamp_autogenerated.h: No such file or directory compilation terminated.
/c/cosarm/buildall/u-boot/include/timestamp.h:27: fatal error: generated/timesta mp_autogenerated.h: No such file or directory compilation terminated. make[1]: *** No rule to make target `/c/cosarm/buildall/u-boot/build/imx31_phyco re/arch/arm/cpu/arm1136/.depend', needed by `_depend'. Stop. In file included from /c/cosarm/buildall/u-boot/include/version.h:27:0, from mkimage.c:26: /c/cosarm/buildall/u-boot/include/timestamp.h:27:47: fatal error: generated/time stamp_autogenerated.h: No such file or directory compilation terminated. make[1]: *** [/c/cosarm/buildall/u-boot/build/imx31_phycore/tools/mkimage.o] Err or 1 make: *** [tools] Error 2
I really can't see that Andy's change can affect this, but perhaps it provokes an existing bug by changing the timing. I see the issue when building 16 U-Boots at a time.
I haven't looked into it in detail - maybe there is a dependency missing in the Makefile. But I would have thought that this line would ensure that these files exist before anything else is done:
depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) \ ...
Regards, Simon