[U-Boot] [PATCH] Makefile: fix parallel build
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top Makefile.
Signed-off-by: Daniel Hobi daniel.hobi@schmid-telecom.ch --- Makefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/Makefile b/Makefile index 19b5ac0..2fd22c7 100644 --- a/Makefile +++ b/Makefile @@ -400,7 +400,7 @@ env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1
depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU); do $(MAKE) -C $$dir _depend ; done
TAG_SUBDIRS += include TAG_SUBDIRS += lib_generic board/$(BOARDDIR)
On Thursday 10 December 2009 08:41:07 Daniel Hobi wrote:
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top Makefile.
seems to happen in lib_$(ARCH)/ too, but in reviewing my logs from the last few months, most parallel .depend failures have indeed been in cpu/$(CPU)/. maybe this is just coincidence though ... perhaps the depend target should walk all subdirs instead of a just random few ones (use LIBS). -mike
On 11.12.2009 20:25, Mike Frysinger wrote:
On Thursday 10 December 2009 08:41:07 Daniel Hobi wrote:
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top Makefile.
seems to happen in lib_$(ARCH)/ too, but in reviewing my logs from the last few months, most parallel .depend failures have indeed been in cpu/$(CPU)/. maybe this is just coincidence though ... perhaps the depend target should walk all subdirs instead of a just random few ones (use LIBS).
The problem with cpu/$(CPU)/ is that the top Makefile builds multiple targets within this directory in parallel - at least start.o and lib$(CPU).a. For all other directories, parallel build should work, since there is only one target per directory.
Can you provide any commit ID where building lib_$(ARCH)/ failed?
Thus, adding all LIBS to the depend target should not make any difference. In order to properly track such dependencies we should switch to non-recursive Makefiles (as seen in Linux) since Recursive Make [is] Considered Harmful[1].
[1] http://miller.emu.id.au/pmiller/books/rmch/
Best regards, Daniel
On Tuesday 15 December 2009 04:21:02 Daniel Hobi wrote:
On 11.12.2009 20:25, Mike Frysinger wrote:
On Thursday 10 December 2009 08:41:07 Daniel Hobi wrote:
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU). If cpu/$(CPU)/.depend is not present, the sub-makes may end up generating this file simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of cpu/$(CPU)/.depend by adding cpu/$(CPU) to the depend target in the top Makefile.
seems to happen in lib_$(ARCH)/ too, but in reviewing my logs from the last few months, most parallel .depend failures have indeed been in cpu/$(CPU)/. maybe this is just coincidence though ... perhaps the depend target should walk all subdirs instead of a just random few ones (use LIBS).
The problem with cpu/$(CPU)/ is that the top Makefile builds multiple targets within this directory in parallel - at least start.o and lib$(CPU).a. For all other directories, parallel build should work, since there is only one target per directory.
OK, that answers the next part
Can you provide any commit ID where building lib_$(ARCH)/ failed?
it isnt specific to lib_$(ARCH). wherever the LDSCRiPT is located will cause a problem. in the Blackfin case, it's in lib_blackfin/.
so maybe to extend this workaround, add like $(dir $(LDSCRIPT)) to the list. -mike
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of .depend in cpu/$(CPU) and $(dir $(LDSCRIPT)) by adding these directories to the depend target in the top Makefile.
Other directories in $(LIBS) are not affected since they contain only one Make target and thus only one sub-make is spawned per directory.
Signed-off-by: Daniel Hobi daniel.hobi@schmid-telecom.ch --- v2: - Break overlong line - Add comment about other directories to commit message - Also build target _depend in $(dir $(LDSCRIPT)) (as suggested by Mike) This requires the Makefile in $(dir $(LDSCRIPT)) to provide a _depend target, which is the case for all LDSCRIPT values in the current tree
diff --git a/Makefile b/Makefile index 19b5ac0..e9be7a5 100644 --- a/Makefile +++ b/Makefile @@ -400,7 +400,8 @@ env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1
depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \ + $(MAKE) -C $$dir _depend ; done
TAG_SUBDIRS += include TAG_SUBDIRS += lib_generic board/$(BOARDDIR)
On Tuesday 15 December 2009 08:51:46 Daniel Hobi wrote:
depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk
for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done
for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \$(MAKE) -C $$dir _depend ; done
i'd stick a short comment here explaining the reason for the specific subdirs. otherwise, the new additions all look good to me. this is something that has annoyed me for quite a long time, but has been hard to reproduce to figure out what exactly was failing. thanks ! Signed-off-by: Mike Frysinger vapier@gentoo.org -mike
Dear Daniel Hobi,
In message 1260885106-3494-1-git-send-email-daniel.hobi@schmid-telecom.ch you wrote:
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
...
diff --git a/Makefile b/Makefile index 19b5ac0..e9be7a5 100644 --- a/Makefile +++ b/Makefile @@ -400,7 +400,8 @@ env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1
depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk
for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done
for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \$(MAKE) -C $$dir _depend ; done
I ageree with Mike: please add a comment here.
Best regards,
Wolfgang Denk
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of .depend in cpu/$(CPU) and $(dir $(LDSCRIPT)) by adding these directories to the depend target in the top Makefile.
Other directories in $(LIBS) are not affected since they contain only one Make target and thus only one sub-make is spawned per directory.
Signed-off-by: Daniel Hobi daniel.hobi@schmid-telecom.ch --- v3: - Add a comment why make _depend is required for these two subdirs v2: - Also build target depend in $(dir $(LDSCRIPT)) (suggested by Mike) - Break overlong line
Makefile | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/Makefile b/Makefile index 793fcec..1ab197b 100644 --- a/Makefile +++ b/Makefile @@ -398,8 +398,11 @@ updater: env: $(MAKE) -C tools/env all MTD_VERSION=${MTD_VERSION} || exit 1
+# Explicitly make _depend in subdirs containing multiple targets to prevent +# parallel sub-makes creating .depend files simultaneously. depend dep: $(TIMESTAMP_FILE) $(VERSION_FILE) $(obj)include/autoconf.mk - for dir in $(SUBDIRS) ; do $(MAKE) -C $$dir _depend ; done + for dir in $(SUBDIRS) cpu/$(CPU) $(dir $(LDSCRIPT)) ; do \ + $(MAKE) -C $$dir _depend ; done
TAG_SUBDIRS = $(SUBDIRS) TAG_SUBDIRS += $(dir $(__LIBS))
Dear Daniel Hobi,
In message 1263834819-15315-1-git-send-email-daniel.hobi@schmid-telecom.ch you wrote:
During parallel build, the top Makefile spawns multiple sub-makes for targets in cpu/$(CPU) and $(dir $(LDSCRIPT)). If the .depend files are not present in these directories, the sub-makes may end up generating these files simultaneously which leads to corrupted content.
A typical error message is:
.depend:39: *** multiple target patterns. Stop.
This patch serializes the creation of .depend in cpu/$(CPU) and $(dir $(LDSCRIPT)) by adding these directories to the depend target in the top Makefile.
Other directories in $(LIBS) are not affected since they contain only one Make target and thus only one sub-make is spawned per directory.
Signed-off-by: Daniel Hobi daniel.hobi@schmid-telecom.ch
v3:
- Add a comment why make _depend is required for these two subdirs
v2:
- Also build target depend in $(dir $(LDSCRIPT)) (suggested by Mike)
- Break overlong line
Makefile | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
Applied, thanks.
Best regards,
Wolfgang Denk
participants (3)
-
Daniel Hobi -
Mike Frysinger -
Wolfgang Denk