[U-Boot] RFC: auto-generate ARM mach-types.h file from ARM machine database
Hi,
to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down mach types", I tried a short patch that demonstrates how to automatically generate the mach-types.h file from a database dump (from http://www.arm.linux.org.uk/developer/machines/?action=new).
This has multiple advantages: - pulling in new machine types is easier (drop in a new downloaded database dump), and produces a much smaller diff. - adding new machines is decoupled from the time they appear in Linux. - boards that are not in mainline Linux will not be break due to removal of their mach types from the Linux headers.
The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. I think the AWK script is simple enough that it will not require big maintenance efforts (unless the machine database format changes).
The patch is edited down - I removed the diff for the deletion of the old mach-types.h file, and shortened the new mach-types file to a few entries just to show the concept - otherwise, the patch would be much too big for the list.
Is this an acceptable solution? Should I go on and produce a full-fledged patch?
cu Michael
From 2cb8bfd1b387a4a49d9e0cebd96824c879000420 Mon Sep 17 00:00:00 2001
From: Michael Schwingen michael@schwingen.org Date: Thu, 5 May 2011 23:04:00 +0200 Subject: [ARM: auto-generate mach-types.h 1/1] auto-generate mach-types.h include file from ARM machine database dump
Signed-off-by: Michael Schwingen michael@schwingen.org based directly on Makefile/script from Linux-2.8.38.3
--- Makefile | 9 +- arch/arm/include/asm/mach-types.h |42924 ------------------------------------- arch/arm/tools/gen-mach-types | 72 + arch/arm/tools/mach-types | 3448 +++ 4 files changed, 3527 insertions(+), 42926 deletions(-) delete mode 100644 arch/arm/include/asm/mach-types.h create mode 100644 arch/arm/tools/gen-mach-types create mode 100644 arch/arm/tools/mach-types
diff --git a/Makefile b/Makefile index 384a59e..07ab7fb 100644 --- a/Makefile +++ b/Makefile @@ -469,7 +469,7 @@ $(obj)System.map: $(obj)u-boot # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep. # the dep file is only include in this top level makefile to determine when # to regenerate the autoconf.mk file. -$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h +$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h @$(XECHO) Generating $@ ; \ set -e ; \ : Generate the dependancies ; \ @@ -530,13 +530,18 @@ unconfig: $(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \ $(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
-%_config:: unconfig +%_config:: unconfig $(obj)include/asm/mach-types.h @$(MKCONFIG) -A $(@:_config=)
sinclude $(obj).boards.depend $(obj).boards.depend: boards.cfg awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
+ +$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types + @mkdir -p $(obj)include/asm + awk -f $^ > $@ || { rm -f $@; /bin/false; } + # # Functions to generate common board directory names # diff --git a/arch/arm/include/asm/mach-types.h b/arch/arm/include/asm/mach-types.h deleted file mode 100644
diff --git a/arch/arm/tools/gen-mach-types b/arch/arm/tools/gen-mach-types new file mode 100644 index 0000000..04fef71 --- /dev/null +++ b/arch/arm/tools/gen-mach-types @@ -0,0 +1,72 @@ +#!/bin/awk +# +# Awk script to generate include/generated/mach-types.h +# +BEGIN { nr = 0 } +/^#/ { next } +/^[ ]*$/ { next } + +NF == 4 { + machine_is[nr] = "machine_is_"$1; + config[nr] = "CONFIG_"$2; + mach_type[nr] = "MACH_TYPE_"$3; + num[nr] = $4; nr++ + } + +NF == 3 { + machine_is[nr] = "machine_is_"$1; + config[nr] = "CONFIG_"$2; + mach_type[nr] = "MACH_TYPE_"$3; + num[nr] = ""; nr++ + } + + +END { + printf("/*\n"); + printf(" * This was automagically generated from %s!\n", FILENAME); + printf(" * Do NOT edit\n"); + printf(" */\n\n"); + printf("#ifndef __ASM_ARM_MACH_TYPE_H\n"); + printf("#define __ASM_ARM_MACH_TYPE_H\n\n"); + printf("#ifndef __ASSEMBLY__\n"); + printf("/* The type of machine we're running on */\n"); + printf("extern unsigned int __machine_arch_type;\n"); + printf("#endif\n\n"); + + printf("/* see arch/arm/kernel/arch.c for a description of these */\n"); + for (i = 0; i < nr; i++) + if (num[i] ~ /..*/) + printf("#define %-30s %d\n", mach_type[i], num[i]); + + printf("\n"); + + for (i = 0; i < nr; i++) + if (num[i] ~ /..*/) { + printf("#ifdef %s\n", config[i]); + printf("# ifdef machine_arch_type\n"); + printf("# undef machine_arch_type\n"); + printf("# define machine_arch_type\t__machine_arch_type\n"); + printf("# else\n"); + printf("# define machine_arch_type\t%s\n", mach_type[i]); + printf("# endif\n"); + printf("# define %s()\t(machine_arch_type == %s)\n", machine_is[i], mach_type[i]); + printf("#else\n"); + printf("# define %s()\t(0)\n", machine_is[i]); + printf("#endif\n\n"); + } + + printf("/*\n * These have not yet been registered\n */\n"); + for (i = 0; i < nr; i++) + if (num[i] !~ /..*/) + printf("/* #define %-30s <<not registered>> */\n", mach_type[i]); + + for (i = 0; i < nr; i++) + if (num[i] !~ /..*/) { + printf("#define %s()\t(0)\n", machine_is[i]); + } + + printf("\n#ifndef machine_arch_type\n"); + printf("#define machine_arch_type\t__machine_arch_type\n"); + printf("#endif\n\n"); + printf("#endif\n"); + } diff --git a/arch/arm/tools/mach-types b/arch/arm/tools/mach-types new file mode 100644 index 0000000..957e4c9 --- /dev/null +++ b/arch/arm/tools/mach-types @@ -0,1 +1,38 @@ +# Database of machine macros and numbers +# +# This file is linux/arch/arm/tools/mach-types +# +# Up to date versions of this file can be obtained from: +# +# http://www.arm.linux.org.uk/developer/machines/download.php +# +# Please do not send patches to this file; it is automatically generated! +# To add an entry into this database, please see Documentation/arm/README, +# or visit: +# +# http://www.arm.linux.org.uk/developer/machines/?action=new +# +# Last update: Thu May 5 21:54:46 2011 +# +# machine_is_xxx CONFIG_xxxx MACH_TYPE_xxx number +# +ebsa110 ARCH_EBSA110 EBSA110 0 +riscpc ARCH_RPC RISCPC 1 +nexuspci ARCH_NEXUSPCI NEXUSPCI 3 +ebsa285 ARCH_EBSA285 EBSA285 4 +netwinder ARCH_NETWINDER NETWINDER 5 +cats ARCH_CATS CATS 6 +tbox ARCH_TBOX TBOX 7 +co285 ARCH_CO285 CO285 8 +clps7110 ARCH_CLPS7110 CLPS7110 9 +archimedes ARCH_ARC ARCHIMEDES 10 +a5k ARCH_A5K A5K 11 +etoile ARCH_ETOILE ETOILE 12 +lacie_nas ARCH_LACIE_NAS LACIE_NAS 13 +clps7500 ARCH_CLPS7500 CLPS7500 14 +shark ARCH_SHARK SHARK 15 +brutus SA1100_BRUTUS BRUTUS 16 +personal_server ARCH_PERSONAL_SERVER PERSONAL_SERVER 17 +itsy SA1100_ITSY ITSY 18 +l7200 ARCH_L7200 L7200 19 +pleb SA1100_PLEB PLEB 20
On Thu, May 5, 2011 at 17:48, Michael Schwingen wrote:
--- a/Makefile +++ b/Makefile @@ -469,7 +469,7 @@ $(obj)System.map: $(obj)u-boot # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep. # the dep file is only include in this top level makefile to determine when # to regenerate the autoconf.mk file. -$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h +$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h @$(XECHO) Generating $@ ; \ set -e ; \ : Generate the dependancies ; \ @@ -530,13 +530,18 @@ unconfig: $(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \ $(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
-%_config:: unconfig +%_config:: unconfig $(obj)include/asm/mach-types.h @$(MKCONFIG) -A $(@:_config=)
sinclude $(obj).boards.depend $(obj).boards.depend: boards.cfg awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
+$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types
- @mkdir -p $(obj)include/asm
- awk -f $^ > $@ || { rm -f $@; /bin/false; }
# # Functions to generate common board directory names #
this all belongs in arch/arm/config.mk and not the toplevel makefile
also, dont hardcode full paths to things. there's no reason for it.
might want to add an "update-mach-types" target so people can type `make update-mach-types` and it'll automatically wget the right file to the right place ... -mike
Am 05/06/2011 07:09 AM, schrieb Mike Frysinger:
On Thu, May 5, 2011 at 17:48, Michael Schwingen wrote:
--- a/Makefile +++ b/Makefile @@ -469,7 +469,7 @@ $(obj)System.map: $(obj)u-boot # This target actually generates 2 files; autoconf.mk and autoconf.mk.dep. # the dep file is only include in this top level makefile to determine when # to regenerate the autoconf.mk file. -$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h +$(obj)include/autoconf.mk.dep: $(obj)include/config.h include/common.h $(obj)include/asm/mach-types.h @$(XECHO) Generating $@ ; \ set -e ; \ : Generate the dependancies ; \ @@ -530,13 +530,18 @@ unconfig: $(obj)board/*/config.tmp $(obj)board/*/*/config.tmp \ $(obj)include/autoconf.mk $(obj)include/autoconf.mk.dep
-%_config:: unconfig +%_config:: unconfig $(obj)include/asm/mach-types.h @$(MKCONFIG) -A $(@:_config=)
sinclude $(obj).boards.depend $(obj).boards.depend: boards.cfg awk '(NF && $$1 !~ /^#/) { print $$1 ": " $$1 "_config; $$(MAKE)" }' $< > $@
+$(obj)include/asm/mach-types.h: arch/arm/tools/gen-mach-types arch/arm/tools/mach-types
@mkdir -p $(obj)include/asmawk -f $^ > $@ || { rm -f $@; /bin/false; }# # Functions to generate common board directory names #
this all belongs in arch/arm/config.mk and not the toplevel makefile
Fine with me, however, I am hitting a big problem with the Makefile structure:
mach-types.h needs to be built before autoconf.mk can be generated, and the rules for autoconf.mk are in the top-level Makefile.
If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.
It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.
rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).
Any ideas? Using the current Makefile structure, I see no other solution than defining the mach-types.h generation rules in the toplevel Makefile.
also, dont hardcode full paths to things. there's no reason for it.
Which of these can be omitted?
When putting the rules in arch/arm/config.mk, I can make them trigger only when compiling for ARM, and I can use $(ARCH) instead of arm/, but apart from that, I think I need to specify the paths, no?
might want to add an "update-mach-types" target so people can type `make update-mach-types` and it'll automatically wget the right file to the right place ...
Good idea. I agree with Wolfgang that this is intended to be used by the maintainer mainly, so the mach-types file should be included in the source so that a normal user does not need to download the file.
cu Michael
Dear Michael Schwingen,
In message 4DC5259C.7040208@discworld.dascon.de you wrote:
If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.
You are not suppoed to put any make rules in config.mk files.
It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.
As the name implies, these files contain configuration (= variable) settings. Nothing else.
rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).
But lower level directories inherit all settings from the top level Makefile?
Best regards,
Wolfgang Denk
Am 05/09/2011 07:37 AM, schrieb Wolfgang Denk:
Dear Michael Schwingen,
In message 4DC5259C.7040208@discworld.dascon.de you wrote:
If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.
You are not suppoed to put any make rules in config.mk files.
It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.
As the name implies, these files contain configuration (= variable) settings. Nothing else.
rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).
But lower level directories inherit all settings from the top level Makefile?
Yes. However, the rule to generate autoconf.mk is in the toplevel Makefile, and I need the rule to generate mach-types.h at the same level, as autoconf.mk depends on it. The lower-level *Makefiles* are executed too late to generate anything that is needed for autoconf.mk.
cu Michael
On Saturday, May 07, 2011 06:57:32 Michael Schwingen wrote:
mach-types.h needs to be built before autoconf.mk can be generated, and the rules for autoconf.mk are in the top-level Makefile.
If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.
It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.
rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).
Any ideas? Using the current Makefile structure, I see no other solution than defining the mach-types.h generation rules in the toplevel Makefile.
do you need the mach-types file for anything else ? if not, dont keep that in git, keep the generated header.
also, dont hardcode full paths to things. there's no reason for it.
Which of these can be omitted?
look for "/bin/" or "/sbin/". if your patch contains those, it is wrong.
might want to add an "update-mach-types" target so people can type `make update-mach-types` and it'll automatically wget the right file to the right place ...
Good idea. I agree with Wolfgang that this is intended to be used by the maintainer mainly, so the mach-types file should be included in the source so that a normal user does not need to download the file.
i didnt suggest this in the first place -mike
Mike Frysinger wrote:
On Saturday, May 07, 2011 06:57:32 Michael Schwingen wrote:
mach-types.h needs to be built before autoconf.mk can be generated, and the rules for autoconf.mk are in the top-level Makefile.
If I put rules in arch/arm/config.mk, then the first of these rules becomes the default rule which is executed in subdir makes (like "make -C arch"), which breaks compilation completely, since config.mk is included before the rules are defined in the subdir Makefiles.
It seems the current scheme allows only variable definitions in config.mk files, which is not sufficient here.
rules.mk would be fine, however, there is no provision to include rules from lower directories, since all the building in subdirectories is handled by recursively calling make (this is one of the problems that arise by recursively calling make for each directory, but that is a different topic).
Any ideas? Using the current Makefile structure, I see no other solution than defining the mach-types.h generation rules in the toplevel Makefile.
do you need the mach-types file for anything else ? if not, dont keep that in git, keep the generated header.
Then we are back at basically the current state. If I do this, we need no Makefile support at all - we just need a script that is run by the maintainer that downloads the current mach-types from the web, and generates mach-types.h, which is then checked in.
cu Michael
On Monday, May 09, 2011 11:54:15 Michael Schwingen wrote:
Then we are back at basically the current state. If I do this, we need no Makefile support at all - we just need a script that is run by the maintainer that downloads the current mach-types from the web, and generates mach-types.h, which is then checked in.
i'm ok with that, but i'm not an arm maintainer ;) -mike
Hi all,
Le 09/05/2011 18:16, Mike Frysinger a écrit :
On Monday, May 09, 2011 11:54:15 Michael Schwingen wrote:
Then we are back at basically the current state. If I do this, we need no Makefile support at all - we just need a script that is run by the maintainer that downloads the current mach-types from the web, and generates mach-types.h, which is then checked in.
i'm ok with that, but i'm not an arm maintainer ;) -mike
I'm the ARM maintainer -- custodian, actually -- and I approve this message. :)
Amicalement,
Dear Michael Schwingen,
In message 20110505214836.GA5313@discworld.dascon.de you wrote:
to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down mach types", I tried a short patch that demonstrates how to automatically generate the mach-types.h file from a database dump (from http://www.arm.linux.org.uk/developer/machines/?action=new).
This has multiple advantages:
- pulling in new machine types is easier (drop in a new downloaded database dump), and produces a much smaller diff.
- adding new machines is decoupled from the time they appear in Linux.
- boards that are not in mainline Linux will not be break due to removal of their mach types from the Linux headers.
The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. I think the AWK script is simple enough that it will not require big maintenance efforts (unless the machine database format changes).
The patch is edited down - I removed the diff for the deletion of the old mach-types.h file, and shortened the new mach-types file to a few entries just to show the concept - otherwise, the patch would be much too big for the list.
Is this an acceptable solution? Should I go on and produce a full-fledged patch?
I agree with the comments alreay made by Mike Frysinger.
In addition, I would like to point out that I consider it mandatory thatthe normal build process does NOT involve any network download. Instead, it must be run completely based on files checked into the git repository only.
That means, that the "update-mach-types" target Mike mentions is something that the ARM maintainer can use to update the in-tree copy of the mach-types.h file, which then gets commited into the git repository. Also users can run this if they want to use the latest version, but it MUST NOT be used for a regular build.
Best regards,
Wolfgang Denk
Hi Wolfgang,
Le 06/05/2011 12:14, Wolfgang Denk a écrit :
Dear Michael Schwingen,
In message20110505214836.GA5313@discworld.dascon.de you wrote:
to conclude the discussion in the thread "Re: [U-Boot] Update and Cut down mach types", I tried a short patch that demonstrates how to automatically generate the mach-types.h file from a database dump (from http://www.arm.linux.org.uk/developer/machines/?action=new).
This has multiple advantages:
- pulling in new machine types is easier (drop in a new downloaded database dump), and produces a much smaller diff.
- adding new machines is decoupled from the time they appear in Linux.
- boards that are not in mainline Linux will not be break due to removal of their mach types from the Linux headers.
The AWK and Makefile fragment script is taken verbatim from Linux 2.6.38.3. I think the AWK script is simple enough that it will not require big maintenance efforts (unless the machine database format changes).
The patch is edited down - I removed the diff for the deletion of the old mach-types.h file, and shortened the new mach-types file to a few entries just to show the concept - otherwise, the patch would be much too big for the list.
Is this an acceptable solution? Should I go on and produce a full-fledged patch?
I agree with the comments alreay made by Mike Frysinger.
In addition, I would like to point out that I consider it mandatory thatthe normal build process does NOT involve any network download. Instead, it must be run completely based on files checked into the git repository only.
That means, that the "update-mach-types" target Mike mentions is something that the ARM maintainer can use to update the in-tree copy of the mach-types.h file, which then gets commited into the git repository. Also users can run this if they want to use the latest version, but it MUST NOT be used for a regular build.
Agreed on all points.
Best regards,
Wolfgang Denk
Amicalement,
participants (4)
-
Albert ARIBAUD -
Michael Schwingen -
Mike Frysinger -
Wolfgang Denk