[U-Boot] [PATCH 0/3] buildman updates
Hi,
This patchset was created to update buildman, and fix a couple small issues that were discovered. kernel.org has a new set of toolchains (8.1.0), and I ran into some issues trying to download them.
I'm not a python guru, so if there are better ways to code these fixes, please let me know.
Best regards, Trevor
Trevor Woerner (3): buildman/toolchain.py: update versions buildman/toolchain.py: fix toolchain directory buildman/toolchain.py: handle inconsistent tarball names
tools/buildman/toolchain.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
On kernel.org, a newer 8.1.0 version of the toolchains is available. Also, update the kernel.org version list to include some of the older versions so more architectures can be supported.
Signed-off-by: Trevor Woerner trevor@toganlabs.com --- tools/buildman/toolchain.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 4b35f400e9..442a7f977a 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -430,7 +430,7 @@ class Toolchains: """ arch = command.OutputOneLine('uname', '-m') base = 'https://www.kernel.org/pub/tools/crosstool/files/bin' - versions = ['7.3.0', '6.4.0', '4.9.4'] + versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4', '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4'] links = [] for version in versions: url = '%s/%s/%s/' % (base, arch, version)
On Wed, 21 Nov 2018 at 01:32, Trevor Woerner trevor@toganlabs.com wrote:
On kernel.org, a newer 8.1.0 version of the toolchains is available. Also, update the kernel.org version list to include some of the older versions so more architectures can be supported.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Simon Glass sjg@chromium.org
Am 21.11.18 um 09:31 schrieb Trevor Woerner:
On kernel.org, a newer 8.1.0 version of the toolchains is available. Also, update the kernel.org version list to include some of the older versions so more architectures can be supported.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 4b35f400e9..442a7f977a 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -430,7 +430,7 @@ class Toolchains: """ arch = command.OutputOneLine('uname', '-m') base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
versions = ['7.3.0', '6.4.0', '4.9.4']
versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4', '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']
with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and unneeded versions. So it doesn't make sense to add them again.
Also did you a test run on Travis CI to verify that all archs still compile where a 8.1.0 toolchain is available? Travis CI will automatically use the latest avaiable toolchain for each arch. Possible new warnings or errors need to be fixed before doing the switch to 8.1.0.
links = [] for version in versions: url = '%s/%s/%s/' % (base, arch, version)
On Thu, Nov 29, 2018 at 02:35:56AM +0100, Daniel Schwierzeck wrote:
Am 21.11.18 um 09:31 schrieb Trevor Woerner:
On kernel.org, a newer 8.1.0 version of the toolchains is available. Also, update the kernel.org version list to include some of the older versions so more architectures can be supported.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 4b35f400e9..442a7f977a 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -430,7 +430,7 @@ class Toolchains: """ arch = command.OutputOneLine('uname', '-m') base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
versions = ['7.3.0', '6.4.0', '4.9.4']
versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4', '4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and unneeded versions. So it doesn't make sense to add them again.
Also did you a test run on Travis CI to verify that all archs still compile where a 8.1.0 toolchain is available? Travis CI will automatically use the latest avaiable toolchain for each arch. Possible new warnings or errors need to be fixed before doing the switch to 8.1.0.
Agreed. Are you using buildman outside of U-Boot where you have a usecase for adding in old versions / other architectures? I do want to move up to 8.1.0 but I'm sure we have warnings to fix before we can do that. I _think_ I even posted a link to what was failing a while ago and fixed a few things, but there's some harder to solve problems left.
On Thu, Nov 29, 2018 at 2:03 PM Tom Rini trini@konsulko.com wrote:
On Thu, Nov 29, 2018 at 02:35:56AM +0100, Daniel Schwierzeck wrote:
Am 21.11.18 um 09:31 schrieb Trevor Woerner:
On kernel.org, a newer 8.1.0 version of the toolchains is available.
Also,
update the kernel.org version list to include some of the older
versions so
more architectures can be supported.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 4b35f400e9..442a7f977a 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -430,7 +430,7 @@ class Toolchains: """ arch = command.OutputOneLine('uname', '-m') base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
versions = ['7.3.0', '6.4.0', '4.9.4']
versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4','4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']
with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and unneeded versions. So it doesn't make sense to add them again.
Also did you a test run on Travis CI to verify that all archs still compile where a 8.1.0 toolchain is available? Travis CI will automatically use the latest avaiable toolchain for each arch. Possible new warnings or errors need to be fixed before doing the switch to 8.1.0.
Agreed. Are you using buildman outside of U-Boot where you have a usecase for adding in old versions / other architectures?
Yes. I was using it as a general-purpose toolchain-downloading tool.
I do want to move up to 8.1.0 but I'm sure we have warnings to fix before we can do that. I _think_ I even posted a link to what was failing a while ago and fixed a few things, but there's some harder to solve problems left.
Okay, no problem.
In any case I'm partway through doing a test with and without my patches. I'll post my results for the sake of completeness.
I'm also interested in exploring whether it might be possible to have buildman download pre-built toolchains from bootlin. Maybe that would be of some interest?
On Fri, Nov 30, 2018 at 06:48:35AM -0500, Trevor Woerner wrote:
On Thu, Nov 29, 2018 at 2:03 PM Tom Rini trini@konsulko.com wrote:
On Thu, Nov 29, 2018 at 02:35:56AM +0100, Daniel Schwierzeck wrote:
Am 21.11.18 um 09:31 schrieb Trevor Woerner:
On kernel.org, a newer 8.1.0 version of the toolchains is available.
Also,
update the kernel.org version list to include some of the older
versions so
more architectures can be supported.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 4b35f400e9..442a7f977a 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -430,7 +430,7 @@ class Toolchains: """ arch = command.OutputOneLine('uname', '-m') base = 'https://www.kernel.org/pub/tools/crosstool/files/bin'
versions = ['7.3.0', '6.4.0', '4.9.4']
versions = ['8.1.0', '7.3.0', '6.4.0', '5.5.0', '4.9.4','4.9.0', '4.8.5', '4.8.0', '4.7.3', '4.6.3', '4.6.2', '4.6.1', '4.5.1', '4.2.4']
with 4c58d273e0f2cc21821c1de32494abd5de411d9b we removed the older and unneeded versions. So it doesn't make sense to add them again.
Also did you a test run on Travis CI to verify that all archs still compile where a 8.1.0 toolchain is available? Travis CI will automatically use the latest avaiable toolchain for each arch. Possible new warnings or errors need to be fixed before doing the switch to 8.1.0.
Agreed. Are you using buildman outside of U-Boot where you have a usecase for adding in old versions / other architectures?
Yes. I was using it as a general-purpose toolchain-downloading tool.
I do want to move up to 8.1.0 but I'm sure we have warnings to fix before we can do that. I _think_ I even posted a link to what was failing a while ago and fixed a few things, but there's some harder to solve problems left.
Okay, no problem.
In any case I'm partway through doing a test with and without my patches. I'll post my results for the sake of completeness.
I'm also interested in exploring whether it might be possible to have buildman download pre-built toolchains from bootlin. Maybe that would be of some interest?
We had thought about that for a bit, for a while, but Arnd is keeping the kernel.org ones more up to date than Bootlin was, when we had need of easy access to a specific new version. Things may have changed since then since it was 1-2 years ago, at this point. But yes, a patch to add an option to download from elsewhere (perhaps by base URL?) would be useful I think. I can see cases of pointing buildman at an internal mirror instead for example.
On Fri 2018-11-30 @ 06:48:35 AM, Trevor Woerner wrote:
In any case I'm partway through doing a test with and without my patches. I'll post my results for the sake of completeness.
Using a relatively recent pull, I have the "master" test, and the "twoerner" test which simply adds my patches on top of "master":
$ git show-branch --sha1-name master contrib/twoerner/buildman ! [master] Merge branch '2018-11-28-master-imports' * [contrib/twoerner/buildman] buildman/toolchain.py: handle inconsistent tarball names -- * [11028a5d2a] buildman/toolchain.py: handle inconsistent tarball names * [ab4e8c6d2c] buildman/toolchain.py: fix toolchain directory * [e8b8be4d32] buildman/toolchain.py: update versions -- [e16c888fab] Merge branch '2018-11-28-master-imports'
Cleaning out ~/.buildman and ~/.buildman-toolchains between each test, here are my results after fetching the toolchains:
$ ./tools/buildman/buildman --fetch-arch all
$ diff -u master/toolchains twoerner/toolchains --- master/toolchains 2018-11-30 20:32:33.593062110 -0500 +++ twoerner/toolchains 2018-11-30 20:33:20.025307799 -0500 @@ -1,14 +1,17 @@ -List of available toolchains (35): +List of available toolchains (41): aarch64 alpha am33_2.0 arc arm +avr32 bfin c6x cris +crisv32 frv h8300 +hexagon hppa hppa64 i386 @@ -18,14 +21,17 @@ microblaze mips mips64 +nds32le nios2 or1k +or32 powerpc powerpc64 ppc64le riscv32 riscv64 s390 +s390x sh2 sh4 sparc
including version information:
$ diff -u master/toolchains-versions twoerner/toolchains-versions --- master/toolchains-versions 2018-11-30 20:31:56.328864954 -0500 +++ twoerner/toolchains-versions 2018-11-30 20:31:51.088837231 -0500 @@ -1,36 +1,42 @@ -List of available toolchains (35): -aarch64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc -alpha : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/alpha-linux/bin/alpha-linux-gcc +List of available toolchains (41): +aarch64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/aarch64-linux/bin/aarch64-linux-gcc +alpha : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/alpha-linux/bin/alpha-linux-gcc am33_2.0 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/am33_2.0-linux/bin/am33_2.0-linux-gcc -arc : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/arc-elf/bin/arc-elf-gcc -arm : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-gcc +arc : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/arc-linux/bin/arc-linux-gcc +arm : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/arm-linux-gnueabi/bin/arm-linux-gnueabi-gcc +avr32 : /home/trevor/.buildman-toolchains/gcc-4.2.4-nolibc/avr32-linux/bin/avr32-linux-gcc bfin : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/bfin-uclinux/bin/bfin-uclinux-gcc -c6x : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/c6x-elf/bin/c6x-elf-gcc +c6x : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/c6x-elf/bin/c6x-elf-gcc cris : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/cris-linux/bin/cris-linux-gcc +crisv32 : /home/trevor/.buildman-toolchains/gcc-4.6.3-nolibc/crisv32-linux/bin/crisv32-linux-gcc frv : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/frv-linux/bin/frv-linux-gcc -h8300 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/h8300-linux/bin/h8300-linux-gcc -hppa : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/hppa-linux/bin/hppa-linux-gcc -hppa64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc -i386 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/i386-linux/bin/i386-linux-gcc -ia64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/ia64-linux/bin/ia64-linux-gcc +h8300 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/h8300-linux/bin/h8300-linux-gcc +hexagon : /home/trevor/.buildman-toolchains/gcc-4.6.1-nolibc/hexagon-linux/bin/hexagon-linux-gcc +hppa : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/hppa-linux/bin/hppa-linux-gcc +hppa64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/hppa64-linux/bin/hppa64-linux-gcc +i386 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/i386-linux/bin/i386-linux-gcc +ia64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/ia64-linux/bin/ia64-linux-gcc m32r : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/m32r-linux/bin/m32r-linux-gcc -m68k : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/m68k-linux/bin/m68k-linux-gcc -microblaze: /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc -mips : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/mips-linux/bin/mips-linux-gcc -mips64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/mips64-linux/bin/mips64-linux-gcc -nios2 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/nios2-linux/bin/nios2-linux-gcc +m68k : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/m68k-linux/bin/m68k-linux-gcc +microblaze: /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/microblaze-linux/bin/microblaze-linux-gcc +mips : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/mips-linux/bin/mips-linux-gcc +mips64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/mips64-linux/bin/mips64-linux-gcc +nds32le : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/nds32le-elf/bin/nds32le-elf-gcc +nios2 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/nios2-linux/bin/nios2-linux-gcc or1k : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/or1k-linux/bin/or1k-linux-gcc -powerpc : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc -powerpc64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc +or32 : /home/trevor/.buildman-toolchains/gcc-4.5.1-nolibc/or32-linux/bin/or32-linux-gcc +powerpc : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/powerpc-linux/bin/powerpc-linux-gcc +powerpc64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/powerpc64-linux/bin/powerpc64-linux-gcc ppc64le : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/ppc64le-linux/bin/ppc64le-linux-gcc -riscv32 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/riscv32-linux/bin/riscv32-linux-gcc -riscv64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc -s390 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/s390-linux/bin/s390-linux-gcc -sh2 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sh2-linux/bin/sh2-linux-gcc -sh4 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sh4-linux/bin/sh4-linux-gcc -sparc : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sparc-linux/bin/sparc-linux-gcc -sparc64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc +riscv32 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/riscv32-linux/bin/riscv32-linux-gcc +riscv64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/riscv64-linux/bin/riscv64-linux-gcc +s390 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/s390-linux/bin/s390-linux-gcc +s390x : /home/trevor/.buildman-toolchains/gcc-4.9.0-nolibc/s390x-linux/bin/s390x-linux-gcc +sh2 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sh2-linux/bin/sh2-linux-gcc +sh4 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sh4-linux/bin/sh4-linux-gcc +sparc : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sparc-linux/bin/sparc-linux-gcc +sparc64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/sparc64-linux/bin/sparc64-linux-gcc tilegx : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/tilegx-linux/bin/tilegx-linux-gcc tilepro : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/tilepro-linux/bin/tilepro-linux-gcc -x86_64 : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc -xtensa : /home/trevor/.buildman-toolchains/gcc-7.3.0-nolibc/xtensa-linux/bin/xtensa-linux-gcc +x86_64 : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/x86_64-linux/bin/x86_64-linux-gcc +xtensa : /home/trevor/.buildman-toolchains/gcc-8.1.0-nolibc/xtensa-linux/bin/xtensa-linux-gcc
Getting buildman to build the head revision for everything that it can, the command I used is:
$ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/master/output-dir 2>&1 | tee ~/tmp/u-boot-buildman/master/build.log and $ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/twoerner/output-dir 2>&1 | tee ~/tmp/u-boot-buildman/twoerner/build.log
Then, for results:
$ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/master/output-dir -s > ~/tmp/u-boot-buildman/master/build-summary and $ ./tools/buildman/buildman --output-dir ~/tmp/u-boot-buildman/twoerner/output-dir -s > ~/tmp/u-boot-buildman/twoerner/build-summary
I editied these result files so that each build target was listed on its own line (which makes the comparison easier). Comparing between these two builds yields:
$ diff -u master/build-summary.clean twoerner/build-summary.clean --- master/build-summary.clean 2018-11-30 20:10:33.590069536 -0500 +++ twoerner/build-summary.clean 2018-11-30 20:09:16.325652908 -0500 @@ -1,5 +1,6 @@ boards.cfg is up to date. Nothing to do. Summary of current source for 1305 boards (20 threads, 1 job per thread) + x86: + efi-x86_app aarch64: w+ khadas-vim pine_h64 pinebook @@ -34,7 +35,13 @@ sopine_baseboard orangepi_zero_plus2 mvebu_db-88f3720 - arc: w+ hsdk + powerpc: w+ MPC8641HPCN_36BIT + MPC8610HPCD + MPC8641HPCN + xpedite517x + MCR3000 + sbc8641d + arc: + iot_devkit nios2: w+ 10m50 mips: w+ imgtec_xilfpga bcm968380gerg_ram @@ -65,6 +72,7 @@ mk802_a10s am43xx_hs_evm ls1021aqds_nor_lpuart + h2200 mx7ulp_evk_plugin Linksprite_pcDuino display5 @@ -101,6 +109,7 @@ q8_a23_tablet_800x480 A20-OLinuXino-Lime imx6qdl_icore_mipi + zipitz2 ls1021atwr_nor_lpuart k2e_hs_evm ls1021atwr_sdcard_qspi @@ -139,6 +148,7 @@ Bananapro Cubieboard4 ls1021aiot_sdcard + colibri_pxa270 ls1021aiot_qspi Yones_Toptech_BS1078_V2 imx6qdl_icore_nand @@ -272,3 +282,5 @@ q8_a13_tablet A13-OLinuXinoM + openrd_base openrd_client openrd_ultimate + nds32: + adp-ag101p + adp-ae3xx
So we see 3 who new architectures are added: x86, powerpc, and nds32. And even with the existing architectures, 3 new targets are built, but arc switches from "hsdk" to "iot_devkit".
Is this meaningful?
The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in a way that is different from most toolchains. The first entry when unpacking most toolchain tarballs is:
gcc-<version>-nolib/<targetarch>-<system>
e.g.:
gcc-8.1.0-nolibc/aarch64-linux/
The first entry of the hexagon toolchain, however, is:
gcc-4.6.1-nolibc/
This causes the buildman logic in toolchain.py::ScanPath() to not be able to find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin} instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman tries to download a set of toolchains that includes hexagon, the script fails.
This update takes the second line of the tarball unpacking (which works for all the toolchains I've tested from kernel.org) and parses it to take the first two elements, separated by '/'. It makes this logic a bit more robust.
Signed-off-by: Trevor Woerner trevor@toganlabs.com --- tools/buildman/toolchain.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 442a7f977a..326c609a41 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -502,7 +502,8 @@ class Toolchains: trailing / """ stdout = command.Output('tar', 'xvfJ', fname, '-C', dest) - return stdout.splitlines()[0][:-1] + dirs = stdout.splitlines()[1].split('/')[:2] + return '/'.join(dirs)
def TestSettingsHasPath(self, path): """Check if buildman will find this toolchain
On Wed, 21 Nov 2018 at 01:32, Trevor Woerner trevor@toganlabs.com wrote:
The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in a way that is different from most toolchains. The first entry when unpacking most toolchain tarballs is:
gcc-<version>-nolib/<targetarch>-<system>e.g.:
gcc-8.1.0-nolibc/aarch64-linux/The first entry of the hexagon toolchain, however, is:
gcc-4.6.1-nolibc/This causes the buildman logic in toolchain.py::ScanPath() to not be able to find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin} instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman tries to download a set of toolchains that includes hexagon, the script fails.
This update takes the second line of the tarball unpacking (which works for all the toolchains I've tested from kernel.org) and parses it to take the first two elements, separated by '/'. It makes this logic a bit more robust.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
Reviewed-by: Simon Glass sjg@chromium.org
BTW I really appreciate your great commit msg.
- Simon
Unfortunately, for some releases the kernel.org toolchain tarball names adhere to the following pattern:
<hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xz
e.g.: x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz
while others use the following pattern:
<hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xz
Notice that the first pattern has dashes throughout, while the second has dashes throughout except just before the target architecture which has an underscore.
The "dash throughout" versions from kernel.org are:
8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1
while the "dash and underscore" versions from kernel.org are:
7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4
This tweak allows the code to handle both versions. Note that this tweak also causes the architecture parsing to get confused and find the following two bogus architectures, "2.0" and "64", which are explicitly checked for, and removed.
Signed-off-by: Trevor Woerner trevor@toganlabs.com --- tools/buildman/toolchain.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 326c609a41..4db70572e1 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -517,13 +517,14 @@ class Toolchains: def ListArchs(self): """List architectures with available toolchains to download""" host_arch, archives = self.LocateArchUrl('list') - re_arch = re.compile('[-a-z0-9.]*_([^-]*)-.*') + re_arch = re.compile('[-a-z0-9.]*[-_]([^-]*)-.*') arch_set = set() for archive in archives: # Remove the host architecture from the start arch = re_arch.match(archive[len(host_arch):]) if arch: - arch_set.add(arch.group(1)) + if arch.group(1) != "2.0" and arch.group(1) != "64": + arch_set.add(arch.group(1)) return sorted(arch_set)
def FetchAndInstall(self, arch):
On Wed, 21 Nov 2018 at 01:32, Trevor Woerner trevor@toganlabs.com wrote:
Unfortunately, for some releases the kernel.org toolchain tarball names adhere to the following pattern:
<hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xze.g.: x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz
while others use the following pattern:
<hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xze.g.:
x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xzNotice that the first pattern has dashes throughout, while the second has dashes throughout except just before the target architecture which has an underscore.
The "dash throughout" versions from kernel.org are:
8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1while the "dash and underscore" versions from kernel.org are:
7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4This tweak allows the code to handle both versions. Note that this tweak also causes the architecture parsing to get confused and find the following two bogus architectures, "2.0" and "64", which are explicitly checked for, and removed.
Signed-off-by: Trevor Woerner trevor@toganlabs.com
tools/buildman/toolchain.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Reviewed-by: Simon Glass sjg@chromium.org
But please use single quotes for strings.
On Mon, Nov 26, 2018 at 8:06 PM Simon Glass sjg@chromium.org wrote:
Reviewed-by: Simon Glass sjg@chromium.org But please use single quotes for strings.
I can send a v2 if you like?
Hi Trevor,
On Tue, 27 Nov 2018 at 17:51, Trevor Woerner twoerner@gmail.com wrote:
On Mon, Nov 26, 2018 at 8:06 PM Simon Glass sjg@chromium.org wrote:
Reviewed-by: Simon Glass sjg@chromium.org But please use single quotes for strings.
I can send a v2 if you like?
If you like, but I can fix it up when applying.
- Simon
On Wed, Nov 28, 2018 at 2:38 PM Simon Glass sjg@chromium.org wrote:
Hi Trevor,
On Tue, 27 Nov 2018 at 17:51, Trevor Woerner twoerner@gmail.com wrote:
On Mon, Nov 26, 2018 at 8:06 PM Simon Glass sjg@chromium.org wrote:
Reviewed-by: Simon Glass sjg@chromium.org But please use single quotes for strings.
I can send a v2 if you like?
If you like, but I can fix it up when applying.
Yes, please go ahead and fix it up.
On Wed, Nov 28, 2018 at 2:38 PM Simon Glass sjg@chromium.org wrote:
Hi Trevor,
On Tue, 27 Nov 2018 at 17:51, Trevor Woerner twoerner@gmail.com wrote:
On Mon, Nov 26, 2018 at 8:06 PM Simon Glass sjg@chromium.org wrote:
Reviewed-by: Simon Glass sjg@chromium.org But please use single quotes for strings.
I can send a v2 if you like?
If you like, but I can fix it up when applying.
Yes, please go ahead and fix it up.
Applied to u-boot-dm/master, thanks!
participants (6)
-
Daniel Schwierzeck -
Simon Glass -
sjg＠google.com -
Tom Rini -
Trevor Woerner -
Trevor Woerner