[U-Boot] How to test new bootloaders on Jetson TX1?
Hello,
I would like to test the latest version of U-Boot on the Jetson TX1.
Unfortunately U-Boot is lacking a README that would explain how to do that:
http://git.denx.de/?p=u-boot.git;a=tree;f=board/nvidia/p2371-2180;h=097fb9aa...
I understand that the Jetson TK1's Python based https://github.com/NVIDIA/tegra-uboot-flasher-scripts have never been updated for Tegra X1.
That leaves L4T Jetson TX1 Driver Package, latest version being R28.1.0.
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
On openSUSE Leap 42.3 this keeps failing the first time:
copying bctfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/BCT/P2180_A00_LP4_DSC_204Mhz.cfg)... done. copying bootloader(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cboot.bin)... done. populating kernel to rootfs... done. populating initrd to rootfs... done. populating extlinux.conf.emmc to rootfs... done. populating /home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb to rootfs... done. done. Making Boot image... done. copying bcffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cfg/board_config_p2597-devkit.xml)... done. Existing sosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/nvtboot_recovery.bin) reused. copying tegraboot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/nvtboot.bin)... done. Existing bpffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/bpmp.bin) reused. copying wb0boot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/warmboot.bin)... done. Existing tosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/tos.img) reused. Existing eksfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/eks.img) reused. copying dtbfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb)... done. Making system.img... /dev/loop0 is not block device. Terminating..
I have managed to work around that by running sudo losetup -f once.
That gets me to a:
U-Boot 2016.07-g0ce7ca2 (Jul 20 2017 - 00:37:03 -0700)
(where our efiboot command doesn't quite work yet)
https://elinux.org/Jetson/TX1_Upstream_Kernel
claims: "The distributed tarball contains a pre-built U-Boot and kernel, but these may be replaced with any images you wish to flash, by replacing the distributed images in the directory tree." and "To flash only the primary bootloader and U-Boot: ./flash.sh -k EBT p2371-2180 mmcblk0p1"
Apparently that's not true anymore (R23.1.1 armhf vs. R28.1.0 aarch64): Operation succeeds, but U-Boot is still the old one.
Possible cause: The EBT partition gets a cboot.bin, not u-boot-dtb.bin.
If I repeat without -k EBT and with the openSUSE OBS builds of U-Boot v2018.01 then again flashing appears to succeed, but boot gets stuck at:
[0003.260] Starting Bpmp FW [0003.263] BPMP-FW Carveout: Base = 0xff2c0000 and Size = 0x40000
(short of showing the U-Boot banner)
Also, ATF shows up as:
NOTICE: BL31: v1.2(release):cc5fd7c NOTICE: BL31: Built : 00:37:02, Jul 20 2017
(which is most certainly lacking Jan 2018 Spectre mitigations for CA57)
Security Bulletin http://nvidia.custhelp.com/app/answers/detail/a_id/4616 announces an R28.2 for later this month, but with t210 in mainline ATF I would really like to use my own patched ATF build already.
https://github.com/ARM-software/arm-trusted-firmware/blob/6f62574767546b1119... doesn't explain how to get the t210 bl31.bin onto a board either.
https://elinux.org/Jetson_TX1 links to multiple App Notes on a non-reachable Nvidia FTP server. I found that replacing ftp with http, they can still be reached (dated 2015), but http://download.nvidia.com/tegra-public-appnotes/t210-nvtboot-flow.html isn't too helpful either, other than giving a rough overview.
Colleagues have succeeded in loading a U-Boot via RCM into RAM and executing from there, but I figure that won't quite work for ATF. And it's no permanent solution anyway.
So could someone please comment on how to perform a minimally-invasive update of the individual Open Source firmware components for Jetson TX1?
Thanks, Andreas
Hi Andreas,
In the cboot + U-Boot combination, cboot loads U-Boot from the usual kernel partition (LNX or kernel depending on system), so flashing U-Boot there should do the trick. I believe this did indeed change in some L4T version, so the wiki page needs to be updated. Tom should know more about this.
I expect Varun to know the details about ATF, but I'll check if I can find some answer myself.
Thanks, Mikko
On 02/15/2018 03:51 AM, Andreas Färber wrote:
Hello,
I would like to test the latest version of U-Boot on the Jetson TX1.
Unfortunately U-Boot is lacking a README that would explain how to do that:
http://git.denx.de/?p=u-boot.git;a=tree;f=board/nvidia/p2371-2180;h=097fb9aa...
I understand that the Jetson TK1's Python based https://github.com/NVIDIA/tegra-uboot-flasher-scripts have never been updated for Tegra X1.
That leaves L4T Jetson TX1 Driver Package, latest version being R28.1.0.
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
On openSUSE Leap 42.3 this keeps failing the first time:
copying bctfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/BCT/P2180_A00_LP4_DSC_204Mhz.cfg)... done. copying bootloader(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cboot.bin)... done. populating kernel to rootfs... done. populating initrd to rootfs... done. populating extlinux.conf.emmc to rootfs... done. populating /home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb to rootfs... done. done. Making Boot image... done. copying bcffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cfg/board_config_p2597-devkit.xml)... done. Existing sosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/nvtboot_recovery.bin) reused. copying tegraboot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/nvtboot.bin)... done. Existing bpffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/bpmp.bin) reused. copying wb0boot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/warmboot.bin)... done. Existing tosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/tos.img) reused. Existing eksfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/eks.img) reused. copying dtbfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb)... done. Making system.img... /dev/loop0 is not block device. Terminating..
I have managed to work around that by running sudo losetup -f once.
That gets me to a:
U-Boot 2016.07-g0ce7ca2 (Jul 20 2017 - 00:37:03 -0700)
(where our efiboot command doesn't quite work yet)
https://elinux.org/Jetson/TX1_Upstream_Kernel
claims: "The distributed tarball contains a pre-built U-Boot and kernel, but these may be replaced with any images you wish to flash, by replacing the distributed images in the directory tree." and "To flash only the primary bootloader and U-Boot: ./flash.sh -k EBT p2371-2180 mmcblk0p1"
Apparently that's not true anymore (R23.1.1 armhf vs. R28.1.0 aarch64): Operation succeeds, but U-Boot is still the old one.
Possible cause: The EBT partition gets a cboot.bin, not u-boot-dtb.bin.
If I repeat without -k EBT and with the openSUSE OBS builds of U-Boot v2018.01 then again flashing appears to succeed, but boot gets stuck at:
[0003.260] Starting Bpmp FW [0003.263] BPMP-FW Carveout: Base = 0xff2c0000 and Size = 0x40000
(short of showing the U-Boot banner)
Also, ATF shows up as:
NOTICE: BL31: v1.2(release):cc5fd7c NOTICE: BL31: Built : 00:37:02, Jul 20 2017
(which is most certainly lacking Jan 2018 Spectre mitigations for CA57)
Security Bulletin http://nvidia.custhelp.com/app/answers/detail/a_id/4616 announces an R28.2 for later this month, but with t210 in mainline ATF I would really like to use my own patched ATF build already.
https://github.com/ARM-software/arm-trusted-firmware/blob/6f62574767546b1119... doesn't explain how to get the t210 bl31.bin onto a board either.
https://elinux.org/Jetson_TX1 links to multiple App Notes on a non-reachable Nvidia FTP server. I found that replacing ftp with http, they can still be reached (dated 2015), but http://download.nvidia.com/tegra-public-appnotes/t210-nvtboot-flow.html isn't too helpful either, other than giving a rough overview.
Colleagues have succeeded in loading a U-Boot via RCM into RAM and executing from there, but I figure that won't quite work for ATF. And it's no permanent solution anyway.
So could someone please comment on how to perform a minimally-invasive update of the individual Open Source firmware components for Jetson TX1?
Thanks, Andreas
Hi Mikko,
Am 15.02.2018 um 08:56 schrieb Mikko Perttunen:
In the cboot + U-Boot combination, cboot loads U-Boot from the usual kernel partition (LNX or kernel depending on system), so flashing U-Boot there should do the trick. I believe this did indeed change in some L4T version, so the wiki page needs to be updated. Tom should know more about this.
Thanks for explaining.
The LNX partition is getting a boot.img - what is the relation to the four U-Boot files? flash.sh source is not really helping here.
Might any changes to RP1 and/or DTB partitions be necessary to match my newer U-Boot, or does U-Boot use an internal .dtb these days?
Is there any more efficient way to flash just U-Boot? -k LNX possibly?
I had played with the -L option before (which mentions u-boot-dtb.bin), but recall it ran into some form of checksum error on boot when passing it my file...
I expect Varun to know the details about ATF, but I'll check if I can find some answer myself.
Thanks again for your efforts.
Regards, Andreas
On 15.02.2018 15:25, Andreas Färber wrote:
Hi Mikko,
Am 15.02.2018 um 08:56 schrieb Mikko Perttunen:
In the cboot + U-Boot combination, cboot loads U-Boot from the usual kernel partition (LNX or kernel depending on system), so flashing U-Boot there should do the trick. I believe this did indeed change in some L4T version, so the wiki page needs to be updated. Tom should know more about this.
Thanks for explaining.
The LNX partition is getting a boot.img - what is the relation to the four U-Boot files? flash.sh source is not really helping here.
cboot is only capable of booting Android .img's so the U-Boot binary (u-boot-dtb.bin according ton Jon) is packaged as one using mkbootimg or similar - I expect flash.sh to do this automatically.
Might any changes to RP1 and/or DTB partitions be necessary to match my newer U-Boot, or does U-Boot use an internal .dtb these days?
My understanding is that it uses an internal .dtb.
Is there any more efficient way to flash just U-Boot? -k LNX possibly?
That might work. Jon should know the details.
I had played with the -L option before (which mentions u-boot-dtb.bin), but recall it ran into some form of checksum error on boot when passing it my file...
I expect Varun to know the details about ATF, but I'll check if I can find some answer myself.
Thanks again for your efforts.
I found out that ATF and optionally a secure OS (Trusty usually) are contained in the file tos.img. The image contains a padded header, followed by bl31.bin and the secure OS binary concatenated.
The header format should be pretty straightforward to reverse-engineer (need to change some fields specifying lengths), but I'll ask around if we can get a script to generate it released.
Cheers, Mikko
Regards, Andreas
On 15/02/18 01:51, Andreas Färber wrote:
Hello,
I would like to test the latest version of U-Boot on the Jetson TX1.
Unfortunately U-Boot is lacking a README that would explain how to do that:
http://git.denx.de/?p=u-boot.git;a=tree;f=board/nvidia/p2371-2180;h=097fb9aa...
I understand that the Jetson TK1's Python based https://github.com/NVIDIA/tegra-uboot-flasher-scripts have never been updated for Tegra X1.
That leaves L4T Jetson TX1 Driver Package, latest version being R28.1.0.
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
Cheers Jon
Am 15.02.2018 um 10:22 schrieb Jon Hunter:
On 15/02/18 01:51, Andreas Färber wrote:
I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
I downloaded the .rpm from https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180 extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described it makes a difference in that it then ceases to boot to a U-Boot prompt.
I then have to use jumper J9 to enter RCM again.
So something is getting flashed, it's just not working right.
Regards, Andreas
On 15/02/18 12:32, Andreas Färber wrote:
Am 15.02.2018 um 10:22 schrieb Jon Hunter:
On 15/02/18 01:51, Andreas Färber wrote:
I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
I downloaded the .rpm from https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180 extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described it makes a difference in that it then ceases to boot to a U-Boot prompt.
Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However, I believe you are still booting the wrong u-boot file.
If you look at the p2371-2180-devkit.conf you will see that it has ...
DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
For using the upstream u-boot, we need to use the u-boot-dtb.bin and not the u-boot.bin. So you can either ...
1. Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that you copied. 2. Move the u-boot-dtb.bin from your rpm to bootloader/t210ref/p2371-2180/u-boot.bin.
Cheers Jon
Hi Jon,
Am 15.02.2018 um 15:01 schrieb Jon Hunter:
On 15/02/18 12:32, Andreas Färber wrote:
Am 15.02.2018 um 10:22 schrieb Jon Hunter:
On 15/02/18 01:51, Andreas Färber wrote:
I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
I downloaded the .rpm from https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180 extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described it makes a difference in that it then ceases to boot to a U-Boot prompt.
Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However, I believe you are still booting the wrong u-boot file.
If you look at the p2371-2180-devkit.conf you will see that it has ...
DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
For using the upstream u-boot, we need to use the u-boot-dtb.bin and not the u-boot.bin. So you can either ...
- Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that you
copied. 2. Move the u-boot-dtb.bin from your rpm to bootloader/t210ref/p2371-2180/u-boot.bin.
Both in the Nvidia tarball and in our upstream based package builds, u-boot.bin and u-boot-dtb.bin are identical. Since some time u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.
Should this work with the vanilla upstream files, or do they need any headers or signatures or something?
Do any of you have that upstream version booting successfully, or might it be an upstream U-Boot regression? I was so far assuming it's a user error, so haven't tried bisecting yet.
Regards, Andreas
On 15/02/18 15:12, Andreas Färber wrote:
Hi Jon,
Am 15.02.2018 um 15:01 schrieb Jon Hunter:
On 15/02/18 12:32, Andreas Färber wrote:
Am 15.02.2018 um 10:22 schrieb Jon Hunter:
On 15/02/18 01:51, Andreas Färber wrote:
I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
I downloaded the .rpm from https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180 extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described it makes a difference in that it then ceases to boot to a U-Boot prompt.
Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However, I believe you are still booting the wrong u-boot file.
If you look at the p2371-2180-devkit.conf you will see that it has ...
DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
For using the upstream u-boot, we need to use the u-boot-dtb.bin and not the u-boot.bin. So you can either ...
- Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that you
copied. 2. Move the u-boot-dtb.bin from your rpm to bootloader/t210ref/p2371-2180/u-boot.bin.
Both in the Nvidia tarball and in our upstream based package builds, u-boot.bin and u-boot-dtb.bin are identical. Since some time u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.
Good to know!
Should this work with the vanilla upstream files, or do they need any headers or signatures or something?
Yes should work fine with vanilla upstream.
Do any of you have that upstream version booting successfully, or might it be an upstream U-Boot regression? I was so far assuming it's a user error, so haven't tried bisecting yet.
I am using "U-Boot 2017.03". However, I realise that I am currently based upon L4T rel24.2.1 (as rel28.1 was not available when I started using L4T for testing upstream). I hope it would not matter, but you never know. I plan to move to rel28.2 once it is released. If we think it is a problem with rel28.1 I should be able to test my side.
Cheers Jon
-----Original Message----- From: Jonathan Hunter Sent: Thursday, February 15, 2018 8:39 AM To: Andreas Färber afaerber@suse.de Cc: linux-tegra@vger.kernel.org; U-Boot u-boot@lists.denx.de; Alexander Graf agraf@suse.de; Mian Yousaf Kaukab yousaf.kaukab@suse.com; Tom Warren TWarren@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: How to test new bootloaders on Jetson TX1?
On 15/02/18 15:12, Andreas Färber wrote:
Hi Jon,
Am 15.02.2018 um 15:01 schrieb Jon Hunter:
On 15/02/18 12:32, Andreas Färber wrote:
Am 15.02.2018 um 10:22 schrieb Jon Hunter:
On 15/02/18 01:51, Andreas Färber wrote:
I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
I downloaded the .rpm from https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2 180 extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described it makes a difference in that it then ceases to boot to a U-Boot prompt.
Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However, I believe you are still booting the wrong u-boot file.
If you look at the p2371-2180-devkit.conf you will see that it has ...
DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
For using the upstream u-boot, we need to use the u-boot-dtb.bin and not the u-boot.bin. So you can either ...
- Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that
you copied. 2. Move the u-boot-dtb.bin from your rpm to bootloader/t210ref/p2371-2180/u-boot.bin.
Both in the Nvidia tarball and in our upstream based package builds, u-boot.bin and u-boot-dtb.bin are identical. Since some time u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.
Good to know!
Should this work with the vanilla upstream files, or do they need any headers or signatures or something?
Yes should work fine with vanilla upstream.
Do any of you have that upstream version booting successfully, or might it be an upstream U-Boot regression? I was so far assuming it's a user error, so haven't tried bisecting yet.
I am using "U-Boot 2017.03". However, I realise that I am currently based upon L4T rel24.2.1 (as rel28.1 was not available when I started using L4T for testing upstream). I hope it would not matter, but you never know. I plan to move to rel28.2 once it is released. If we think it is a problem with rel28.1 I should be able to test my side.
[Tom] Current (R28.x) U-Boot requires CBoot to load - we no longer use NVTBoot as we did in R24.x. Hence, upstream U-Boot needs some modifications to be able to be loaded by CBoot instead of NVTBoot. I haven't done that work yet (patches to convert upstream Denx U-Boot source for T210/T186 to use CBoot as the loader - don't know how I'll structure it so old (NVTBoot) bootflow will still work as well as the new (CBoot) bootflow). I've got it on my plate, but other priorities have prevented me from working on it. I'll try to get it into the queue. Until then, you won't be able to load/run upstream U-Boot using the boot flow in R28.x. Sorry.
Cheers Jon
-- nvpublic
Andreas, can you try the TOS packaging script available in our public repo?
http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=t...
Please let me know if this does not work for you.
For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
-----Original Message----- From: Tom Warren Sent: Thursday, February 15, 2018 7:44 AM To: Jonathan Hunter jonathanh@nvidia.com; Andreas Färber afaerber@suse.de Cc: linux-tegra@vger.kernel.org; U-Boot u-boot@lists.denx.de; Alexander Graf agraf@suse.de; Mian Yousaf Kaukab yousaf.kaukab@suse.com; Varun Wadekar vwadekar@nvidia.com Subject: RE: How to test new bootloaders on Jetson TX1?
-----Original Message----- From: Jonathan Hunter Sent: Thursday, February 15, 2018 8:39 AM To: Andreas Färber afaerber@suse.de Cc: linux-tegra@vger.kernel.org; U-Boot u-boot@lists.denx.de; Alexander Graf agraf@suse.de; Mian Yousaf Kaukab yousaf.kaukab@suse.com; Tom Warren TWarren@nvidia.com; Varun Wadekar vwadekar@nvidia.com Subject: Re: How to test new bootloaders on Jetson TX1?
On 15/02/18 15:12, Andreas Färber wrote:
Hi Jon,
Am 15.02.2018 um 15:01 schrieb Jon Hunter:
On 15/02/18 12:32, Andreas Färber wrote:
Am 15.02.2018 um 10:22 schrieb Jon Hunter:
On 15/02/18 01:51, Andreas Färber wrote:
I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
Here's what I have tried:
$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you copying it?
I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
I downloaded the .rpm from https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2 180 extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described it makes a difference in that it then ceases to boot to a U-Boot prompt.
Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However, I believe you are still booting the wrong u-boot file.
If you look at the p2371-2180-devkit.conf you will see that it has ...
DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
For using the upstream u-boot, we need to use the u-boot-dtb.bin and not the u-boot.bin. So you can either ...
- Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that
you copied. 2. Move the u-boot-dtb.bin from your rpm to bootloader/t210ref/p2371-2180/u-boot.bin.
Both in the Nvidia tarball and in our upstream based package builds, u-boot.bin and u-boot-dtb.bin are identical. Since some time u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.
Good to know!
Should this work with the vanilla upstream files, or do they need any headers or signatures or something?
Yes should work fine with vanilla upstream.
Do any of you have that upstream version booting successfully, or might it be an upstream U-Boot regression? I was so far assuming it's a user error, so haven't tried bisecting yet.
I am using "U-Boot 2017.03". However, I realise that I am currently based upon L4T rel24.2.1 (as rel28.1 was not available when I started using L4T for testing upstream). I hope it would not matter, but you never know. I plan to move to rel28.2 once it is released. If we think it is a problem with rel28.1 I should be able to test my side.
[Tom] Current (R28.x) U-Boot requires CBoot to load - we no longer use NVTBoot as we did in R24.x. Hence, upstream U-Boot needs some modifications to be able to be loaded by CBoot instead of NVTBoot. I haven't done that work yet (patches to convert upstream Denx U-Boot source for T210/T186 to use CBoot as the loader - don't know how I'll structure it so old (NVTBoot) bootflow will still work as well as the new (CBoot) bootflow). I've got it on my plate, but other priorities have prevented me from working on it. I'll try to get it into the queue. Until then, you won't be able to load/run upstream U-Boot using the boot flow in R28.x. Sorry.
Cheers Jon
-- nvpublic
Hi Varun,
Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
Andreas, can you try the TOS packaging script available in our public repo?
http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=t...
Great, that script does work. It is lacking usage output, but looking at the code, its arguments were self-documenting.
Please let me know if this does not work for you.
For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports, without SPD. BL31 appears to initialize okay, but later something runs into an unhandled SMC 0x82000015 - things then go south and it doesn't reach the Nvidia U-Boot. Serial log attached.
According to https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-si... that SMC function ID falls into the SiP range, so indeed something Nvidia-specific missing in v1.4?
Regards, Andreas
https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-t...
Yes. That's a custom SMC we have for some non-L4T uses. It has not been upstreamed yet.
You can add dummy handling in tegra_sip_calls.c to move forward.
________________________________ From: Andreas Färber afaerber@suse.de Sent: Thursday, February 15, 2018 7:04 PM To: Varun Wadekar Cc: Tom Warren; Jonathan Hunter; linux-tegra@vger.kernel.org; U-Boot; Alexander Graf; Mian Yousaf Kaukab Subject: Re: How to test new bootloaders on Jetson TX1? - ATF
Hi Varun,
Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
Andreas, can you try the TOS packaging script available in our public repo?
http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=t...
Great, that script does work. It is lacking usage output, but looking at the code, its arguments were self-documenting.
Please let me know if this does not work for you.
For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports, without SPD. BL31 appears to initialize okay, but later something runs into an unhandled SMC 0x82000015 - things then go south and it doesn't reach the Nvidia U-Boot. Serial log attached.
According to https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-si... that SMC function ID falls into the SiP range, so indeed something Nvidia-specific missing in v1.4?
Regards, Andreas
https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-t...
-- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)
Am 16.02.2018 um 07:18 schrieb Varun Wadekar:
Yes. That's a custom SMC we have for some non-L4T uses. It has not been upstreamed yet.
You can add dummy handling in tegra_sip_calls.c to move forward.
Thanks, on my 1.4 based branch I succeeded to boot into Nvidia U-Boot with the following change:
--- a/plat/nvidia/tegra/common/tegra_sip_calls.c +++ b/plat/nvidia/tegra/common/tegra_sip_calls.c @@ -162,6 +162,11 @@ uint64_t tegra_sip_handler(uint32_t smc_fid, */ break;
+ case 0x82000015: + INFO("%s: SMC 0x%x\n", __func__, smc_fid); + SMC_RET1(handle, 0); + break; + default: ERROR("%s: unhandled SMC (0x%x)\n", __func__, smc_fid); break;
The INFO output shows up exactly once.
On master branch I've further needed to fix a regression: https://github.com/ARM-software/arm-trusted-firmware/pull/1271
Regards, Andreas
*From:* Andreas Färber afaerber@suse.de *Sent:* Thursday, February 15, 2018 7:04 PM *To:* Varun Wadekar *Cc:* Tom Warren; Jonathan Hunter; linux-tegra@vger.kernel.org; U-Boot; Alexander Graf; Mian Yousaf Kaukab *Subject:* Re: How to test new bootloaders on Jetson TX1? - ATF
Hi Varun,
Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
Andreas, can you try the TOS packaging script available in our public repo?
http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=t...
Great, that script does work. It is lacking usage output, but looking at the code, its arguments were self-documenting.
Please let me know if this does not work for you.
For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports, without SPD. BL31 appears to initialize okay, but later something runs into an unhandled SMC 0x82000015 - things then go south and it doesn't reach the Nvidia U-Boot. Serial log attached.
According to https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-si... that SMC function ID falls into the SiP range, so indeed something Nvidia-specific missing in v1.4?
Regards, Andreas
https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-t...
Sounds good. I reviewed the ATF change upstream. Looks good.
________________________________ From: Andreas Färber afaerber@suse.de Sent: Friday, February 16, 2018 7:34 PM To: Varun Wadekar Cc: Tom Warren; Jonathan Hunter; linux-tegra@vger.kernel.org; U-Boot; Alexander Graf; Mian Yousaf Kaukab Subject: Re: How to test new bootloaders on Jetson TX1? - ATF
Am 16.02.2018 um 07:18 schrieb Varun Wadekar:
Yes. That's a custom SMC we have for some non-L4T uses. It has not been upstreamed yet.
You can add dummy handling in tegra_sip_calls.c to move forward.
Thanks, on my 1.4 based branch I succeeded to boot into Nvidia U-Boot with the following change:
--- a/plat/nvidia/tegra/common/tegra_sip_calls.c +++ b/plat/nvidia/tegra/common/tegra_sip_calls.c @@ -162,6 +162,11 @@ uint64_t tegra_sip_handler(uint32_t smc_fid, */ break;
+ case 0x82000015: + INFO("%s: SMC 0x%x\n", __func__, smc_fid); + SMC_RET1(handle, 0); + break; + default: ERROR("%s: unhandled SMC (0x%x)\n", __func__, smc_fid); break;
The INFO output shows up exactly once.
On master branch I've further needed to fix a regression: https://github.com/ARM-software/arm-trusted-firmware/pull/1271
Regards, Andreas
*From:* Andreas Färber afaerber@suse.de *Sent:* Thursday, February 15, 2018 7:04 PM *To:* Varun Wadekar *Cc:* Tom Warren; Jonathan Hunter; linux-tegra@vger.kernel.org; U-Boot; Alexander Graf; Mian Yousaf Kaukab *Subject:* Re: How to test new bootloaders on Jetson TX1? - ATF
Hi Varun,
Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
Andreas, can you try the TOS packaging script available in our public repo?
http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=t...
Great, that script does work. It is lacking usage output, but looking at the code, its arguments were self-documenting.
Please let me know if this does not work for you.
For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports, without SPD. BL31 appears to initialize okay, but later something runs into an unhandled SMC 0x82000015 - things then go south and it doesn't reach the Nvidia U-Boot. Serial log attached.
According to https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-si... that SMC function ID falls into the SiP range, so indeed something Nvidia-specific missing in v1.4?
Regards, Andreas
https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-t...
-- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Graham Norton HRB 21284 (AG Nürnberg)
On 02/14/2018 06:51 PM, Andreas Färber wrote:
Hello,
I would like to test the latest version of U-Boot on the Jetson TX1.
Unfortunately U-Boot is lacking a README that would explain how to do that: ...
Here is some consolidated background on U-Boot on TX1:
In all cases, U-Boot uses a built-in DTB for its own driver initialization and other operation.
In T124 and earlier, it was possible to have a completely OSS boot process. So, U-Boot could be the only bootloader. In this case, part of U-Boot (the SPL) would run on the ARM7/AVP/COP, and part on the main CPU (CCPLEX). Details of all that are at:
https://http.download.nvidia.com/tegra-public-appnotes/index.html
With T210, the boot logic that runs on the ARM7/AVP/BPMP-Lite became more complex, so we elected to drop the U-Boot SPL and re-use the NVIDIA binary bootloader for the AVP/BPMP-Lite portion to avoid re-implementing it, leaving U-Boot to run solely on the CCPLEX, and dropping usage of U-Boot SPL.
In L4T r24 and earlier, U-Boot was 99.9% of the boot code that ran at EL2 or lower on the CCPLEX (ATF - ARM Trusted Firmware - would run at EL3). In this case, the other 0.1% of the code running at EL2 runs before U-Boot and passes to U-Boot some parameter block with details such as memory size, carveouts, etc. The L4T r24 T210 port of U-Boot parses this parameter block to determine RAM size, carveout size, etc., so is closely tied to this boot model. The upstream T210 port (as of today at least) ignores the parameter block, determines RAM size from HW registers, and hard-codes some overly-large carveout size to avoid trampling any carveout. In this r24 boot model, U-Boot (L4T downstream or upstream) loads a DTB from disk, does a little manipulation of it (e.g. fills in RAM size, kernel command-line), and passes it a kernel that it loads from disk.
With L4T r24, you can use the script bootloader/exec-uboot.sh to load and run a new U-Boot over the USB port's recovery protocol, without writing anything to flash. This is what my automated upstream U-Boot tester does for usptream branches for T210.
I won't discuss L4T r25/r26 since they weren't broadly distributed and had slightly different boot models, and discussing them would just be confusing.
With L4T r28, we aligned the T210 and T186 boot models. Now, cboot (an NVIDIA binary bootloader) always runs on the CCPLEX at EL2 (ATF still runs EL3) and performs most general system setup. cboot loads the "kernel" from a dedicated partition (which contains no filesystem). This kernel could actually be a Linux kernel, but L4T typically places U-Boot into this partition and hence chain-loads it. In this model, U-Boot receives a DTB from cboot, which is parsed by U-Boot to determine RAM size, carveout size, etc. For L4T, U-Boot passes this same DTB on to whatever kernel it loads and boots (with a couple minor modifications such as over-writing RAM size and kernel command-line). The advantage of passing on the DTB is that any DTB edits performed by cboot don't need to be re-implemnted by U-Boot. For upstream, people typically (always?) have U-Boot load a different DTB to pass to the kernel, since the DTB that cboot uses-and-passes-on must use some downstream-specific DT bindings/schema that are not appropriate to pass to an upstream kernel that uses upstream DT bindings. Only downstream L4T r28 U-Boot supports this boot model; upstream U-Boot is not written to accept and parse a DTB at runtime. You can control whether U-Boot passes on the DTB or loads a new one by (a) extlinux.conf: include a DTB statement or not, (b) in other scripts, by either loading a DTB in the script and passing that address to booti or passing the address of the cboot-supplied DTB to booti; U-Boot sets a variable to point to the cboot-supplied DTB making this easy. When running L4T r28, if you want update U-Boot, you'll need to flash the kernel partition (T210: LNX, T186: kernel) since that's where U-Boot is stored. Obviously L4T's r28 U-Boot release works with this model. I'd actually expect upstream to do so too, although that combination isn't actually tested; my automated upstream U-Boot tester still uses L4T r24 since that boot process is what the upstream T210 port was originally developed against. This may change if we switch upstream U-Boot to the new boot style.
With L4T r28, there is no bootloader/exec-uboot.sh script, so you must write any updated U-Boot to flash in order to test it.
For T186, this new boot model was all we ever supported; cboot is always used, U-Boot if uses is stored in the kernel partition, and upstream T186 U-Boot only supports the new L4T r28 boot model.
ATF and optionally a secure OS are part of "tos.img" in the L4T release, which gets flashed to the TOS partition on T210 and secure-os partition on T186. As Varun mentioned, there's a header format for this partition and you can use the script he linked in order to generate the header. I don't know of anyone using upstream ATF on T210 or T186.
Hopefully that addresses all the questions in this thread; if not let me know!
participants (6)
-
Andreas Färber -
Jon Hunter -
Mikko Perttunen -
Stephen Warren -
Tom Warren -
Varun Wadekar