Re: U-Boot is broken on real N900 HW
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote:
On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote:
This patch series contain fixes for Nokia RX-51 board (aka N900). After these changes it is possible to run U-Boot in qemu emulator again. And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without problem.
But on real Nokia N900 device is U-Boot crashing in reboot loop.
I do not have serial console for Nokia N900 to debug this issue, but seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is that there is no crash and even no error in qemu emulator so I cannot debug this issue.
First problem is around /* reset lp5523 led */ code in rx51.c. On real N900 device it generates repeating messages:
Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000
When I commented that few lines all these messages disappeared. So problem is with OMAP I2C.
Second problem happen after misc_init_r() function finishes. U-Boot just prints on N900 screen message "data abort" and reboots. As I do not have serial console it is hard to debug. but I figured out that problem is in mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put debug info prior to mmc_set_ios() call and after it, and debug info prior was printed, not after.
I remember that somebody had serial jig for Nokia N900, could somebody look at this reboot loop problem?
And any idea how should be OMAP I2C configured in U-Boot to correctly work?
Maybe I will try to find some time to git bisect which change broke U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);
Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote:
On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote:
This patch series contain fixes for Nokia RX-51 board (aka N900). After these changes it is possible to run U-Boot in qemu emulator again. And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without problem.
But on real Nokia N900 device is U-Boot crashing in reboot loop.
I do not have serial console for Nokia N900 to debug this issue, but seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is that there is no crash and even no error in qemu emulator so I cannot debug this issue.
First problem is around /* reset lp5523 led */ code in rx51.c. On real N900 device it generates repeating messages:
Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000
When I commented that few lines all these messages disappeared. So problem is with OMAP I2C.
Second problem happen after misc_init_r() function finishes. U-Boot just prints on N900 screen message "data abort" and reboots. As I do not have serial console it is hard to debug. but I figured out that problem is in mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put debug info prior to mmc_set_ios() call and after it, and debug info prior was printed, not after.
I remember that somebody had serial jig for Nokia N900, could somebody look at this reboot loop problem?
And any idea how should be OMAP I2C configured in U-Boot to correctly work?
Maybe I will try to find some time to git bisect which change broke U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);
Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote:
On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote:
This patch series contain fixes for Nokia RX-51 board (aka N900). After these changes it is possible to run U-Boot in qemu emulator again. And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without problem.
But on real Nokia N900 device is U-Boot crashing in reboot loop.
I do not have serial console for Nokia N900 to debug this issue, but seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is that there is no crash and even no error in qemu emulator so I cannot debug this issue.
First problem is around /* reset lp5523 led */ code in rx51.c. On real N900 device it generates repeating messages:
Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000
When I commented that few lines all these messages disappeared. So problem is with OMAP I2C.
Second problem happen after misc_init_r() function finishes. U-Boot just prints on N900 screen message "data abort" and reboots. As I do not have serial console it is hard to debug. but I figured out that problem is in mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put debug info prior to mmc_set_ios() call and after it, and debug info prior was printed, not after.
I remember that somebody had serial jig for Nokia N900, could somebody look at this reboot loop problem?
And any idea how should be OMAP I2C configured in U-Boot to correctly work?
Maybe I will try to find some time to git bisect which change broke U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote:
On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote:
This patch series contain fixes for Nokia RX-51 board (aka N900). After these changes it is possible to run U-Boot in qemu emulator again. And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without problem.
But on real Nokia N900 device is U-Boot crashing in reboot loop.
I do not have serial console for Nokia N900 to debug this issue, but seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is that there is no crash and even no error in qemu emulator so I cannot debug this issue.
First problem is around /* reset lp5523 led */ code in rx51.c. On real N900 device it generates repeating messages:
Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000
When I commented that few lines all these messages disappeared. So problem is with OMAP I2C.
Second problem happen after misc_init_r() function finishes. U-Boot just prints on N900 screen message "data abort" and reboots. As I do not have serial console it is hard to debug. but I figured out that problem is in mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put debug info prior to mmc_set_ios() call and after it, and debug info prior was printed, not after.
I remember that somebody had serial jig for Nokia N900, could somebody look at this reboot loop problem?
And any idea how should be OMAP I2C configured in U-Boot to correctly work?
Maybe I will try to find some time to git bisect which change broke U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote:
On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: > This patch series contain fixes for Nokia RX-51 board (aka N900). > After these changes it is possible to run U-Boot in qemu emulator again. > And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without > problem.
But on real Nokia N900 device is U-Boot crashing in reboot loop.
I do not have serial console for Nokia N900 to debug this issue, but seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is that there is no crash and even no error in qemu emulator so I cannot debug this issue.
First problem is around /* reset lp5523 led */ code in rx51.c. On real N900 device it generates repeating messages:
Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000
When I commented that few lines all these messages disappeared. So problem is with OMAP I2C.
Second problem happen after misc_init_r() function finishes. U-Boot just prints on N900 screen message "data abort" and reboots. As I do not have serial console it is hard to debug. but I figured out that problem is in mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put debug info prior to mmc_set_ios() call and after it, and debug info prior was printed, not after.
I remember that somebody had serial jig for Nokia N900, could somebody look at this reboot loop problem?
And any idea how should be OMAP I2C configured in U-Boot to correctly work?
Maybe I will try to find some time to git bisect which change broke U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
I would highly consider migrating to the driver model if you can. Using the driver model, I was able to greatly reduce the size of board file and the manual initialization of drivers, because the DM does that for you. I also know they're advocating removing boards which do not comply with the DM requirements.
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote: > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >> This patch series contain fixes for Nokia RX-51 board (aka N900). >> After these changes it is possible to run U-Boot in qemu emulator again. >> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without >> problem. > > But on real Nokia N900 device is U-Boot crashing in reboot loop. > > I do not have serial console for Nokia N900 to debug this issue, but > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is > that there is no crash and even no error in qemu emulator so I cannot > debug this issue. > > First problem is around /* reset lp5523 led */ code in rx51.c. On real > N900 device it generates repeating messages: > > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > > When I commented that few lines all these messages disappeared. So > problem is with OMAP I2C. > > Second problem happen after misc_init_r() function finishes. U-Boot just > prints on N900 screen message "data abort" and reboots. As I do not have > serial console it is hard to debug. but I figured out that problem is in > mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put > debug info prior to mmc_set_ios() call and after it, and debug info > prior was printed, not after. > > I remember that somebody had serial jig for Nokia N900, could somebody > look at this reboot loop problem? > > And any idea how should be OMAP I2C configured in U-Boot to correctly > work? > > Maybe I will try to find some time to git bisect which change broke > U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
I would highly consider migrating to the driver model if you can.
Until it works correctly, I cannot do migration.
Using the driver model, I was able to greatly reduce the size of board file and the manual initialization of drivers, because the DM does that for you. I also know they're advocating removing boards which do not comply with the DM requirements.
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: > Hi, > > On 01/04/2020 00:42, Pali Rohár wrote: > > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: > >> This patch series contain fixes for Nokia RX-51 board (aka N900). > >> After these changes it is possible to run U-Boot in qemu emulator again. > >> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without > >> problem. > > > > But on real Nokia N900 device is U-Boot crashing in reboot loop. > > > > I do not have serial console for Nokia N900 to debug this issue, but > > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is > > that there is no crash and even no error in qemu emulator so I cannot > > debug this issue. > > > > First problem is around /* reset lp5523 led */ code in rx51.c. On real > > N900 device it generates repeating messages: > > > > Check if pads/pull-ups of bus are properly configured > > Timed out in wait_for_event: status=0000 > > > > When I commented that few lines all these messages disappeared. So > > problem is with OMAP I2C.
...
> > I remember that somebody had serial jig for Nokia N900, could somebody > > look at this reboot loop problem? > > > > And any idea how should be OMAP I2C configured in U-Boot to correctly > > work? > > > > Maybe I will try to find some time to git bisect which change broke > > U-Boot on real N900 hardware. > > Took latest u-boot master, applied patches and this is the result on > serial (first part is NOLO booting, I think that can be ignored) [1].
...
> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz > Nokia RX-51 + LPDDR/OneNAND > I2C: ready > DRAM: 256 MiB > NAND: 0 Bytes
Looks like that something with NAND is broken.
> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 > In: vga > Out: vga > Err: vga > Timed out in wait_for_event: status=0100 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > i2c_read (addr phase): pads on bus probably not configured (status=0x10) > i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
...
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed
Adds support for set-speed on the OMAP24xx I2C Adapter.
Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...).
Minor cosmetical changes.
Signed-off-by: Hannes Petermaier oe5hpm@oevsv.at Cc: Heiko Schocher hs@denx.de
U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: > On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >> Hi, >> >> On 01/04/2020 00:42, Pali Rohár wrote: >>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>> This patch series contain fixes for Nokia RX-51 board (aka N900). >>>> After these changes it is possible to run U-Boot in qemu emulator again. >>>> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without >>>> problem. >>> >>> But on real Nokia N900 device is U-Boot crashing in reboot loop. >>> >>> I do not have serial console for Nokia N900 to debug this issue, but >>> seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is >>> that there is no crash and even no error in qemu emulator so I cannot >>> debug this issue. >>> >>> First problem is around /* reset lp5523 led */ code in rx51.c. On real >>> N900 device it generates repeating messages: >>> >>> Check if pads/pull-ups of bus are properly configured >>> Timed out in wait_for_event: status=0000 >>> >>> When I commented that few lines all these messages disappeared. So >>> problem is with OMAP I2C.
...
>>> I remember that somebody had serial jig for Nokia N900, could somebody >>> look at this reboot loop problem? >>> >>> And any idea how should be OMAP I2C configured in U-Boot to correctly >>> work? >>> >>> Maybe I will try to find some time to git bisect which change broke >>> U-Boot on real N900 hardware. >> >> Took latest u-boot master, applied patches and this is the result on >> serial (first part is NOLO booting, I think that can be ignored) [1]. > > ... > >> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) >> >> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >> Nokia RX-51 + LPDDR/OneNAND >> I2C: ready >> DRAM: 256 MiB >> NAND: 0 Bytes > > Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
> >> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >> In: vga >> Out: vga >> Err: vga >> Timed out in wait_for_event: status=0100 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> Timed out in wait_for_event: status=0000 >> Check if pads/pull-ups of bus are properly configured >> i2c_read (addr phase): pads on bus probably not configured (status=0x10) >> i2c_write: timed out writig last byte! > > These i2c errors are caused by > > /* reset lp5523 led */ > i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
> state = 0xff; > i2c_write(0x32, 0x3d, 1, &state, 1); > i2c_set_bus_num(0); > > Is there anything which needs to be done to initialize i2c bus 1? > Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
...
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier <oe5hpm@oevsv.at> Cc: Heiko Schocher <hs@denx.de>U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
- udelay(I2C_WAIT); + udelay(adap->waitdelay);
May you can check, if adap->waitdelay is the same as I2C_WAIT ?
bye, Heiko
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote: > > On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: > > On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: > > > Hi, > > > > > > On 01/04/2020 00:42, Pali Rohár wrote: > > > > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: > > > > > This patch series contain fixes for Nokia RX-51 board (aka N900). > > > > > After these changes it is possible to run U-Boot in qemu emulator again. > > > > > And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without > > > > > problem. > > > > > > > > But on real Nokia N900 device is U-Boot crashing in reboot loop. > > > > > > > > I do not have serial console for Nokia N900 to debug this issue, but > > > > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is > > > > that there is no crash and even no error in qemu emulator so I cannot > > > > debug this issue. > > > > > > > > First problem is around /* reset lp5523 led */ code in rx51.c. On real > > > > N900 device it generates repeating messages: > > > > > > > > Check if pads/pull-ups of bus are properly configured > > > > Timed out in wait_for_event: status=0000 > > > > > > > > When I commented that few lines all these messages disappeared. So > > > > problem is with OMAP I2C.
...
> > > > I remember that somebody had serial jig for Nokia N900, could somebody > > > > look at this reboot loop problem? > > > > > > > > And any idea how should be OMAP I2C configured in U-Boot to correctly > > > > work? > > > > > > > > Maybe I will try to find some time to git bisect which change broke > > > > U-Boot on real N900 hardware. > > > > > > Took latest u-boot master, applied patches and this is the result on > > > serial (first part is NOLO booting, I think that can be ignored) [1]. > > > > ... > > > > > U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) > > > > > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz > > > Nokia RX-51 + LPDDR/OneNAND > > > I2C: ready > > > DRAM: 256 MiB > > > NAND: 0 Bytes > > > > Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
> > > > > MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 > > > In: vga > > > Out: vga > > > Err: vga > > > Timed out in wait_for_event: status=0100 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > Timed out in wait_for_event: status=0000 > > > Check if pads/pull-ups of bus are properly configured > > > i2c_read (addr phase): pads on bus probably not configured (status=0x10) > > > i2c_write: timed out writig last byte! > > > > These i2c errors are caused by > > > > /* reset lp5523 led */ > > i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
> > state = 0xff; > > i2c_write(0x32, 0x3d, 1, &state, 1); > > i2c_set_bus_num(0); > > > > Is there anything which needs to be done to initialize i2c bus 1? > > Because this code is working fine on older U-Boot version. > > Above code worked fine for U-Boot 2013.04, but in git version from > January 2015 it prints above error messages. > > On on internet forums I see these error messages also from other OMAP3 > board, e.g. beagle board. > > Has somebody some working OMAP3 board? And can test if it works with > recent version of U-Boot? I guess that above i2c problem would happen > also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
...
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier <oe5hpm@oevsv.at> Cc: Heiko Schocher <hs@denx.de>U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
udelay(I2C_WAIT);
udelay(adap->waitdelay);May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } } + /* + * poll ARDY bit for making sure that last byte really has been + * transferred on the bus. + */ + do { + status = wait_for_event(adap); + } while (!(status & I2C_STAT_ARDY) && timeout--); + if (timeout <= 0) + printf("i2c_write: timed out writig last byte!\n");
wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } }
- /* - * poll ARDY bit for making sure that last byte really has been - * transferred on the bus. - */ - do { - status = wait_for_event(i2c_base, ip_rev, waitdelay); - } while (!(status & I2C_STAT_ARDY) && timeout--); - if (timeout <= 0) - printf("i2c_write: timed out writig last byte!\n"); - wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
bye, Heiko -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-52 Fax: +49-8142-66989-80 Email: hs@denx.de
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote: > On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote: >> >> On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: >>> On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >>>> Hi, >>>> >>>> On 01/04/2020 00:42, Pali Rohár wrote: >>>>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>>>> This patch series contain fixes for Nokia RX-51 board (aka N900). >>>>>> After these changes it is possible to run U-Boot in qemu emulator again. >>>>>> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without >>>>>> problem. >>>>> >>>>> But on real Nokia N900 device is U-Boot crashing in reboot loop. >>>>> >>>>> I do not have serial console for Nokia N900 to debug this issue, but >>>>> seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is >>>>> that there is no crash and even no error in qemu emulator so I cannot >>>>> debug this issue. >>>>> >>>>> First problem is around /* reset lp5523 led */ code in rx51.c. On real >>>>> N900 device it generates repeating messages: >>>>> >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> >>>>> When I commented that few lines all these messages disappeared. So >>>>> problem is with OMAP I2C.
...
>>>>> I remember that somebody had serial jig for Nokia N900, could somebody >>>>> look at this reboot loop problem? >>>>> >>>>> And any idea how should be OMAP I2C configured in U-Boot to correctly >>>>> work? >>>>> >>>>> Maybe I will try to find some time to git bisect which change broke >>>>> U-Boot on real N900 hardware. >>>> >>>> Took latest u-boot master, applied patches and this is the result on >>>> serial (first part is NOLO booting, I think that can be ignored) [1]. >>> >>> ... >>> >>>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) >>>> >>>> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >>>> Nokia RX-51 + LPDDR/OneNAND >>>> I2C: ready >>>> DRAM: 256 MiB >>>> NAND: 0 Bytes >>> >>> Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
:-(
>>>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >>>> In: vga >>>> Out: vga >>>> Err: vga >>>> Timed out in wait_for_event: status=0100 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> Timed out in wait_for_event: status=0000 >>>> Check if pads/pull-ups of bus are properly configured >>>> i2c_read (addr phase): pads on bus probably not configured (status=0x10) >>>> i2c_write: timed out writig last byte! >>> >>> These i2c errors are caused by >>> >>> /* reset lp5523 led */ >>> i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
Yes, my first thought too!
This board needs a complete cleanup.
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
Ok.
>>> state = 0xff; >>> i2c_write(0x32, 0x3d, 1, &state, 1); >>> i2c_set_bus_num(0); >>> >>> Is there anything which needs to be done to initialize i2c bus 1? >>> Because this code is working fine on older U-Boot version. >> >> Above code worked fine for U-Boot 2013.04, but in git version from >> January 2015 it prints above error messages. >> >> On on internet forums I see these error messages also from other OMAP3 >> board, e.g. beagle board. >> >> Has somebody some working OMAP3 board? And can test if it works with >> recent version of U-Boot? I guess that above i2c problem would happen >> also on other OMAP3 boards. > > There was a conversion a while ago to dm_i2c, and I converted my board > to support using the device tree during the SPL phase, and whenever I > am aware any driver has driver model (DM) support, I try to convert my > board. > > I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
...
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier <oe5hpm@oevsv.at> Cc: Heiko Schocher <hs@denx.de>U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
udelay(I2C_WAIT);
udelay(adap->waitdelay);May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Ok, fine.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } }
/*
* poll ARDY bit for making sure that last byte really has been* transferred on the bus.*/do {
status = wait_for_event(adap);} while (!(status & I2C_STAT_ARDY) && timeout--);
if (timeout <= 0)
printf("i2c_write: timed out writig last byte!\n");wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
Ok, this part definetely changes behaviour in timing...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } }
- /*
* poll ARDY bit for making sure that last byte really has been* transferred on the bus.*/- do {
status = wait_for_event(i2c_base, ip_rev, waitdelay);- } while (!(status & I2C_STAT_ARDY) && timeout--);
- if (timeout <= 0)
printf("i2c_write: timed out writig last byte!\n");- wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
Hmm... so the question why does this bit not pop up on transfer end?
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
or reverse asked, why does this bit may not pop up for you Pali?
I have not yet time to check this... also I am unsure if I have a hw where I can try ...
bye, Heiko
On Wednesday 28 October 2020 06:42:39 Heiko Schocher wrote:
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote: > On Saturday 25 April 2020 06:36:58 Adam Ford wrote: > > On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote: > > > On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: > > > > On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: > > > > > MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 > > > > > In: vga > > > > > Out: vga > > > > > Err: vga > > > > > Timed out in wait_for_event: status=0100 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > Timed out in wait_for_event: status=0000 > > > > > Check if pads/pull-ups of bus are properly configured > > > > > i2c_read (addr phase): pads on bus probably not configured (status=0x10) > > > > > i2c_write: timed out writig last byte!
...
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier <oe5hpm@oevsv.at> Cc: Heiko Schocher <hs@denx.de>U-Boot version prior this command does not report those i2c errors.
...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } }
- /*
* poll ARDY bit for making sure that last byte really has been* transferred on the bus.*/- do {
status = wait_for_event(i2c_base, ip_rev, waitdelay);- } while (!(status & I2C_STAT_ARDY) && timeout--);
- if (timeout <= 0)
printf("i2c_write: timed out writig last byte!\n");- wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
Hmm... so the question why does this bit not pop up on transfer end?
I do not know. I even do not have documentation for this i2c bus IP. More suspicious is that this happens only for bus num 1.
Bus num 0 does not cause any issue and is working with and also without that patch.
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
Yes, it is for every bus num 1 transfer. But in N900 code there is just one write on i2c bus 1 (to reset LED). All other devices which are accessed from U-Boot on N900 are on i2c bus 0.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
or reverse asked, why does this bit may not pop up for you Pali?
Nokia N900 is Omap3430 HS device. I do not know how many people are using latest U-Boot on HS devices and maybe Omap3430 has some i2c bugs?
But my main question is, why kernel's i2c driver does not do that check for ARDY bit? And U-Boot is doing it?
I have not yet time to check this... also I am unsure if I have a hw where I can try ...
bye, Heiko -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-52 Fax: +49-8142-66989-80 Email: hs@denx.de
Hi,
On 28.10.20 г. 7:42 ч., Heiko Schocher wrote:
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote: > > On Saturday 25 April 2020 06:36:58 Adam Ford wrote: >> On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote: >>> >>> On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: >>>> On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >>>>> Hi, >>>>> >>>>> On 01/04/2020 00:42, Pali Rohár wrote: >>>>>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>>>>> This patch series contain fixes for Nokia RX-51 board (aka >>>>>>> N900). >>>>>>> After these changes it is possible to run U-Boot in qemu >>>>>>> emulator again. >>>>>>> And U-Boot can boot kernel image from RAM, eMMC or OneNAND >>>>>>> memory without >>>>>>> problem. >>>>>> >>>>>> But on real Nokia N900 device is U-Boot crashing in reboot >>>>>> loop. >>>>>> >>>>>> I do not have serial console for Nokia N900 to debug this >>>>>> issue, but >>>>>> seems that it is related to OMAP I2C and OMAP HS MMC code. >>>>>> Problem is >>>>>> that there is no crash and even no error in qemu emulator so >>>>>> I cannot >>>>>> debug this issue. >>>>>> >>>>>> First problem is around /* reset lp5523 led */ code in >>>>>> rx51.c. On real >>>>>> N900 device it generates repeating messages: >>>>>> >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> >>>>>> When I commented that few lines all these messages >>>>>> disappeared. So >>>>>> problem is with OMAP I2C.
...
>>>>>> I remember that somebody had serial jig for Nokia N900, >>>>>> could somebody >>>>>> look at this reboot loop problem? >>>>>> >>>>>> And any idea how should be OMAP I2C configured in U-Boot to >>>>>> correctly >>>>>> work? >>>>>> >>>>>> Maybe I will try to find some time to git bisect which >>>>>> change broke >>>>>> U-Boot on real N900 hardware. >>>>> >>>>> Took latest u-boot master, applied patches and this is the >>>>> result on >>>>> serial (first part is NOLO booting, I think that can be >>>>> ignored) [1]. >>>> >>>> ... >>>> >>>>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - >>>>> 12:15:47 +0200) >>>>> >>>>> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >>>>> Nokia RX-51 + LPDDR/OneNAND >>>>> I2C: ready >>>>> DRAM: 256 MiB >>>>> NAND: 0 Bytes >>>> >>>> Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
:-(
>>>>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >>>>> In: vga >>>>> Out: vga >>>>> Err: vga >>>>> Timed out in wait_for_event: status=0100 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> Timed out in wait_for_event: status=0000 >>>>> Check if pads/pull-ups of bus are properly configured >>>>> i2c_read (addr phase): pads on bus probably not configured >>>>> (status=0x10) >>>>> i2c_write: timed out writig last byte! >>>> >>>> These i2c errors are caused by >>>> >>>> /* reset lp5523 led */ >>>> i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
Yes, my first thought too!
This board needs a complete cleanup.
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
Ok.
>>>> state = 0xff; >>>> i2c_write(0x32, 0x3d, 1, &state, 1); >>>> i2c_set_bus_num(0); >>>> >>>> Is there anything which needs to be done to initialize i2c bus 1? >>>> Because this code is working fine on older U-Boot version. >>> >>> Above code worked fine for U-Boot 2013.04, but in git version from >>> January 2015 it prints above error messages. >>> >>> On on internet forums I see these error messages also from >>> other OMAP3 >>> board, e.g. beagle board. >>> >>> Has somebody some working OMAP3 board? And can test if it works >>> with >>> recent version of U-Boot? I guess that above i2c problem would >>> happen >>> also on other OMAP3 boards. >> >> There was a conversion a while ago to dm_i2c, and I converted my >> board >> to support using the device tree during the SPL phase, and >> whenever I >> am aware any driver has driver model (DM) support, I try to >> convert my >> board. >> >> I have a DM3730 and the last check I did was 2020.04-rc1, and it >> was working > > Ok, so it either OMAP3430 specific problem or N900 board specific > problem. N900 does not use driver model.
i have an OMAP3530 which is basically a 3430, and it works too. I am guessing the issue is unique to the N900 or the fact that it's high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
...
> Before calling i2c_write(0x32, ...) I tried to call > i2c_probe(0x32) and > it returned error. > > If I tried to call "i2c dev 1" in U-Boot console, I got tons of > errors > and basically U-Boot stopped responding. > > So by above observation it looks like I2C bus num 1 does not > work, but > I2C bus num 0 works fine. > > Do I need to call i2c_probe(...) before calling i2c_write(...)? > > And is something special needed for initializing omap i2c bus num 1? > Because from my above observation it looks like that something is > missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier oe5hpm@oevsv.at Cc: Heiko Schocher hs@denx.de
U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
- udelay(I2C_WAIT); + udelay(adap->waitdelay);
May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Ok, fine.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } } + /* + * poll ARDY bit for making sure that last byte really has been + * transferred on the bus. + */ + do { + status = wait_for_event(adap); + } while (!(status & I2C_STAT_ARDY) && timeout--); + if (timeout <= 0) + printf("i2c_write: timed out writig last byte!\n"); wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
Ok, this part definetely changes behaviour in timing...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } } - /* - * poll ARDY bit for making sure that last byte really has been - * transferred on the bus. - */ - do { - status = wait_for_event(i2c_base, ip_rev, waitdelay); - } while (!(status & I2C_STAT_ARDY) && timeout--); - if (timeout <= 0) - printf("i2c_write: timed out writig last byte!\n");
wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Correct, but this waiting seems to be described in the TRM (see Figure 18-46 and Figure 18-47), albeit for the polling mode. Though, in general flow diagrams (Figure 18-29 and Figure 18-31), there is no such loop.
However, by looking to __omap24_i2c_init(), it is not clear to me what mode uboot uses as it enables almost all interrupt bits in OMAP_I2C_IE_REG and loops waiting for events at the same time. Could someone confirm if uboot uses interrupt or polling mode? As this changes the things in regards to ARDY bit a lot, IIUC.
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
But it seems there is something weird about ARDY bit, at least in interrupt mode, see linux kernel commits cb527ede1b and 4cdbf7d346. So it seems ARDY bit shall be cleared twice.
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
What looks wrong to me is that __omap24_i2c_init() enables all interrupts in OMAP_I2C_IE_REG, however, the precondition for polling mode according to Figure 18-29 is that all interrupts shall be disabled.
Hmm... so the question why does this bit not pop up on transfer end?
It seems it never pops out. Moreover, if we look at the logs, the first wait_for_event() seems to timeout with status=0100, that is with BF bit set. What looks suspicions here, is that the only bit we ever see set, is the bit we don't have interrupt bit enabled for.
Pali, maybe you should try to comment the code that sets interrupt bits in __omap24_i2c_init() (the block that starts with "if (ip_rev == OMAP_I2C_REV_V1)") and see if it makes any difference. Also, maybe add more traces in __omap24_i2c_write() to see which exactly wait_for_event() call times out.
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
or reverse asked, why does this bit may not pop up for you Pali?
I have not yet time to check this... also I am unsure if I have a hw where I can try ...
bye, Heiko
Regards, Ivo
Hello Ivaylo,
Am 29.10.2020 um 08:51 schrieb Ivaylo Dimitrov:
Hi,
On 28.10.20 г. 7:42 ч., Heiko Schocher wrote:
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote:
On Saturday 25 April 2020 07:00:58 Adam Ford wrote: > On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote: >> >> On Saturday 25 April 2020 06:36:58 Adam Ford wrote: >>> On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote: >>>> >>>> On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: >>>>> On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >>>>>> Hi, >>>>>> >>>>>> On 01/04/2020 00:42, Pali Rohár wrote: >>>>>>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>>>>>> This patch series contain fixes for Nokia RX-51 board (aka N900). >>>>>>>> After these changes it is possible to run U-Boot in qemu emulator again. >>>>>>>> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without >>>>>>>> problem. >>>>>>> >>>>>>> But on real Nokia N900 device is U-Boot crashing in reboot loop. >>>>>>> >>>>>>> I do not have serial console for Nokia N900 to debug this issue, but >>>>>>> seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is >>>>>>> that there is no crash and even no error in qemu emulator so I cannot >>>>>>> debug this issue. >>>>>>> >>>>>>> First problem is around /* reset lp5523 led */ code in rx51.c. On real >>>>>>> N900 device it generates repeating messages: >>>>>>> >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> >>>>>>> When I commented that few lines all these messages disappeared. So >>>>>>> problem is with OMAP I2C.
...
>>>>>>> I remember that somebody had serial jig for Nokia N900, could somebody >>>>>>> look at this reboot loop problem? >>>>>>> >>>>>>> And any idea how should be OMAP I2C configured in U-Boot to correctly >>>>>>> work? >>>>>>> >>>>>>> Maybe I will try to find some time to git bisect which change broke >>>>>>> U-Boot on real N900 hardware. >>>>>> >>>>>> Took latest u-boot master, applied patches and this is the result on >>>>>> serial (first part is NOLO booting, I think that can be ignored) [1]. >>>>> >>>>> ... >>>>> >>>>>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) >>>>>> >>>>>> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >>>>>> Nokia RX-51 + LPDDR/OneNAND >>>>>> I2C: ready >>>>>> DRAM: 256 MiB >>>>>> NAND: 0 Bytes >>>>> >>>>> Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
:-(
>>>>>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >>>>>> In: vga >>>>>> Out: vga >>>>>> Err: vga >>>>>> Timed out in wait_for_event: status=0100 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> Timed out in wait_for_event: status=0000 >>>>>> Check if pads/pull-ups of bus are properly configured >>>>>> i2c_read (addr phase): pads on bus probably not configured (status=0x10) >>>>>> i2c_write: timed out writig last byte! >>>>> >>>>> These i2c errors are caused by >>>>> >>>>> /* reset lp5523 led */ >>>>> i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
Yes, my first thought too!
This board needs a complete cleanup.
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
Ok.
>>>>> state = 0xff; >>>>> i2c_write(0x32, 0x3d, 1, &state, 1); >>>>> i2c_set_bus_num(0); >>>>> >>>>> Is there anything which needs to be done to initialize i2c bus 1? >>>>> Because this code is working fine on older U-Boot version. >>>> >>>> Above code worked fine for U-Boot 2013.04, but in git version from >>>> January 2015 it prints above error messages. >>>> >>>> On on internet forums I see these error messages also from other OMAP3 >>>> board, e.g. beagle board. >>>> >>>> Has somebody some working OMAP3 board? And can test if it works with >>>> recent version of U-Boot? I guess that above i2c problem would happen >>>> also on other OMAP3 boards. >>> >>> There was a conversion a while ago to dm_i2c, and I converted my board >>> to support using the device tree during the SPL phase, and whenever I >>> am aware any driver has driver model (DM) support, I try to convert my >>> board. >>> >>> I have a DM3730 and the last check I did was 2020.04-rc1, and it was working >> >> Ok, so it either OMAP3430 specific problem or N900 board specific >> problem. N900 does not use driver model. > > i have an OMAP3530 which is basically a 3430, and it works too. I am > guessing the issue is unique to the N900 or the fact that it's > high-security. Neither of my boards are HS parts. They are both GP.
N900 is HS device, but I guess that should be caused by GP vs HS difference. Working i2c bus 0 and non-working i2c bus 1 could not be caused by GP vs HS difference. Also I guess that omap hs mmc would be same on GP and HS boards.
...
>> Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and >> it returned error. >> >> If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors >> and basically U-Boot stopped responding. >> >> So by above observation it looks like I2C bus num 1 does not work, but >> I2C bus num 0 works fine. >> >> Do I need to call i2c_probe(...) before calling i2c_write(...)? >> >> And is something special needed for initializing omap i2c bus num 1? >> Because from my above observation it looks like that something is >> missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier oe5hpm@oevsv.at Cc: Heiko Schocher hs@denx.de
U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
- udelay(I2C_WAIT); + udelay(adap->waitdelay);
May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Ok, fine.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } } + /* + * poll ARDY bit for making sure that last byte really has been + * transferred on the bus. + */ + do { + status = wait_for_event(adap); + } while (!(status & I2C_STAT_ARDY) && timeout--); + if (timeout <= 0) + printf("i2c_write: timed out writig last byte!\n"); wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
Ok, this part definetely changes behaviour in timing...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } } - /* - * poll ARDY bit for making sure that last byte really has been - * transferred on the bus. - */ - do { - status = wait_for_event(i2c_base, ip_rev, waitdelay); - } while (!(status & I2C_STAT_ARDY) && timeout--); - if (timeout <= 0) - printf("i2c_write: timed out writig last byte!\n");
wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Correct, but this waiting seems to be described in the TRM (see Figure 18-46 and Figure 18-47), albeit for the polling mode. Though, in general flow diagrams (Figure 18-29 and Figure 18-31), there is no such loop.
Could you provide a link to the TRM?
However, by looking to __omap24_i2c_init(), it is not clear to me what mode uboot uses as it enables almost all interrupt bits in OMAP_I2C_IE_REG and loops waiting for events at the same time. Could someone confirm if uboot uses interrupt or polling mode? As this changes the things in regards to ARDY bit a lot, IIUC.
I think, u-boot only polls the registers, while enabling all interrupt bits ...
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
But it seems there is something weird about ARDY bit, at least in interrupt mode, see linux kernel commits cb527ede1b and 4cdbf7d346. So it seems ARDY bit shall be cleared twice.
Hmm.. yes.
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
What looks wrong to me is that __omap24_i2c_init() enables all interrupts in OMAP_I2C_IE_REG, however, the precondition for polling mode according to Figure 18-29 is that all interrupts shall be disabled.
Hmm... so the question why does this bit not pop up on transfer end?
It seems it never pops out. Moreover, if we look at the logs, the first wait_for_event() seems to
yes.
timeout with status=0100, that is with BF bit set. What looks suspicions here, is that the only bit we ever see set, is the bit we don't have interrupt bit enabled for.
Pali, maybe you should try to comment the code that sets interrupt bits in __omap24_i2c_init() (the block that starts with "if (ip_rev == OMAP_I2C_REV_V1)") and see if it makes any difference. Also, maybe add more traces in __omap24_i2c_write() to see which exactly wait_for_event() call times out.
May worth a try.
More mystically is, that it works fine for Pali on bus 0 but makes problems on bus 1 ... !?
Pali: Do you have also on bus 0 i2c writes?
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
or reverse asked, why does this bit may not pop up for you Pali?
I have not yet time to check this... also I am unsure if I have a hw where I can try ...
bye, Heiko
Regards, Ivo
bye, Heiko
On Thursday 29 October 2020 10:32:04 Heiko Schocher wrote:
More mystically is, that it works fine for Pali on bus 0 but makes problems on bus 1 ... !?
Pali: Do you have also on bus 0 i2c writes?
Yes, there are many, search for "i2c_write" and see: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
And they works fine.
Everything is for bus 0 except that one problematic i2c write which is for bus 1.
Hi,
On 29.10.20 г. 11:32 ч., Heiko Schocher wrote:
Hello Ivaylo,
Am 29.10.2020 um 08:51 schrieb Ivaylo Dimitrov:
Hi,
On 28.10.20 г. 7:42 ч., Heiko Schocher wrote:
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár:
Adding Hannes and Heiko to the loop, please look at this problem.
On Saturday 25 April 2020 14:11:32 Pali Rohár wrote: > On Saturday 25 April 2020 07:00:58 Adam Ford wrote: >> On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote: >>> >>> On Saturday 25 April 2020 06:36:58 Adam Ford wrote: >>>> On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org >>>> wrote: >>>>> >>>>> On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: >>>>>> On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >>>>>>> Hi, >>>>>>> >>>>>>> On 01/04/2020 00:42, Pali Rohár wrote: >>>>>>>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>>>>>>> This patch series contain fixes for Nokia RX-51 board >>>>>>>>> (aka N900). >>>>>>>>> After these changes it is possible to run U-Boot in qemu >>>>>>>>> emulator again. >>>>>>>>> And U-Boot can boot kernel image from RAM, eMMC or >>>>>>>>> OneNAND memory without >>>>>>>>> problem. >>>>>>>> >>>>>>>> But on real Nokia N900 device is U-Boot crashing in reboot >>>>>>>> loop. >>>>>>>> >>>>>>>> I do not have serial console for Nokia N900 to debug this >>>>>>>> issue, but >>>>>>>> seems that it is related to OMAP I2C and OMAP HS MMC code. >>>>>>>> Problem is >>>>>>>> that there is no crash and even no error in qemu emulator >>>>>>>> so I cannot >>>>>>>> debug this issue. >>>>>>>> >>>>>>>> First problem is around /* reset lp5523 led */ code in >>>>>>>> rx51.c. On real >>>>>>>> N900 device it generates repeating messages: >>>>>>>> >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> >>>>>>>> When I commented that few lines all these messages >>>>>>>> disappeared. So >>>>>>>> problem is with OMAP I2C. ... >>>>>>>> I remember that somebody had serial jig for Nokia N900, >>>>>>>> could somebody >>>>>>>> look at this reboot loop problem? >>>>>>>> >>>>>>>> And any idea how should be OMAP I2C configured in U-Boot >>>>>>>> to correctly >>>>>>>> work? >>>>>>>> >>>>>>>> Maybe I will try to find some time to git bisect which >>>>>>>> change broke >>>>>>>> U-Boot on real N900 hardware. >>>>>>> >>>>>>> Took latest u-boot master, applied patches and this is the >>>>>>> result on >>>>>>> serial (first part is NOLO booting, I think that can be >>>>>>> ignored) [1]. >>>>>> >>>>>> ... >>>>>> >>>>>>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - >>>>>>> 12:15:47 +0200) >>>>>>> >>>>>>> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >>>>>>> Nokia RX-51 + LPDDR/OneNAND >>>>>>> I2C: ready >>>>>>> DRAM: 256 MiB >>>>>>> NAND: 0 Bytes >>>>>> >>>>>> Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
:-(
>>>>>>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >>>>>>> In: vga >>>>>>> Out: vga >>>>>>> Err: vga >>>>>>> Timed out in wait_for_event: status=0100 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> Timed out in wait_for_event: status=0000 >>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>> i2c_read (addr phase): pads on bus probably not configured >>>>>>> (status=0x10)
How is that trace even possible? I built and tested u-boot here on my devices and I see the same message, but unless I am becoming blind, i2c_read() is never called from within i2c_write(). This is really suspicious.
>>>>>>> i2c_write: timed out writig last byte! >>>>>> >>>>>> These i2c errors are caused by >>>>>> >>>>>> /* reset lp5523 led */ >>>>>> i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
Yes, my first thought too!
This board needs a complete cleanup.
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
Ok.
>>>>>> state = 0xff; >>>>>> i2c_write(0x32, 0x3d, 1, &state, 1); >>>>>> i2c_set_bus_num(0); >>>>>> >>>>>> Is there anything which needs to be done to initialize i2c >>>>>> bus 1? >>>>>> Because this code is working fine on older U-Boot version. >>>>> >>>>> Above code worked fine for U-Boot 2013.04, but in git version >>>>> from >>>>> January 2015 it prints above error messages. >>>>> >>>>> On on internet forums I see these error messages also from >>>>> other OMAP3 >>>>> board, e.g. beagle board. >>>>> >>>>> Has somebody some working OMAP3 board? And can test if it >>>>> works with >>>>> recent version of U-Boot? I guess that above i2c problem >>>>> would happen >>>>> also on other OMAP3 boards. >>>> >>>> There was a conversion a while ago to dm_i2c, and I converted >>>> my board >>>> to support using the device tree during the SPL phase, and >>>> whenever I >>>> am aware any driver has driver model (DM) support, I try to >>>> convert my >>>> board. >>>> >>>> I have a DM3730 and the last check I did was 2020.04-rc1, and >>>> it was working >>> >>> Ok, so it either OMAP3430 specific problem or N900 board specific >>> problem. N900 does not use driver model. >> >> i have an OMAP3530 which is basically a 3430, and it works too. >> I am >> guessing the issue is unique to the N900 or the fact that it's >> high-security. Neither of my boards are HS parts. They are >> both GP. > > N900 is HS device, but I guess that should be caused by GP vs HS > difference. Working i2c bus 0 and non-working i2c bus 1 could not be > caused by GP vs HS difference. Also I guess that omap hs mmc > would be > same on GP and HS boards. ... >>> Before calling i2c_write(0x32, ...) I tried to call >>> i2c_probe(0x32) and >>> it returned error. >>> >>> If I tried to call "i2c dev 1" in U-Boot console, I got tons of >>> errors >>> and basically U-Boot stopped responding. >>> >>> So by above observation it looks like I2C bus num 1 does not >>> work, but >>> I2C bus num 0 works fine. >>> >>> Do I need to call i2c_probe(...) before calling i2c_write(...)? >>> >>> And is something special needed for initializing omap i2c bus >>> num 1? >>> Because from my above observation it looks like that something is >>> missing for bus 1 which in older u-boot version was not needed.
Now I was able to find commit which is causing above i2c problems: "Check if pads/pull-ups of bus are properly configured"
It is d5243359e1afc957acd373dbbde1cf6c70ee5485:
OMAP24xx I2C: Add support for set-speed Adds support for set-speed on the OMAP24xx I2C Adapter. Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. Otherwise on a subsequent call the transfer of last byte from the predecessor is aborted and therefore lost. For exmaple when i2c_write(...) is followed by a i2c_setspeed(...) (which has to deactivate and activate master for changing psc,...). Minor cosmetical changes. Signed-off-by: Hannes Petermaier oe5hpm@oevsv.at Cc: Heiko Schocher hs@denx.de
U-Boot version prior this command does not report those i2c errors.
Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
- udelay(I2C_WAIT); + udelay(adap->waitdelay);
May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Ok, fine.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } } + /* + * poll ARDY bit for making sure that last byte really has been + * transferred on the bus. + */ + do { + status = wait_for_event(adap); + } while (!(status & I2C_STAT_ARDY) && timeout--); + if (timeout <= 0) + printf("i2c_write: timed out writig last byte!\n"); wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
Ok, this part definetely changes behaviour in timing...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } } - /* - * poll ARDY bit for making sure that last byte really has been - * transferred on the bus. - */ - do { - status = wait_for_event(i2c_base, ip_rev, waitdelay); - } while (!(status & I2C_STAT_ARDY) && timeout--); - if (timeout <= 0) - printf("i2c_write: timed out writig last byte!\n");
wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Correct, but this waiting seems to be described in the TRM (see Figure 18-46 and Figure 18-47), albeit for the polling mode. Though, in general flow diagrams (Figure 18-29 and Figure 18-31), there is no such loop.
Could you provide a link to the TRM?
https://www.ti.com/pdfs/wtbu/OMAP34xx_ES3.1x_PUBLIC_TRM_vZN.pdf
However, by looking to __omap24_i2c_init(), it is not clear to me what mode uboot uses as it enables almost all interrupt bits in OMAP_I2C_IE_REG and loops waiting for events at the same time. Could someone confirm if uboot uses interrupt or polling mode? As this changes the things in regards to ARDY bit a lot, IIUC.
I think, u-boot only polls the registers, while enabling all interrupt bits ...
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
But it seems there is something weird about ARDY bit, at least in interrupt mode, see linux kernel commits cb527ede1b and 4cdbf7d346. So it seems ARDY bit shall be cleared twice.
Hmm.. yes.
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
What looks wrong to me is that __omap24_i2c_init() enables all interrupts in OMAP_I2C_IE_REG, however, the precondition for polling mode according to Figure 18-29 is that all interrupts shall be disabled.
Hmm... so the question why does this bit not pop up on transfer end?
It seems it never pops out. Moreover, if we look at the logs, the first wait_for_event() seems to
yes.
timeout with status=0100, that is with BF bit set. What looks suspicions here, is that the only bit we ever see set, is the bit we don't have interrupt bit enabled for.
Pali, maybe you should try to comment the code that sets interrupt bits in __omap24_i2c_init() (the block that starts with "if (ip_rev == OMAP_I2C_REV_V1)") and see if it makes any difference. Also, maybe add more traces in __omap24_i2c_write() to see which exactly wait_for_event() call times out.
May worth a try.
More mystically is, that it works fine for Pali on bus 0 but makes problems on bus 1 ... !?
Pali: Do you have also on bus 0 i2c writes?
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
And this is what happens, we have that once, as we write only one byte to bus 1.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
I can confirm I see it on the 2 devices I tested here.
What is worse, is that writing on bus 1 does not fail every time. I increased I2C_TIMEOUT to 100000 (the value from the TRM) and it seems now after power-cycle, write succeeds almost every time, however, after a reset command from u-boot, it usually fails. And with that increased timeout,when it fails I see:
Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10)
message 5 times during the failing write.
How we end up there, is a mystery to me.
Regards, Ivo
Hello Ivaylo,
Am 30.10.2020 um 08:00 schrieb Ivaylo Dimitrov:
Hi,
On 29.10.20 г. 11:32 ч., Heiko Schocher wrote:
Hello Ivaylo,
Am 29.10.2020 um 08:51 schrieb Ivaylo Dimitrov:
Hi,
On 28.10.20 г. 7:42 ч., Heiko Schocher wrote:
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote:
Hello Pali,
Am 26.04.2020 um 01:54 schrieb Pali Rohár: > Adding Hannes and Heiko to the loop, please look at this problem. > > On Saturday 25 April 2020 14:11:32 Pali Rohár wrote: >> On Saturday 25 April 2020 07:00:58 Adam Ford wrote: >>> On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org wrote: >>>> >>>> On Saturday 25 April 2020 06:36:58 Adam Ford wrote: >>>>> On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote: >>>>>> >>>>>> On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: >>>>>>> On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 01/04/2020 00:42, Pali Rohár wrote: >>>>>>>>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>>>>>>>> This patch series contain fixes for Nokia RX-51 board (aka N900). >>>>>>>>>> After these changes it is possible to run U-Boot in qemu emulator again. >>>>>>>>>> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without >>>>>>>>>> problem. >>>>>>>>> >>>>>>>>> But on real Nokia N900 device is U-Boot crashing in reboot loop. >>>>>>>>> >>>>>>>>> I do not have serial console for Nokia N900 to debug this issue, but >>>>>>>>> seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is >>>>>>>>> that there is no crash and even no error in qemu emulator so I cannot >>>>>>>>> debug this issue. >>>>>>>>> >>>>>>>>> First problem is around /* reset lp5523 led */ code in rx51.c. On real >>>>>>>>> N900 device it generates repeating messages: >>>>>>>>> >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> >>>>>>>>> When I commented that few lines all these messages disappeared. So >>>>>>>>> problem is with OMAP I2C. > ... >>>>>>>>> I remember that somebody had serial jig for Nokia N900, could somebody >>>>>>>>> look at this reboot loop problem? >>>>>>>>> >>>>>>>>> And any idea how should be OMAP I2C configured in U-Boot to correctly >>>>>>>>> work? >>>>>>>>> >>>>>>>>> Maybe I will try to find some time to git bisect which change broke >>>>>>>>> U-Boot on real N900 hardware. >>>>>>>> >>>>>>>> Took latest u-boot master, applied patches and this is the result on >>>>>>>> serial (first part is NOLO booting, I think that can be ignored) [1]. >>>>>>> >>>>>>> ... >>>>>>> >>>>>>>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) >>>>>>>> >>>>>>>> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >>>>>>>> Nokia RX-51 + LPDDR/OneNAND >>>>>>>> I2C: ready >>>>>>>> DRAM: 256 MiB >>>>>>>> NAND: 0 Bytes >>>>>>> >>>>>>> Looks like that something with NAND is broken.
The board code in U-Boot is in a very old state... :-(
:-(
>>>>>>>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >>>>>>>> In: vga >>>>>>>> Out: vga >>>>>>>> Err: vga >>>>>>>> Timed out in wait_for_event: status=0100 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>> i2c_read (addr phase): pads on bus probably not configured (status=0x10)
How is that trace even possible? I built and tested u-boot here on my devices and I see the same message, but unless I am becoming blind, i2c_read() is never called from within i2c_write(). This is really suspicious.
Puh..
>>>>>>>> i2c_write: timed out writig last byte! >>>>>>> >>>>>>> These i2c errors are caused by >>>>>>> >>>>>>> /* reset lp5523 led */ >>>>>>> i2c_set_bus_num(1);
deprecated ... the board code needs cleanup ...
Yes, my first thought too!
This board needs a complete cleanup.
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
Ok.
>>>>>>> state = 0xff; >>>>>>> i2c_write(0x32, 0x3d, 1, &state, 1); >>>>>>> i2c_set_bus_num(0); >>>>>>> >>>>>>> Is there anything which needs to be done to initialize i2c bus 1? >>>>>>> Because this code is working fine on older U-Boot version. >>>>>> >>>>>> Above code worked fine for U-Boot 2013.04, but in git version from >>>>>> January 2015 it prints above error messages. >>>>>> >>>>>> On on internet forums I see these error messages also from other OMAP3 >>>>>> board, e.g. beagle board. >>>>>> >>>>>> Has somebody some working OMAP3 board? And can test if it works with >>>>>> recent version of U-Boot? I guess that above i2c problem would happen >>>>>> also on other OMAP3 boards. >>>>> >>>>> There was a conversion a while ago to dm_i2c, and I converted my board >>>>> to support using the device tree during the SPL phase, and whenever I >>>>> am aware any driver has driver model (DM) support, I try to convert my >>>>> board. >>>>> >>>>> I have a DM3730 and the last check I did was 2020.04-rc1, and it was working >>>> >>>> Ok, so it either OMAP3430 specific problem or N900 board specific >>>> problem. N900 does not use driver model. >>> >>> i have an OMAP3530 which is basically a 3430, and it works too. I am >>> guessing the issue is unique to the N900 or the fact that it's >>> high-security. Neither of my boards are HS parts. They are both GP. >> >> N900 is HS device, but I guess that should be caused by GP vs HS >> difference. Working i2c bus 0 and non-working i2c bus 1 could not be >> caused by GP vs HS difference. Also I guess that omap hs mmc would be >> same on GP and HS boards. > ... >>>> Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and >>>> it returned error. >>>> >>>> If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors >>>> and basically U-Boot stopped responding. >>>> >>>> So by above observation it looks like I2C bus num 1 does not work, but >>>> I2C bus num 0 works fine. >>>> >>>> Do I need to call i2c_probe(...) before calling i2c_write(...)? >>>> >>>> And is something special needed for initializing omap i2c bus num 1? >>>> Because from my above observation it looks like that something is >>>> missing for bus 1 which in older u-boot version was not needed. > > Now I was able to find commit which is causing above i2c problems: > "Check if pads/pull-ups of bus are properly configured" > > It is d5243359e1afc957acd373dbbde1cf6c70ee5485: > > OMAP24xx I2C: Add support for set-speed > Adds support for set-speed on the OMAP24xx I2C Adapter. > Changes to omap24_i2c_write(...) for polling ARDY Bit from IRQ-Status. > Otherwise on a subsequent call the transfer of last byte from the > predecessor is aborted and therefore lost. For exmaple when > i2c_write(...) is followed by a i2c_setspeed(...) (which has to > deactivate and activate master for changing psc,...). > Minor cosmetical changes. > Signed-off-by: Hannes Petermaier oe5hpm@oevsv.at > Cc: Heiko Schocher hs@denx.de > > U-Boot version prior this command does not report those i2c errors. > > Hannes, any idea how your patch could broke omap i2c i2c bus num 1 on > Nokia N900?
Hard to say here anything useful, as I do not have the hardware...
The above commit changes:
- udelay(I2C_WAIT); + udelay(adap->waitdelay);
May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Ok, fine.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } } + /* + * poll ARDY bit for making sure that last byte really has been + * transferred on the bus. + */ + do { + status = wait_for_event(adap); + } while (!(status & I2C_STAT_ARDY) && timeout--); + if (timeout <= 0) + printf("i2c_write: timed out writig last byte!\n"); wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
Ok, this part definetely changes behaviour in timing...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } } - /* - * poll ARDY bit for making sure that last byte really has been - * transferred on the bus. - */ - do { - status = wait_for_event(i2c_base, ip_rev, waitdelay); - } while (!(status & I2C_STAT_ARDY) && timeout--); - if (timeout <= 0) - printf("i2c_write: timed out writig last byte!\n");
wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Correct, but this waiting seems to be described in the TRM (see Figure 18-46 and Figure 18-47), albeit for the polling mode. Though, in general flow diagrams (Figure 18-29 and Figure 18-31), there is no such loop.
Could you provide a link to the TRM?
https://www.ti.com/pdfs/wtbu/OMAP34xx_ES3.1x_PUBLIC_TRM_vZN.pdf
thanks! Seems I was blind ...
However, by looking to __omap24_i2c_init(), it is not clear to me what mode uboot uses as it enables almost all interrupt bits in OMAP_I2C_IE_REG and loops waiting for events at the same time. Could someone confirm if uboot uses interrupt or polling mode? As this changes the things in regards to ARDY bit a lot, IIUC.
I think, u-boot only polls the registers, while enabling all interrupt bits ...
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
But it seems there is something weird about ARDY bit, at least in interrupt mode, see linux kernel commits cb527ede1b and 4cdbf7d346. So it seems ARDY bit shall be cleared twice.
Hmm.. yes.
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
What looks wrong to me is that __omap24_i2c_init() enables all interrupts in OMAP_I2C_IE_REG, however, the precondition for polling mode according to Figure 18-29 is that all interrupts shall be disabled.
Hmm... so the question why does this bit not pop up on transfer end?
It seems it never pops out. Moreover, if we look at the logs, the first wait_for_event() seems to
yes.
timeout with status=0100, that is with BF bit set. What looks suspicions here, is that the only bit we ever see set, is the bit we don't have interrupt bit enabled for.
Pali, maybe you should try to comment the code that sets interrupt bits in __omap24_i2c_init() (the block that starts with "if (ip_rev == OMAP_I2C_REV_V1)") and see if it makes any difference. Also, maybe add more traces in __omap24_i2c_write() to see which exactly wait_for_event() call times out.
May worth a try.
More mystically is, that it works fine for Pali on bus 0 but makes problems on bus 1 ... !?
Pali: Do you have also on bus 0 i2c writes?
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
And this is what happens, we have that once, as we write only one byte to bus 1.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
I can confirm I see it on the 2 devices I tested here.
What is worse, is that writing on bus 1 does not fail every time. I increased I2C_TIMEOUT to 100000 (the value from the TRM) and it seems now after power-cycle, write succeeds almost every time, however, after a reset command from u-boot, it usually fails. And with that increased timeout,when it fails I see:
Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10)
message 5 times during the failing write.
How we end up there, is a mystery to me.
Yes...
Ok, on page 2671 is described when this ARDY event is triggered, so if we wait for it, we must first check, if the prerequisite is met...
Hmm.. the ARDY bit is also for interrupt mode described, see page 2778 Figure 18-31 ... so I think it is correct to check it ... but I do not see, why we should wait for it in a loop and print an error if bit does not come up
But I have no time to dig into this ...
bye, Heiko
On 30.10.20 г. 9:24 ч., Heiko Schocher wrote:
Hello Ivaylo,
Am 30.10.2020 um 08:00 schrieb Ivaylo Dimitrov:
Hi,
On 29.10.20 г. 11:32 ч., Heiko Schocher wrote:
Hello Ivaylo,
Am 29.10.2020 um 08:51 schrieb Ivaylo Dimitrov:
Hi,
On 28.10.20 г. 7:42 ч., Heiko Schocher wrote:
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
On Monday 27 April 2020 09:03:13 Heiko Schocher wrote: > Hello Pali, > > Am 26.04.2020 um 01:54 schrieb Pali Rohár: >> Adding Hannes and Heiko to the loop, please look at this problem. >> >> On Saturday 25 April 2020 14:11:32 Pali Rohár wrote: >>> On Saturday 25 April 2020 07:00:58 Adam Ford wrote: >>>> On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár pali@kernel.org >>>> wrote: >>>>> >>>>> On Saturday 25 April 2020 06:36:58 Adam Ford wrote: >>>>>> On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org >>>>>> wrote: >>>>>>> >>>>>>> On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: >>>>>>>> On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 01/04/2020 00:42, Pali Rohár wrote: >>>>>>>>>> On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >>>>>>>>>>> This patch series contain fixes for Nokia RX-51 board >>>>>>>>>>> (aka N900). >>>>>>>>>>> After these changes it is possible to run U-Boot in >>>>>>>>>>> qemu emulator again. >>>>>>>>>>> And U-Boot can boot kernel image from RAM, eMMC or >>>>>>>>>>> OneNAND memory without >>>>>>>>>>> problem. >>>>>>>>>> >>>>>>>>>> But on real Nokia N900 device is U-Boot crashing in >>>>>>>>>> reboot loop. >>>>>>>>>> >>>>>>>>>> I do not have serial console for Nokia N900 to debug >>>>>>>>>> this issue, but >>>>>>>>>> seems that it is related to OMAP I2C and OMAP HS MMC >>>>>>>>>> code. Problem is >>>>>>>>>> that there is no crash and even no error in qemu >>>>>>>>>> emulator so I cannot >>>>>>>>>> debug this issue. >>>>>>>>>> >>>>>>>>>> First problem is around /* reset lp5523 led */ code in >>>>>>>>>> rx51.c. On real >>>>>>>>>> N900 device it generates repeating messages: >>>>>>>>>> >>>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>>> >>>>>>>>>> When I commented that few lines all these messages >>>>>>>>>> disappeared. So >>>>>>>>>> problem is with OMAP I2C. >> ... >>>>>>>>>> I remember that somebody had serial jig for Nokia N900, >>>>>>>>>> could somebody >>>>>>>>>> look at this reboot loop problem? >>>>>>>>>> >>>>>>>>>> And any idea how should be OMAP I2C configured in U-Boot >>>>>>>>>> to correctly >>>>>>>>>> work? >>>>>>>>>> >>>>>>>>>> Maybe I will try to find some time to git bisect which >>>>>>>>>> change broke >>>>>>>>>> U-Boot on real N900 hardware. >>>>>>>>> >>>>>>>>> Took latest u-boot master, applied patches and this is >>>>>>>>> the result on >>>>>>>>> serial (first part is NOLO booting, I think that can be >>>>>>>>> ignored) [1]. >>>>>>>> >>>>>>>> ... >>>>>>>> >>>>>>>>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - >>>>>>>>> 12:15:47 +0200) >>>>>>>>> >>>>>>>>> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 >>>>>>>>> MHz >>>>>>>>> Nokia RX-51 + LPDDR/OneNAND >>>>>>>>> I2C: ready >>>>>>>>> DRAM: 256 MiB >>>>>>>>> NAND: 0 Bytes >>>>>>>> >>>>>>>> Looks like that something with NAND is broken. > > The board code in U-Boot is in a very old state... :-(
:-(
>>>>>>>>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 >>>>>>>>> In: vga >>>>>>>>> Out: vga >>>>>>>>> Err: vga >>>>>>>>> Timed out in wait_for_event: status=0100 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> Timed out in wait_for_event: status=0000 >>>>>>>>> Check if pads/pull-ups of bus are properly configured >>>>>>>>> i2c_read (addr phase): pads on bus probably not >>>>>>>>> configured (status=0x10)
How is that trace even possible? I built and tested u-boot here on my devices and I see the same message, but unless I am becoming blind, i2c_read() is never called from within i2c_write(). This is really suspicious.
Puh..
It turned out it is WD that kicks in.
>>>>>>>>> i2c_write: timed out writig last byte! >>>>>>>> >>>>>>>> These i2c errors are caused by >>>>>>>> >>>>>>>> /* reset lp5523 led */ >>>>>>>> i2c_set_bus_num(1); > > deprecated ... the board code needs cleanup ...
Yes, my first thought too!
This board needs a complete cleanup.
I converted code to CONFIG_DM_I2C and nothing was changed, issue is still there...
Ok.
>>>>>>>> state = 0xff; >>>>>>>> i2c_write(0x32, 0x3d, 1, &state, 1); >>>>>>>> i2c_set_bus_num(0); >>>>>>>> >>>>>>>> Is there anything which needs to be done to initialize i2c >>>>>>>> bus 1? >>>>>>>> Because this code is working fine on older U-Boot version. >>>>>>> >>>>>>> Above code worked fine for U-Boot 2013.04, but in git >>>>>>> version from >>>>>>> January 2015 it prints above error messages. >>>>>>> >>>>>>> On on internet forums I see these error messages also from >>>>>>> other OMAP3 >>>>>>> board, e.g. beagle board. >>>>>>> >>>>>>> Has somebody some working OMAP3 board? And can test if it >>>>>>> works with >>>>>>> recent version of U-Boot? I guess that above i2c problem >>>>>>> would happen >>>>>>> also on other OMAP3 boards. >>>>>> >>>>>> There was a conversion a while ago to dm_i2c, and I >>>>>> converted my board >>>>>> to support using the device tree during the SPL phase, and >>>>>> whenever I >>>>>> am aware any driver has driver model (DM) support, I try to >>>>>> convert my >>>>>> board. >>>>>> >>>>>> I have a DM3730 and the last check I did was 2020.04-rc1, >>>>>> and it was working >>>>> >>>>> Ok, so it either OMAP3430 specific problem or N900 board >>>>> specific >>>>> problem. N900 does not use driver model. >>>> >>>> i have an OMAP3530 which is basically a 3430, and it works >>>> too. I am >>>> guessing the issue is unique to the N900 or the fact that it's >>>> high-security. Neither of my boards are HS parts. They are >>>> both GP. >>> >>> N900 is HS device, but I guess that should be caused by GP vs HS >>> difference. Working i2c bus 0 and non-working i2c bus 1 could >>> not be >>> caused by GP vs HS difference. Also I guess that omap hs mmc >>> would be >>> same on GP and HS boards. >> ... >>>>> Before calling i2c_write(0x32, ...) I tried to call >>>>> i2c_probe(0x32) and >>>>> it returned error. >>>>> >>>>> If I tried to call "i2c dev 1" in U-Boot console, I got tons >>>>> of errors >>>>> and basically U-Boot stopped responding. >>>>> >>>>> So by above observation it looks like I2C bus num 1 does not >>>>> work, but >>>>> I2C bus num 0 works fine. >>>>> >>>>> Do I need to call i2c_probe(...) before calling i2c_write(...)? >>>>> >>>>> And is something special needed for initializing omap i2c bus >>>>> num 1? >>>>> Because from my above observation it looks like that >>>>> something is >>>>> missing for bus 1 which in older u-boot version was not needed. >> >> Now I was able to find commit which is causing above i2c problems: >> "Check if pads/pull-ups of bus are properly configured" >> >> It is d5243359e1afc957acd373dbbde1cf6c70ee5485: >> >> OMAP24xx I2C: Add support for set-speed >> Adds support for set-speed on the OMAP24xx I2C Adapter. >> Changes to omap24_i2c_write(...) for polling ARDY Bit from >> IRQ-Status. >> Otherwise on a subsequent call the transfer of last byte >> from the >> predecessor is aborted and therefore lost. For exmaple when >> i2c_write(...) is followed by a i2c_setspeed(...) (which >> has to >> deactivate and activate master for changing psc,...). >> Minor cosmetical changes. >> Signed-off-by: Hannes Petermaier oe5hpm@oevsv.at >> Cc: Heiko Schocher hs@denx.de >> >> U-Boot version prior this command does not report those i2c errors. >> >> Hannes, any idea how your patch could broke omap i2c i2c bus num >> 1 on >> Nokia N900? > > Hard to say here anything useful, as I do not have the hardware... > > The above commit changes: > > - udelay(I2C_WAIT); > + udelay(adap->waitdelay); > > May you can check, if adap->waitdelay is the same as I2C_WAIT ?
Yes, it is the same value.
Ok, fine.
Anyway, I have deeply looked at that commit again and it just adds support for omap24_i2c_setspeed and into omap24_i2c_write adds following change:
@@ -464,6 +502,15 @@ static int omap24_i2c_write(struct i2c_adapter *adap, uchar chip, uint addr, goto wr_exit; } } + /* + * poll ARDY bit for making sure that last byte really has been + * transferred on the bus. + */ + do { + status = wait_for_event(adap); + } while (!(status & I2C_STAT_ARDY) && timeout--); + if (timeout <= 0) + printf("i2c_write: timed out writig last byte!\n"); wr_exit: flush_fifo(adap);
And this change is causing that non-functional i2c bus.
Ok, this part definetely changes behaviour in timing...
I applied revert of above change on top of the master u-boot branch and i2c bus num 1 (second) started working on N900 hw:
diff --git a/drivers/i2c/omap24xx_i2c.c b/drivers/i2c/omap24xx_i2c.c index 0af4e333c4..a49cf89712 100644 --- a/drivers/i2c/omap24xx_i2c.c +++ b/drivers/i2c/omap24xx_i2c.c @@ -820,16 +820,6 @@ static int __omap24_i2c_write(void __iomem *i2c_base, int ip_rev, int waitdelay, } } - /* - * poll ARDY bit for making sure that last byte really has been - * transferred on the bus. - */ - do { - status = wait_for_event(i2c_base, ip_rev, waitdelay); - } while (!(status & I2C_STAT_ARDY) && timeout--); - if (timeout <= 0) - printf("i2c_write: timed out writig last byte!\n");
wr_exit: flush_fifo(i2c_base, ip_rev); omap_i2c_write_reg(i2c_base, ip_rev, 0xFFFF, OMAP_I2C_STAT_REG);
I have looked into i2c-omap.c linux kernel driver and its transfer function does not have any such code for waiting ARDY bit.
Correct, but this waiting seems to be described in the TRM (see Figure 18-46 and Figure 18-47), albeit for the polling mode. Though, in general flow diagrams (Figure 18-29 and Figure 18-31), there is no such loop.
Could you provide a link to the TRM?
https://www.ti.com/pdfs/wtbu/OMAP34xx_ES3.1x_PUBLIC_TRM_vZN.pdf
thanks! Seems I was blind ...
However, by looking to __omap24_i2c_init(), it is not clear to me what mode uboot uses as it enables almost all interrupt bits in OMAP_I2C_IE_REG and loops waiting for events at the same time. Could someone confirm if uboot uses interrupt or polling mode? As this changes the things in regards to ARDY bit a lot, IIUC.
I think, u-boot only polls the registers, while enabling all interrupt bits ...
Ok...
Why it is there? I have not able to find any information and that comment is strange... it looks like it was incomplete/broken? workaround about other issue.
Hmm.. ARDY bit means: """ The current transaction is finished and the module registers can be accessed. """
But it seems there is something weird about ARDY bit, at least in interrupt mode, see linux kernel commits cb527ede1b and 4cdbf7d346. So it seems ARDY bit shall be cleared twice.
Hmm.. yes.
Seems not to bad to me to check this bit! ... but may missing on transaction start some prerequisite is missing for triggering this bit? And so, this additional check only leads in a loop going into timeout?
As you can see in log, at the first call status flags contains value 0x0100 and on all other calls it contains just 0x000 status flags.
Therefore ARDY bit is never set, but i2c transfer works fine.
What looks wrong to me is that __omap24_i2c_init() enables all interrupts in OMAP_I2C_IE_REG, however, the precondition for polling mode according to Figure 18-29 is that all interrupts shall be disabled.
Hmm... so the question why does this bit not pop up on transfer end?
It seems it never pops out. Moreover, if we look at the logs, the first wait_for_event() seems to
yes.
timeout with status=0100, that is with BF bit set. What looks suspicions here, is that the only bit we ever see set, is the bit we don't have interrupt bit enabled for.
Pali, maybe you should try to comment the code that sets interrupt bits in __omap24_i2c_init() (the block that starts with "if (ip_rev == OMAP_I2C_REV_V1)") and see if it makes any difference. Also, maybe add more traces in __omap24_i2c_write() to see which exactly wait_for_event() call times out.
May worth a try.
More mystically is, that it works fine for Pali on bus 0 but makes problems on bus 1 ... !?
Pali: Do you have also on bus 0 i2c writes?
I just can speculate that adding this polling ARDY bit loop changes the timing... and fixed an underlying bug, yes...
but if this bit never pop up, there must come the printf "i2c_write: timed out writig last byte!" for each i2c transfer.
And this is what happens, we have that once, as we write only one byte to bus 1.
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
I can confirm I see it on the 2 devices I tested here.
What is worse, is that writing on bus 1 does not fail every time. I increased I2C_TIMEOUT to 100000 (the value from the TRM) and it seems now after power-cycle, write succeeds almost every time, however, after a reset command from u-boot, it usually fails. And with that increased timeout,when it fails I see:
Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10)
message 5 times during the failing write.
How we end up there, is a mystery to me.
Yes...
Ok, on page 2671 is described when this ARDY event is triggered, so if we wait for it, we must first check, if the prerequisite is met...
Hmm.. the ARDY bit is also for interrupt mode described, see page 2778 Figure 18-31 ... so I think it is correct to check it ... but I do not see, why we should wait for it in a loop and print an error if bit does not come up
But I have no time to dig into this ...
Looks like slave device is misbehaving after a reset command. We changed the write to not reset the slave, but instead to disable the LED engine and it seems there are no more timeouts/errors. However, I guess it still makes sense some more complicated logic to be implemented there (like, if we write only one byte, do not wait for ARDY), as I doubt lp5523 is the only device that misbehaves on reset.
Ivo
Hello Ivaylo,
Am 31.10.2020 um 12:47 schrieb Ivaylo Dimitrov:
On 30.10.20 г. 9:24 ч., Heiko Schocher wrote:
Hello Ivaylo,
Am 30.10.2020 um 08:00 schrieb Ivaylo Dimitrov:
Hi,
On 29.10.20 г. 11:32 ч., Heiko Schocher wrote:
Hello Ivaylo,
Am 29.10.2020 um 08:51 schrieb Ivaylo Dimitrov:
Hi,
On 28.10.20 г. 7:42 ч., Heiko Schocher wrote:
Hello Pali,
sorry for late response ...
Am 26.10.2020 um 22:48 schrieb Pali Rohár:
[snip]
Hannes may you can check if this is the case for you?
why does nobody claimed that this message pops up in the last 5 years?
I can confirm I see it on the 2 devices I tested here.
What is worse, is that writing on bus 1 does not fail every time. I increased I2C_TIMEOUT to 100000 (the value from the TRM) and it seems now after power-cycle, write succeeds almost every time, however, after a reset command from u-boot, it usually fails. And with that increased timeout,when it fails I see:
Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10)
message 5 times during the failing write.
How we end up there, is a mystery to me.
Yes...
Ok, on page 2671 is described when this ARDY event is triggered, so if we wait for it, we must first check, if the prerequisite is met...
Hmm.. the ARDY bit is also for interrupt mode described, see page 2778 Figure 18-31 ... so I think it is correct to check it ... but I do not see, why we should wait for it in a loop and print an error if bit does not come up
But I have no time to dig into this ...
Looks like slave device is misbehaving after a reset command. We changed the write to not reset the slave, but instead to disable the LED engine and it seems there are no more timeouts/errors. However, I guess it still makes sense some more complicated logic to be implemented there (like, if we write only one byte, do not wait for ARDY), as I doubt lp5523 is the only device that misbehaves on reset.
Uff... Ok, I cannot do/help here, as lack of time (and may hardware).
Hmm... may you try unblocking sequence instead?
Do you have time to look here deeper?
bye, Heiko
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote:
On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: > This patch series contain fixes for Nokia RX-51 board (aka N900). > After these changes it is possible to run U-Boot in qemu emulator again. > And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without > problem.
But on real Nokia N900 device is U-Boot crashing in reboot loop.
I do not have serial console for Nokia N900 to debug this issue, but seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is that there is no crash and even no error in qemu emulator so I cannot debug this issue.
First problem is around /* reset lp5523 led */ code in rx51.c. On real N900 device it generates repeating messages:
Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000
When I commented that few lines all these messages disappeared. So problem is with OMAP I2C.
Second problem happen after misc_init_r() function finishes. U-Boot just prints on N900 screen message "data abort" and reboots. As I do not have serial console it is hard to debug. but I figured out that problem is in mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put debug info prior to mmc_set_ios() call and after it, and debug info prior was printed, not after.
I remember that somebody had serial jig for Nokia N900, could somebody look at this reboot loop problem?
And any idea how should be OMAP I2C configured in U-Boot to correctly work?
Maybe I will try to find some time to git bisect which change broke U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
I did one mistake here. i2c_probe() returns non-zero value for error. In qemu it returns non-zero and on real HW it returns zero.
So i2c_probe() passes on real HW and then following i2c_write() cause above "Check if pads/pull-ups of bus are properly configured" error message. In qemu it looks like i2c writes are just discarded (maybe qemu does not emulate this i2c device).
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
On Saturday 25 April 2020 14:07:31 Pali Rohár wrote:
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
Hi,
On 01/04/2020 00:42, Pali Rohár wrote: > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: >> This patch series contain fixes for Nokia RX-51 board (aka N900). >> After these changes it is possible to run U-Boot in qemu emulator again. >> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without >> problem. > > But on real Nokia N900 device is U-Boot crashing in reboot loop. > > I do not have serial console for Nokia N900 to debug this issue, but > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is > that there is no crash and even no error in qemu emulator so I cannot > debug this issue. > > First problem is around /* reset lp5523 led */ code in rx51.c. On real > N900 device it generates repeating messages: > > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > > When I commented that few lines all these messages disappeared. So > problem is with OMAP I2C. > > Second problem happen after misc_init_r() function finishes. U-Boot just > prints on N900 screen message "data abort" and reboots. As I do not have > serial console it is hard to debug. but I figured out that problem is in > mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put > debug info prior to mmc_set_ios() call and after it, and debug info > prior was printed, not after. > > I remember that somebody had serial jig for Nokia N900, could somebody > look at this reboot loop problem? > > And any idea how should be OMAP I2C configured in U-Boot to correctly > work? > > Maybe I will try to find some time to git bisect which change broke > U-Boot on real N900 hardware.
Took latest u-boot master, applied patches and this is the result on serial (first part is NOLO booting, I think that can be ignored) [1].
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
Looks like that something with NAND is broken.
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 In: vga Out: vga Err: vga Timed out in wait_for_event: status=0100 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured Timed out in wait_for_event: status=0000 Check if pads/pull-ups of bus are properly configured i2c_read (addr phase): pads on bus probably not configured (status=0x10) i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
I did one mistake here. i2c_probe() returns non-zero value for error. In qemu it returns non-zero and on real HW it returns zero.
So i2c_probe() passes on real HW and then following i2c_write() cause above "Check if pads/pull-ups of bus are properly configured" error message. In qemu it looks like i2c writes are just discarded (maybe qemu does not emulate this i2c device).
I see that in past somebody had same problem on omap3 beagle board: https://lists.denx.de/pipermail/u-boot/2013-November/167969.html
I tried to add code from above email
struct control_prog_io *prog_io_base; prog_io_base = (struct control_prog_io *)OMAP34XX_CTRL_BASE; /* Enable i2c2 pullup resisters */ writel(~(PRG_I2C2_PULLUPRESX | PRG_I2C1_PULLUPRESX), &prog_io_base->io1);
but did not helped.
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
On Saturday 25 April 2020 15:19:12 Pali Rohár wrote:
On Saturday 25 April 2020 14:07:31 Pali Rohár wrote:
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote: > Hi, > > On 01/04/2020 00:42, Pali Rohár wrote: > > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote: > >> This patch series contain fixes for Nokia RX-51 board (aka N900). > >> After these changes it is possible to run U-Boot in qemu emulator again. > >> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without > >> problem. > > > > But on real Nokia N900 device is U-Boot crashing in reboot loop. > > > > I do not have serial console for Nokia N900 to debug this issue, but > > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is > > that there is no crash and even no error in qemu emulator so I cannot > > debug this issue. > > > > First problem is around /* reset lp5523 led */ code in rx51.c. On real > > N900 device it generates repeating messages: > > > > Check if pads/pull-ups of bus are properly configured > > Timed out in wait_for_event: status=0000 > > > > When I commented that few lines all these messages disappeared. So > > problem is with OMAP I2C. > > > > Second problem happen after misc_init_r() function finishes. U-Boot just > > prints on N900 screen message "data abort" and reboots. As I do not have > > serial console it is hard to debug. but I figured out that problem is in > > mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put > > debug info prior to mmc_set_ios() call and after it, and debug info > > prior was printed, not after. > > > > I remember that somebody had serial jig for Nokia N900, could somebody > > look at this reboot loop problem? > > > > And any idea how should be OMAP I2C configured in U-Boot to correctly > > work? > > > > Maybe I will try to find some time to git bisect which change broke > > U-Boot on real N900 hardware. > > Took latest u-boot master, applied patches and this is the result on > serial (first part is NOLO booting, I think that can be ignored) [1].
...
> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz > Nokia RX-51 + LPDDR/OneNAND > I2C: ready > DRAM: 256 MiB > NAND: 0 Bytes
Looks like that something with NAND is broken.
> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1 > In: vga > Out: vga > Err: vga > Timed out in wait_for_event: status=0100 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > Timed out in wait_for_event: status=0000 > Check if pads/pull-ups of bus are properly configured > i2c_read (addr phase): pads on bus probably not configured (status=0x10) > i2c_write: timed out writig last byte!
These i2c errors are caused by
/* reset lp5523 led */ i2c_set_bus_num(1); state = 0xff; i2c_write(0x32, 0x3d, 1, &state, 1); i2c_set_bus_num(0);Is there anything which needs to be done to initialize i2c bus 1? Because this code is working fine on older U-Boot version.
Above code worked fine for U-Boot 2013.04, but in git version from January 2015 it prints above error messages.
On on internet forums I see these error messages also from other OMAP3 board, e.g. beagle board.
Has somebody some working OMAP3 board? And can test if it works with recent version of U-Boot? I guess that above i2c problem would happen also on other OMAP3 boards.
There was a conversion a while ago to dm_i2c, and I converted my board to support using the device tree during the SPL phase, and whenever I am aware any driver has driver model (DM) support, I try to convert my board.
I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
Ok, so it either OMAP3430 specific problem or N900 board specific problem. N900 does not use driver model.
Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and it returned error.
If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors and basically U-Boot stopped responding.
So by above observation it looks like I2C bus num 1 does not work, but I2C bus num 0 works fine.
Do I need to call i2c_probe(...) before calling i2c_write(...)?
And is something special needed for initializing omap i2c bus num 1? Because from my above observation it looks like that something is missing for bus 1 which in older u-boot version was not needed.
I did one mistake here. i2c_probe() returns non-zero value for error. In qemu it returns non-zero and on real HW it returns zero.
So i2c_probe() passes on real HW and then following i2c_write() cause above "Check if pads/pull-ups of bus are properly configured" error message. In qemu it looks like i2c writes are just discarded (maybe qemu does not emulate this i2c device).
I see that in past somebody had same problem on omap3 beagle board: https://lists.denx.de/pipermail/u-boot/2013-November/167969.html
I tried to add code from above email
struct control_prog_io *prog_io_base; prog_io_base = (struct control_prog_io *)OMAP34XX_CTRL_BASE; /* Enable i2c2 pullup resisters */ writel(~(PRG_I2C2_PULLUPRESX | PRG_I2C1_PULLUPRESX), &prog_io_base->io1);
but did not helped.
Now I figured out that function set_muxconf_regs() which setup board pinmux configuration is not called.
Was something in U-Boot changed which caused that U-Boot does not call board set_muxconf_regs() function anymore?
U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600) Trying to boot from MMC1 spl_load_image_fat_os: error reading image args, err - -2
U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit Logic DM37x/OMAP35x reference board + LPDDR/NAND DRAM: 256 MiB NAND: 512 MiB MMC: OMAP SD/MMC: 0 Loading Environment from NAND... OK OMAP die ID: 619e00029ff800000168300f1502501f Net: eth0: ethernet@08000000 Hit any key to stop autoboot: 2
Was something changed to OMAP i2c bus code in U-Boot?
> OMAP die ID: 031400240000000004036ac10b01100f > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Adding Jean to the loop. Could you please look at this problem? Your commit (described below) is causing reboot loop on Nokia N900 hardware.
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
...
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
...
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off
If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization.
Signed-off-by: Jean-Jacques Hiblot jjhiblot@ti.com
:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Adding Jean to the loop. Could you please look at this problem? Your commit (described below) is causing reboot loop on Nokia N900 hardware.
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
...
U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz Nokia RX-51 + LPDDR/OneNAND I2C: ready DRAM: 256 MiB NAND: 0 Bytes
...
MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
...
OMAP die ID: 031400240000000004036ac10b01100f OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc); - if (err) { - /* - * if power cycling is not supported, we should not try - * to use the UHS modes, because we wouldn't be able to - * recover from an error during the UHS initialization. - */ - pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n"); - uhs_en = false; - mmc->host_caps &= ~UHS_CAPS; - err = mmc_power_on(mmc); - } + err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
Adding also Faiz to loop...
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Adding Jean to the loop. Could you please look at this problem? Your commit (described below) is causing reboot loop on Nokia N900 hardware.
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
...
> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz > Nokia RX-51 + LPDDR/OneNAND > I2C: ready > DRAM: 256 MiB > NAND: 0 Bytes
...
> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
...
> OMAP die ID: 031400240000000004036ac10b01100f > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
And second bisect finished. Result is:
d2c05f50e12f87128597a28146de7092aaa847c3 is the first bad commit commit d2c05f50e12f87128597a28146de7092aaa847c3 Author: Faiz Abbas faiz_abbas@ti.com Date: Fri Apr 5 14:18:46 2019 +0530
mmc: omap_hsmmc: Set 3.3V for IO voltage
Pbias voltage should match the IO voltage set for the SD card. With the latest pbias change to 3.3V, update the capabilities and IO voltages settings to 3.3V.
Signed-off-by: Faiz Abbas faiz_abbas@ti.com
:040000 040000 819148217b64a6beaf8247464de6b4384d4581a4 e4fd9288ddb794f33596339813d5386f3bed8fd7 M drivers
I revered above commit on top of master and eMMC on Nokia N900 finally started working again!
Faiz, what is the reason for above commit? It basically changes in omap hs mmc driver IO voltage from 3.0V to 3.3V. And seems this change is incompatible with Nokia N900. Are there any problems with 3.0V on some omap3 boards? If not I would vote for revering that commit. Or at least making IO voltage configurable.
Pali,
On 26/04/20 3:59 am, Pali Rohár wrote:
Adding also Faiz to loop...
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Adding Jean to the loop. Could you please look at this problem? Your commit (described below) is causing reboot loop on Nokia N900 hardware.
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote: > On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
...
>> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) >> >> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >> Nokia RX-51 + LPDDR/OneNAND >> I2C: ready >> DRAM: 256 MiB >> NAND: 0 Bytes
...
>> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
...
>> OMAP die ID: 031400240000000004036ac10b01100f >> OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >> > > And then U-Boot freeze, right? > > Any idea how to debug this issue? > > On my N900 I'm getting "data abort" error on display and then instant > reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
And second bisect finished. Result is:
d2c05f50e12f87128597a28146de7092aaa847c3 is the first bad commit commit d2c05f50e12f87128597a28146de7092aaa847c3 Author: Faiz Abbas faiz_abbas@ti.com Date: Fri Apr 5 14:18:46 2019 +0530
mmc: omap_hsmmc: Set 3.3V for IO voltage Pbias voltage should match the IO voltage set for the SD card. With the latest pbias change to 3.3V, update the capabilities and IO voltages settings to 3.3V. Signed-off-by: Faiz Abbas <faiz_abbas@ti.com>:040000 040000 819148217b64a6beaf8247464de6b4384d4581a4 e4fd9288ddb794f33596339813d5386f3bed8fd7 M drivers
I revered above commit on top of master and eMMC on Nokia N900 finally started working again!
Faiz, what is the reason for above commit? It basically changes in omap hs mmc driver IO voltage from 3.0V to 3.3V. And seems this change is incompatible with Nokia N900. Are there any problems with 3.0V on some omap3 boards? If not I would vote for revering that commit. Or at least making IO voltage configurable.
For the power_cycle patch, it looks like there is a null pointer dereference that might be causing the reboot loops (probably because your platform doesn't have DM_MMC enabled) Can you add a few more prints to see where the data abort comes from exactly?
For the second patch IO voltage patch, what exactly do you see without reverting it? Are you able to reach prompt but mmc commands fail? Its possible that your platform requires a pbias of 3.0V to work.
PS: In the next version of these patches, please follow file prefixes for each commit instead of one blanket NOKIA RX-511: prefix.
Thanks, Faiz
On Thursday 07 May 2020 19:10:14 Faiz Abbas wrote:
On 26/04/20 3:59 am, Pali Rohár wrote:
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
And second bisect finished. Result is:
d2c05f50e12f87128597a28146de7092aaa847c3 is the first bad commit commit d2c05f50e12f87128597a28146de7092aaa847c3 Author: Faiz Abbas faiz_abbas@ti.com Date: Fri Apr 5 14:18:46 2019 +0530
mmc: omap_hsmmc: Set 3.3V for IO voltage Pbias voltage should match the IO voltage set for the SD card. With the latest pbias change to 3.3V, update the capabilities and IO voltages settings to 3.3V. Signed-off-by: Faiz Abbas <faiz_abbas@ti.com>:040000 040000 819148217b64a6beaf8247464de6b4384d4581a4 e4fd9288ddb794f33596339813d5386f3bed8fd7 M drivers
I revered above commit on top of master and eMMC on Nokia N900 finally started working again!
Faiz, what is the reason for above commit? It basically changes in omap hs mmc driver IO voltage from 3.0V to 3.3V. And seems this change is incompatible with Nokia N900. Are there any problems with 3.0V on some omap3 boards? If not I would vote for revering that commit. Or at least making IO voltage configurable.
For the power_cycle patch, it looks like there is a null pointer dereference that might be causing the reboot loops (probably because your platform doesn't have DM_MMC enabled) Can you add a few more prints to see where the data abort comes from exactly?
Hello Faiz!
Sorry, but this is problematic. Because of reboot loops I cannot read what exactly is put on the display and reboot cause clearing display.
Month ago I was able to detect that problem is somewhere in function mmc_set_ios() from mmc.c file. I put long debug line prior and also after mmc_set_ios() call. And it was printed only prior. Not after. So I think NULL pointer dereference is in mmc_set_ios() function.
For the second patch IO voltage patch, what exactly do you see without reverting it? Are you able to reach prompt but mmc commands fail?
Yes, after reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 there is no reboot loop anymore. Just mmc commands fail and eMMC is not detected at all.
Its possible that your platform requires a pbias of 3.0V to work.
PS: In the next version of these patches, please follow file prefixes for each commit instead of one blanket NOKIA RX-511: prefix.
File prefix is nokia rx-51. So which prefix would you want to see? But I think I would not send new version of these patches as Tom already wrote that is fine with them and I guess he would take them (or maybe already merged?).
On Thursday 07 May 2020 17:19:38 Pali Rohár wrote:
On Thursday 07 May 2020 19:10:14 Faiz Abbas wrote:
On 26/04/20 3:59 am, Pali Rohár wrote:
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
And second bisect finished. Result is:
d2c05f50e12f87128597a28146de7092aaa847c3 is the first bad commit commit d2c05f50e12f87128597a28146de7092aaa847c3 Author: Faiz Abbas faiz_abbas@ti.com Date: Fri Apr 5 14:18:46 2019 +0530
mmc: omap_hsmmc: Set 3.3V for IO voltage Pbias voltage should match the IO voltage set for the SD card. With the latest pbias change to 3.3V, update the capabilities and IO voltages settings to 3.3V. Signed-off-by: Faiz Abbas <faiz_abbas@ti.com>:040000 040000 819148217b64a6beaf8247464de6b4384d4581a4 e4fd9288ddb794f33596339813d5386f3bed8fd7 M drivers
I revered above commit on top of master and eMMC on Nokia N900 finally started working again!
Faiz, what is the reason for above commit? It basically changes in omap hs mmc driver IO voltage from 3.0V to 3.3V. And seems this change is incompatible with Nokia N900. Are there any problems with 3.0V on some omap3 boards? If not I would vote for revering that commit. Or at least making IO voltage configurable.
For the power_cycle patch, it looks like there is a null pointer dereference that might be causing the reboot loops (probably because your platform doesn't have DM_MMC enabled) Can you add a few more prints to see where the data abort comes from exactly?
Hello Faiz!
Sorry, but this is problematic. Because of reboot loops I cannot read what exactly is put on the display and reboot cause clearing display.
Month ago I was able to detect that problem is somewhere in function mmc_set_ios() from mmc.c file. I put long debug line prior and also after mmc_set_ios() call. And it was printed only prior. Not after. So I think NULL pointer dereference is in mmc_set_ios() function.
For the second patch IO voltage patch, what exactly do you see without reverting it? Are you able to reach prompt but mmc commands fail?
Yes, after reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 there is no reboot loop anymore. Just mmc commands fail and eMMC is not detected at all.
Its possible that your platform requires a pbias of 3.0V to work.
Hello Faiz!
Now I figured out what is the root cause of second 3.0V vs 3.3V problem.
In commit d2c05f50e12f87128597a28146de7092aaa847c3 you forgot to replace one usage of 3.0V by 3.3V. Below is patch which changes also this last one usage. Applying it has same effect on Nokia N900 as reverting that problematic commit d2c05f50e12f87128597a28146de7092aaa847c3:
diff --git a/drivers/mmc/omap_hsmmc.c b/drivers/mmc/omap_hsmmc.c index 8636cd713a..dc26e54795 100644 --- a/drivers/mmc/omap_hsmmc.c +++ b/drivers/mmc/omap_hsmmc.c @@ -840,7 +840,7 @@ static int omap_hsmmc_init_setup(struct mmc *mmc) omap_hsmmc_conf_bus_power(mmc, (reg_val & VS33_3V3SUP) ? MMC_SIGNAL_VOLTAGE_330 : MMC_SIGNAL_VOLTAGE_180); #else - writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V0, &mmc_base->hctl); + writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V3, &mmc_base->hctl); writel(readl(&mmc_base->capa) | VS33_3V3SUP | VS18_1V8SUP, &mmc_base->capa); #endif
So eMMC on real N900 is working fine either with 3.0V or 3.3V. But 3.3V needs to be configured on all places.
First problem with reboot loop and crash in mmc_set_ios() still remains. Reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 fixes it.
On Tuesday 26 May 2020 19:49:54 Pali Rohár wrote:
On Thursday 07 May 2020 17:19:38 Pali Rohár wrote:
On Thursday 07 May 2020 19:10:14 Faiz Abbas wrote:
On 26/04/20 3:59 am, Pali Rohár wrote:
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
And second bisect finished. Result is:
d2c05f50e12f87128597a28146de7092aaa847c3 is the first bad commit commit d2c05f50e12f87128597a28146de7092aaa847c3 Author: Faiz Abbas faiz_abbas@ti.com Date: Fri Apr 5 14:18:46 2019 +0530
mmc: omap_hsmmc: Set 3.3V for IO voltage Pbias voltage should match the IO voltage set for the SD card. With the latest pbias change to 3.3V, update the capabilities and IO voltages settings to 3.3V. Signed-off-by: Faiz Abbas <faiz_abbas@ti.com>:040000 040000 819148217b64a6beaf8247464de6b4384d4581a4 e4fd9288ddb794f33596339813d5386f3bed8fd7 M drivers
I revered above commit on top of master and eMMC on Nokia N900 finally started working again!
Faiz, what is the reason for above commit? It basically changes in omap hs mmc driver IO voltage from 3.0V to 3.3V. And seems this change is incompatible with Nokia N900. Are there any problems with 3.0V on some omap3 boards? If not I would vote for revering that commit. Or at least making IO voltage configurable.
For the power_cycle patch, it looks like there is a null pointer dereference that might be causing the reboot loops (probably because your platform doesn't have DM_MMC enabled) Can you add a few more prints to see where the data abort comes from exactly?
Hello Faiz!
Sorry, but this is problematic. Because of reboot loops I cannot read what exactly is put on the display and reboot cause clearing display.
Month ago I was able to detect that problem is somewhere in function mmc_set_ios() from mmc.c file. I put long debug line prior and also after mmc_set_ios() call. And it was printed only prior. Not after. So I think NULL pointer dereference is in mmc_set_ios() function.
For the second patch IO voltage patch, what exactly do you see without reverting it? Are you able to reach prompt but mmc commands fail?
Yes, after reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 there is no reboot loop anymore. Just mmc commands fail and eMMC is not detected at all.
Its possible that your platform requires a pbias of 3.0V to work.
Hello Faiz!
Now I figured out what is the root cause of second 3.0V vs 3.3V problem.
In commit d2c05f50e12f87128597a28146de7092aaa847c3 you forgot to replace one usage of 3.0V by 3.3V. Below is patch which changes also this last one usage. Applying it has same effect on Nokia N900 as reverting that problematic commit d2c05f50e12f87128597a28146de7092aaa847c3:
diff --git a/drivers/mmc/omap_hsmmc.c b/drivers/mmc/omap_hsmmc.c index 8636cd713a..dc26e54795 100644 --- a/drivers/mmc/omap_hsmmc.c +++ b/drivers/mmc/omap_hsmmc.c @@ -840,7 +840,7 @@ static int omap_hsmmc_init_setup(struct mmc *mmc) omap_hsmmc_conf_bus_power(mmc, (reg_val & VS33_3V3SUP) ? MMC_SIGNAL_VOLTAGE_330 : MMC_SIGNAL_VOLTAGE_180); #else
- writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V0, &mmc_base->hctl);
- writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V3, &mmc_base->hctl); writel(readl(&mmc_base->capa) | VS33_3V3SUP | VS18_1V8SUP, &mmc_base->capa);
#endif
So eMMC on real N900 is working fine either with 3.0V or 3.3V. But 3.3V needs to be configured on all places.
Hello! Could you please take a look at this issue and my above fix?
First problem with reboot loop and crash in mmc_set_ios() still remains. Reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 fixes it.
On Friday 12 June 2020 15:03:06 Pali Rohár wrote:
On Tuesday 26 May 2020 19:49:54 Pali Rohár wrote:
On Thursday 07 May 2020 17:19:38 Pali Rohár wrote:
On Thursday 07 May 2020 19:10:14 Faiz Abbas wrote:
On 26/04/20 3:59 am, Pali Rohár wrote:
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote: > Now I tried git bisect and here is problematic commit which caused whole > reboot loop: > > 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit > commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 > Author: Jean-Jacques Hiblot jjhiblot@ti.com > Date: Thu Sep 21 16:30:08 2017 +0200 > > mmc: disable UHS modes if Vcc cannot be switched on and off > > If a power cycle cannot be done on Vcc, it is safer not to try the UHS > modes because we wouldn't be able to recover from an error occurring > during the UHS initialization. > > Signed-off-by: Jean-Jacques Hiblot jjhiblot@ti.com > > :040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers > :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include > > It changes only core mmc code, nothing platform / board specific. > U-Boot compiled from commit before above has fully working eMMC access > on real Nokia N900. I can read files on FAT eMMC partition without any > problem. > > I'm not sure what is happening here, but it looks like that omap hs mmc > driver used on Nokia N900 is maybe not correctly hooked for UHS support. > > The most suspicious it that this problem cannot be reproduced in qemu > n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
But even with above my change eMMC on N900 cannot be initialized... I'm running second git bisect with applying above change on every commit. It is in progress now.
And second bisect finished. Result is:
d2c05f50e12f87128597a28146de7092aaa847c3 is the first bad commit commit d2c05f50e12f87128597a28146de7092aaa847c3 Author: Faiz Abbas faiz_abbas@ti.com Date: Fri Apr 5 14:18:46 2019 +0530
mmc: omap_hsmmc: Set 3.3V for IO voltage Pbias voltage should match the IO voltage set for the SD card. With the latest pbias change to 3.3V, update the capabilities and IO voltages settings to 3.3V. Signed-off-by: Faiz Abbas <faiz_abbas@ti.com>:040000 040000 819148217b64a6beaf8247464de6b4384d4581a4 e4fd9288ddb794f33596339813d5386f3bed8fd7 M drivers
I revered above commit on top of master and eMMC on Nokia N900 finally started working again!
Faiz, what is the reason for above commit? It basically changes in omap hs mmc driver IO voltage from 3.0V to 3.3V. And seems this change is incompatible with Nokia N900. Are there any problems with 3.0V on some omap3 boards? If not I would vote for revering that commit. Or at least making IO voltage configurable.
For the power_cycle patch, it looks like there is a null pointer dereference that might be causing the reboot loops (probably because your platform doesn't have DM_MMC enabled) Can you add a few more prints to see where the data abort comes from exactly?
Hello Faiz!
Sorry, but this is problematic. Because of reboot loops I cannot read what exactly is put on the display and reboot cause clearing display.
Month ago I was able to detect that problem is somewhere in function mmc_set_ios() from mmc.c file. I put long debug line prior and also after mmc_set_ios() call. And it was printed only prior. Not after. So I think NULL pointer dereference is in mmc_set_ios() function.
For the second patch IO voltage patch, what exactly do you see without reverting it? Are you able to reach prompt but mmc commands fail?
Yes, after reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 there is no reboot loop anymore. Just mmc commands fail and eMMC is not detected at all.
Its possible that your platform requires a pbias of 3.0V to work.
Hello Faiz!
Now I figured out what is the root cause of second 3.0V vs 3.3V problem.
In commit d2c05f50e12f87128597a28146de7092aaa847c3 you forgot to replace one usage of 3.0V by 3.3V. Below is patch which changes also this last one usage. Applying it has same effect on Nokia N900 as reverting that problematic commit d2c05f50e12f87128597a28146de7092aaa847c3:
diff --git a/drivers/mmc/omap_hsmmc.c b/drivers/mmc/omap_hsmmc.c index 8636cd713a..dc26e54795 100644 --- a/drivers/mmc/omap_hsmmc.c +++ b/drivers/mmc/omap_hsmmc.c @@ -840,7 +840,7 @@ static int omap_hsmmc_init_setup(struct mmc *mmc) omap_hsmmc_conf_bus_power(mmc, (reg_val & VS33_3V3SUP) ? MMC_SIGNAL_VOLTAGE_330 : MMC_SIGNAL_VOLTAGE_180); #else
- writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V0, &mmc_base->hctl);
- writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V3, &mmc_base->hctl); writel(readl(&mmc_base->capa) | VS33_3V3SUP | VS18_1V8SUP, &mmc_base->capa);
#endif
So eMMC on real N900 is working fine either with 3.0V or 3.3V. But 3.3V needs to be configured on all places.
Hello! Could you please take a look at this issue and my above fix?
Ping. Any comment for above issue or my fix?
First problem with reboot loop and crash in mmc_set_ios() still remains. Reverting 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 fixes it.
Hi Pali,
On 01/07/20 2:02 pm, Pali Rohár wrote:
On Friday 12 June 2020 15:03:06 Pali Rohár wrote:
On Tuesday 26 May 2020 19:49:54 Pali Rohár wrote:
On Thursday 07 May 2020 17:19:38 Pali Rohár wrote:
On Thursday 07 May 2020 19:10:14 Faiz Abbas wrote:
On 26/04/20 3:59 am, Pali Rohár wrote:
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote: > On Saturday 25 April 2020 23:26:15 Pali Rohár wrote: >> Now I tried git bisect and here is problematic commit which caused whole >> reboot loop: >>
...
Hello Faiz!
Now I figured out what is the root cause of second 3.0V vs 3.3V problem.
In commit d2c05f50e12f87128597a28146de7092aaa847c3 you forgot to replace one usage of 3.0V by 3.3V. Below is patch which changes also this last one usage. Applying it has same effect on Nokia N900 as reverting that problematic commit d2c05f50e12f87128597a28146de7092aaa847c3:
diff --git a/drivers/mmc/omap_hsmmc.c b/drivers/mmc/omap_hsmmc.c index 8636cd713a..dc26e54795 100644 --- a/drivers/mmc/omap_hsmmc.c +++ b/drivers/mmc/omap_hsmmc.c @@ -840,7 +840,7 @@ static int omap_hsmmc_init_setup(struct mmc *mmc) omap_hsmmc_conf_bus_power(mmc, (reg_val & VS33_3V3SUP) ? MMC_SIGNAL_VOLTAGE_330 : MMC_SIGNAL_VOLTAGE_180); #else
- writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V0, &mmc_base->hctl);
- writel(DTW_1_BITMODE | SDBP_PWROFF | SDVS_3V3, &mmc_base->hctl); writel(readl(&mmc_base->capa) | VS33_3V3SUP | VS18_1V8SUP, &mmc_base->capa);
#endif
So eMMC on real N900 is working fine either with 3.0V or 3.3V. But 3.3V needs to be configured on all places.
Hello! Could you please take a look at this issue and my above fix?
Ping. Any comment for above issue or my fix?
Sorry I missed this earlier. The fix makes sense to me. I can give my reviewed by to the your fix once you send the patch.
Thanks, Faiz
Pavel suggested to add Tomi into the loop as Jean is not with TI anymore.
Tomi, could you please look at this mmc related problem? See details below.
On Sunday 26 April 2020 00:20:07 Pali Rohár wrote:
On Saturday 25 April 2020 23:26:15 Pali Rohár wrote:
Adding Jean to the loop. Could you please look at this problem? Your commit (described below) is causing reboot loop on Nokia N900 hardware.
On Saturday 25 April 2020 13:50:45 Pali Rohár wrote:
On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár pali@kernel.org wrote:
On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
...
> U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200) > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz > Nokia RX-51 + LPDDR/OneNAND > I2C: ready > DRAM: 256 MiB > NAND: 0 Bytes
...
> MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1
...
> OMAP die ID: 031400240000000004036ac10b01100f > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz >
And then U-Boot freeze, right?
Any idea how to debug this issue?
On my N900 I'm getting "data abort" error on display and then instant reboot.
It looks like that omap hs mmc code cause that freeze/reboot on real HW. Was there some significat change to OMAP3 or omap hs mmc?
I booted by board from MMC as shown above. I also use the pinctrl features from the device tree to mux the pins in U-Boot (not SPL), so my SPL only does manual muxing the essential components it needs during the SPL phase, and lets U-Boot do the rest. I only mention the pinmux because of message regarding pads/pull-ups.
adam
I debugged this problem more. I disabled all preboot commands to get clean U-Boot setup. And it worked.
After I called any "mmc" command (e.g. "mmc info") I got that instant board reboot. Preboot commands for n900 try to read some files from mmc, so this is reason why it get into reboot loop.
Is there any reason why "mmc info" command can cause "data abort" and instant reboot of board?
And do you know what is needed for proper initialization of omap mmc controller for omap3 board? Because it looks like something fundamental is missing.
Currently there are just calls for "omap_mmc_init()" functions and "twl4030_power_mmc_init()" functions. See: https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c
Now I tried git bisect and here is problematic commit which caused whole reboot loop:
04a2ea248f58b3b6216d0cd0a6b8698df8b14355 is the first bad commit commit 04a2ea248f58b3b6216d0cd0a6b8698df8b14355 Author: Jean-Jacques Hiblot jjhiblot@ti.com Date: Thu Sep 21 16:30:08 2017 +0200
mmc: disable UHS modes if Vcc cannot be switched on and off If a power cycle cannot be done on Vcc, it is safer not to try the UHS modes because we wouldn't be able to recover from an error occurring during the UHS initialization. Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>:040000 040000 04de51428c8311a4b2fb3ad876ac3f6071ab57ee ea7a7959a4bd591c92a2c3d413d5643a8457d2ff M drivers :040000 040000 03f639baf2a2f55003cb750981fd8accc5b4a993 fbcb9607d37959f0b5240f5d727133f58cc35379 M include
It changes only core mmc code, nothing platform / board specific. U-Boot compiled from commit before above has fully working eMMC access on real Nokia N900. I can read files on FAT eMMC partition without any problem.
I'm not sure what is happening here, but it looks like that omap hs mmc driver used on Nokia N900 is maybe not correctly hooked for UHS support.
The most suspicious it that this problem cannot be reproduced in qemu n900 emulator. It happens only on real N900 hw.
I took main change from above commit, reverted it on master and U-Boot stopped crashing! No reboot loop anymore. Here is change which fixes reboot loop on Nokia N900:
diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index 523c055967..d07c7745da 100644 --- a/drivers/mmc/mmc.c +++ b/drivers/mmc/mmc.c @@ -2786,18 +2786,7 @@ int mmc_get_op_cond(struct mmc *mmc) MMC_QUIRK_RETRY_APP_CMD; #endif
- err = mmc_power_cycle(mmc);
- if (err) {
/** if power cycling is not supported, we should not try* to use the UHS modes, because we wouldn't be able to* recover from an error during the UHS initialization.*/pr_debug("Unable to do a full power cycle. Disabling the UHS modes for safety\n");uhs_en = false;mmc->host_caps &= ~UHS_CAPS;err = mmc_power_on(mmc);- }
- err = mmc_power_on(mmc); if (err) return err;
Jean, do you have any idea what is happening in above code? It looks like something is broken in power cycle / power on sequence if it cause "data abort" and reboot of board.
Hi!
Adding Jean to the loop. Could you please look at this problem? Your commit (described below) is causing reboot loop on Nokia N900 hardware.
I'm not sure Jean is still with TI. You may want to talk to Tomi Valkeinen tomi.valkeinen@ti.com if you don't get replies.
Best regards, Pavel
participants (6)
-
Adam Ford -
Faiz Abbas -
Heiko Schocher -
Ivaylo Dimitrov -
Pali Rohár -
Pavel Machek