[U-Boot-Users] DDR SDRAM in DIMM module for Yosemite-like PPC440EP board.
Hi:
I'm trying to port u-boot for "Yosemite-like" board with AMCC PPC440EP CPU. The difference between this board and Yosemite itself is that it has 4 128M banks of DDR SDRAM instead of 2 and (which is probably more important) these chips are on DIMM module. I think I've configured everything correct, see SDRAM registers block from BDI2000 config file below:
; Setup SDRAM Controller (DDR SDRAM) WDCR 0x10 0x00000082 ;Select SDRAM0_CLKTR WDCR 0x11 0x40000000 ;CLKTR: Advance 90 degrees WDCR 0x10 0x00000080 ;Select SDRAM0_TR0 WDCR 0x11 0x410a4012 ;TR0: WDCR 0x10 0x00000081 ;Select SDRAM0_TR1 WDCR 0x11 0x80800819 ;TR1: WDCR 0x10 0x00000040 ;Select SDRAM0_B0CR ;;WDCR 0x11 0x000a4001 ;B0CR: 128M first bank, start 0x0, 13x10 WDCR 0x11 0x000a6001 ;B0CR: 128M first bank, start 0x0, 13x11 WDCR 0x10 0x00000044 ;Select SDRAM0_B1CR ;;WDCR 0x11 0x080a4001 ;B1CR: 128M second bank, start 0x08000000, 13x10 WDCR 0x11 0x080a6001 ;B1CR: 128M second bank, start 0x08000000, 13x11 WDCR 0x10 0x00000048 ;Select SDRAM0_B2CR WDCR 0x11 0x100a6001 ;B2CR: 128M third bank, start 0x10000000, 13x11 WDCR 0x10 0x0000004C ;Select SDRAM0_B3CR WDCR 0x11 0x180a6001 ;B3CR: 128M forth bank, start 0x18000000, 13x11 WDCR 0x10 0x00000030 ;Select SDRAM0_RTR WDCR 0x11 0x04080000 ;RTR: WDCR 0x10 0x00000020 ;Select SDRAM0_CFG0 WDCR 0x11 0x36000000 ;CFG0: 32bit, PMU disable, DIMM enable. WDCR 0x11 0x86000000 ;CFG0: enable SDRAM & DIMM WDCR 0x10 0x00000021 ;Select SDRAM0_CFG1 WDCR 0x11 0x00000000 ;CFG1
However MRSC bit of SDRAM0_MCSTS register never gets set meaning SDRAM controller has never successfully completed the Mode Register Set Command. Attempts to read memory also fail (from BDI2000 itself and from u-boot where I've modified memory configuration of course).
That can be HW problem of course, but may be somebody can suggest any configuration problem leading to similar symptoms? Any ideas at all?
Thanks,
Leonid.
Hi Leonid,
On Friday 20 October 2006 08:30, Leonid wrote:
I'm trying to port u-boot for "Yosemite-like" board with AMCC PPC440EP CPU. The difference between this board and Yosemite itself is that it has 4 128M banks of DDR SDRAM instead of 2 and (which is probably more important) these chips are on DIMM module.
If you are using DIMM modules, please don't use the board specific DDR init code of the Yosemite, but the cpu specific DDR init code für DDR DIMM modules, like the other 440 AMCC eval boards do (e.g. ocotea, bamboo).
I think I've configured everything correct, see SDRAM registers block from BDI2000 config file below:
; Setup SDRAM Controller (DDR SDRAM) WDCR 0x10 0x00000082 ;Select SDRAM0_CLKTR
<snip>
However MRSC bit of SDRAM0_MCSTS register never gets set meaning SDRAM controller has never successfully completed the Mode Register Set Command. Attempts to read memory also fail (from BDI2000 itself and from u-boot where I've modified memory configuration of course).
Please modify your U-Boot configuration to use the cpu/ppc4xx/spd_sdram.c code. The following lines should be added to your board config file (with the correct I2C addresses of course):
/*----------------------------------------------------------------------- * DDR SDRAM *----------------------------------------------------------------------*/ #define CONFIG_SPD_EEPROM 1 /* Use SPD EEPROM for setup */ #define SPD_EEPROM_ADDRESS {0x53,0x52} /* SPD i2c spd addresses */
That can be HW problem of course, but may be somebody can suggest any configuration problem leading to similar symptoms? Any ideas at all?
Yes, a HW problem is also possible. What termination are you using? Did you "clone" the HW design from some IBM/AMCC eval boards?
Best regards, Stefan
On Friday, October 20, 2006 4:59 AM Stefan Roese wrote:
If you are using DIMM modules, please don't use the board specific DDR init code of the Yosemite, but the cpu specific DDR init code für DDR DIMM modules, like the other 440 AMCC eval boards do (e.g. ocotea, bamboo).
[Leonid] I'll try to do that, thank you. Yet I probably didn't make myself clear: I connect BDI2000 first whose configuration file can initialize all registers (though one can skip directly to code by "reset run" BDI command). Neither code is running in this moment (except BDI's firmware) and I expect that if BDI config file is correct, SDRAM will be available even BEFORE u-boot code takes over, but it's not: Mode Register Set Command never get completed and WORKSPACE command from config file doesn't work. I'll take a look into BDI config files for other AMCC boards - like ocotea & bamboo if I'll be able to find them.
Please modify your U-Boot configuration to use the cpu/ppc4xx/spd_sdram.c code. The following lines should be added to your board config file (with the correct I2C addresses of course):
#define CONFIG_SPD_EEPROM 1 /* Use SPD EEPROM for setup */ #define SPD_EEPROM_ADDRESS {0x53,0x52} /* SPD i2c spd addresses */
[Leonid] You mean to use option G or H to read bootstrap from serial EEPROM? Or some another exercise with EEPROM I must do because I'm using DIMM module? I'll read this code to see what it does.
Yes, a HW problem is also possible. What termination are you using? Did you "clone" the HW design from some IBM/AMCC eval boards?
[Leonid] The board was supposed to be cloned from Yosemite so it was complete surprise for me when I learnt that HW folks put memory into DIMM module. Can you recommend what exactly should I ask them to look into - what "termination" do you mean?
Best regards, Leonid.
On Fri, 2006-10-20 at 08:57 -0700, Leonid wrote:
On Friday, October 20, 2006 4:59 AM Stefan Roese wrote:
If you are using DIMM modules, please don't use the board specific DDR init code of the Yosemite, but the cpu specific DDR init code für DDR DIMM modules, like the other 440 AMCC eval boards do (e.g. ocotea, bamboo).
[Leonid] I'll try to do that, thank you. Yet I probably didn't make myself clear: I connect BDI2000 first whose configuration file can initialize all registers (though one can skip directly to code by "reset run" BDI command). Neither code is running in this moment (except BDI's firmware) and I expect that if BDI config file is correct, SDRAM will be available even BEFORE u-boot code takes over, but it's not: Mode Register Set Command never get completed and WORKSPACE command from config file doesn't work. I'll take a look into BDI config files for other AMCC boards - like ocotea & bamboo if I'll be able to find them.
Please modify your U-Boot configuration to use the cpu/ppc4xx/spd_sdram.c code. The following lines should be added to your board config file (with the correct I2C addresses of course):
#define CONFIG_SPD_EEPROM 1 /* Use SPD EEPROM for setup */ #define SPD_EEPROM_ADDRESS {0x53,0x52} /* SPD i2c spd addresses */
[Leonid] You mean to use option G or H to read bootstrap from serial EEPROM? Or some another exercise with EEPROM I must do because I'm using DIMM module? I'll read this code to see what it does.
SPD_EEPROM is not a bootstrap EEPROM. It's an I2C eeprom on the DIMM itself that is filled in by the manufacturer to include all sorts of timing and configuration information. You should use this whenever possible because in theory it makes DDR DIMMs plug & play. This of course assumes that your controller is wired and laid out properly and that your spd_eeprom parsing code works...
regards, Ben
On Friday, October 20, 2006 9:04 AM Ben Warren wrote:
SPD_EEPROM is not a bootstrap EEPROM. It's an I2C eeprom on the DIMM itself that is filled in by the manufacturer to include all sorts of timing and configuration information. You should use this whenever possible because in theory it makes DDR DIMMs plug & play. This of course assumes that your controller is wired and laid out properly and that your spd_eeprom parsing code works...
[Leonid] I see, that may be it - this way u-boot may be workable. Is it the only way to work with DIMM module? How I can initialize such SDRAM inside BDI2000 config file? Because I cannot use WORKSPACE otherwise and code burning takes long time...
Regards, Leonid.
On Fri, 2006-10-20 at 09:23 -0700, Leonid wrote:
On Friday, October 20, 2006 9:04 AM Ben Warren wrote:
SPD_EEPROM is not a bootstrap EEPROM. It's an I2C eeprom on the DIMM itself that is filled in by the manufacturer to include all sorts of timing and configuration information. You should use this whenever possible because in theory it makes DDR DIMMs plug & play. This of course assumes that your controller is wired and laid out properly and that your spd_eeprom parsing code works...
[Leonid] I see, that may be it - this way u-boot may be workable. Is it the only way to work with DIMM module? How I can initialize such SDRAM inside BDI2000 config file? Because I cannot use WORKSPACE otherwise and code burning takes long time...
Of course you don't have to use the SPD stuff, but you should. If the BDI won't work as-is, you need to learn more about your DDR controller and your particular DIMM's timing information, both of which should be readily available. While DDR control is fairly complicated, there aren't that many knobs to turn and you may get lucky early on.
Another approach that's less systematic is to get U-boot working, dump the DDR controller register contents and use those in your BDI config file. This will involve a few painful workspace-less flash burns, though... You can save a bit of pain by stripping your U-boot image to bare-bones.
Sorry there's no easier answer, but if your hardware is wired right you'll probably nail this quickly.
regards, Ben
On Friday 20 October 2006 18:50, Ben Warren wrote:
[Leonid] I see, that may be it - this way u-boot may be workable. Is it the only way to work with DIMM module? How I can initialize such SDRAM inside BDI2000 config file? Because I cannot use WORKSPACE otherwise and code burning takes long time...
Of course you don't have to use the SPD stuff, but you should. If the BDI won't work as-is, you need to learn more about your DDR controller and your particular DIMM's timing information, both of which should be readily available. While DDR control is fairly complicated, there aren't that many knobs to turn and you may get lucky early on.
Another approach that's less systematic is to get U-boot working,
Yes. When your hardware layout is ok, you have a good chance that with the correct setup (I2C SPD EEPROM addresses), U-Boot will initialize the DDR controller correctly. Then you have no need to burn flash via the debugger anymore. ;-)
Best regards, Stefan
On Friday, October 20, 2006 9:50 AM Ben Warren wrote:
Of course you don't have to use the SPD stuff, but you should.
[Leonid] Unfortunately SPD stuff is out of question - on our board UDIMM's EEPROM is not connected to CPU I2C bus.
If the BDI won't work as-is, you need to learn more about your DDR controller and your particular DIMM's timing information, both of which should be readily available. While DDR control is fairly complicated, there aren't that many knobs to turn and you may get lucky early on.
[Leonid] That what I'm going to do. I have datasheet, the module name is MT8VDDT12832U. If somebody from this community happens to have BDI or u-boot configuration for such or similar module, I'll appreciate that information.
Another approach that's less systematic is to get U-boot working, dump the DDR controller register contents and use those in your BDI config file.
[Leonid] I'm familiar with this method - I used it to get BDI work with some ARM board. Yet as you said I must have u-boot working with this DIMM first and provided I don't have SPD option it will probably require exactly the same effort as BDI configuration - configure all registers properly manually.
This will involve a few painful workspace-less flash burns, though... You can save a bit of pain by stripping your U-boot image
to
bare-bones.
[Leonid] It's not that bad actually - takes less than 2 minutes to burn entire 512KB u-boot image together with flash erasing even without workspace. But as I see now I may as well concentrate on BDI config file - exactly same registers must be configured - without touching u-boot code at all for the time being.
Sorry there's no easier answer, but if your hardware is wired right you'll probably nail this quickly.
[Leonid] I do appreciate your help, thanks a lot.
Regards, Leonid.
On Friday 20 October 2006 20:02, Leonid wrote:
On Friday, October 20, 2006 9:50 AM Ben Warren wrote:
Of course you don't have to use the SPD stuff, but you should.
[Leonid] Unfortunately SPD stuff is out of question - on our board UDIMM's EEPROM is not connected to CPU I2C bus.
Whow. Never seen this one. So your only chance is to _always_ use the same (or compatible) DIMM module in your board and configure the controller for it.
Best regards, Stefan
On Friday, October 20, 2006 11:33 AM Stefan Roese
Whow. Never seen this one. So your only chance is to _always_ use the
same > (or compatible) DIMM module in your board and configure the controller for
it.
They actually hook it up now, so I may have EEPROM after all. I've learnt that another PPC440EP board - "bamboo" - has exactly this DIMM module, so I now have kind of mix of Yosemite and Bamboo. Do you think I shall go with standard spd_sdram.c or manually port code from bamboo.c to Yosemite.c?
Also does somebody happen to have BDI2000 config file for bamboo board - I couldn't find one...
Best regards, Leonid.
On Friday 20 October 2006 20:47, Leonid wrote:
They actually hook it up now, so I may have EEPROM after all. I've learnt that another PPC440EP board - "bamboo" - has exactly this DIMM module, so I now have kind of mix of Yosemite and Bamboo. Do you think I shall go with standard spd_sdram.c or manually port code from bamboo.c to Yosemite.c?
You should go with standard spd_sdram.c (I did mention this before, right?)! Just remove your board specific sdram init code and include the definitions I mentioned earlier in your board config file.
But please note that bamboo (440EP) is not a perfect example for DDR init on 440 systems, since it has one bank of DDR soldered onboard _plus_ the DIMM module(s).
Also does somebody happen to have BDI2000 config file for bamboo board - I couldn't find one...
Attached is a bamboo config file. But again, this DDR configuration is for the onboard SDRAM IIRC.
Best regards, Stefan
Hi Leonid,
On Friday 20 October 2006 23:00, Stefan Roese wrote:
But please note that bamboo (440EP) is not a perfect example for DDR init on 440 systems, since it has one bank of DDR soldered onboard _plus_ the DIMM module(s).
I just realized that we have another 440EP board in the U-Boot tree that matches your SDRAM setup better. The "pcs440ep". It has one DIMM module and no onboard SDRAM.
Best regards, Stefan
On Saturday, October 21, 2006 3:19 AM Stefan Roese wrote:
But please note that bamboo (440EP) is not a perfect example for DDR
init on 440 systems, since it has one bank of DDR soldered onboard _plus_ the DIMM module(s).
I just realized that we have another 440EP board in the U-Boot tree
that
matches your SDRAM setup better. The "pcs440ep". It has one DIMM
module
and no onboard SDRAM.
[Leonid] Actually I could make spd_sdram work for my board (our HW guy hooked up DIMM EEPROM connections for me). I stepped through code using debugger and found out that EEPROM (address 0x51 on our board) has been successfully read, all parameters were sane and SDRAM registers have been updated accordantly. However sdram0_mcsts register's most significant bit never becomes 1, meaning SDRAM controller cannot complete memory initialization and code stays in endless loop:
/* * wait for SDRAM_CFG0_DC_EN to complete */ while (1) { mfsdram(mem_mcsts, mcsts); if ((mcsts & SDRAM_MCSTS_MRSC) != 0) { break; } }
That looks like HW problem for me. Of course, I'll look through actual registers' values more carefully - may be spd_sdram() just parsed EEPROM parameters wrongly but I doubt that.
Just in case - did you happen to have BDI config file for that board - brief search in the ftp://ftp.denx.de/pub/BDI2000/ directory didn't provide results.
Best regards, Leonid.
On Sunday 22 October 2006 04:38, Leonid wrote:
[Leonid] Actually I could make spd_sdram work for my board (our HW guy hooked up DIMM EEPROM connections for me). I stepped through code using debugger and found out that EEPROM (address 0x51 on our board) has been successfully read, all parameters were sane and SDRAM registers have been updated accordantly.
OK. So far so good.
However sdram0_mcsts register's most significant bit never becomes 1, meaning SDRAM controller cannot complete memory initialization and code stays in endless loop:
/* * wait for SDRAM_CFG0_DC_EN to complete */ while (1) { mfsdram(mem_mcsts, mcsts); if ((mcsts & SDRAM_MCSTS_MRSC) != 0) { break; } }
That looks like HW problem for me.
Yes. I never have seen a board where this did not happen. Did you try to skip this test to see, if the DDR interface perhaps works without this bit set? (Just a test of course, no solution).
I have to admit, that I don't know what problems would result in this bit not being set, apart from DC_EN not set in the SDRAM0_CFG0 register.
Of course, I'll look through actual registers' values more carefully - may be spd_sdram() just parsed EEPROM parameters wrongly but I doubt that.
Just in case - did you happen to have BDI config file for that board - brief search in the ftp://ftp.denx.de/pub/BDI2000/ directory didn't provide results.
Yes. Please find it attached.
Best regards, Stefan
On Sunday, October 22, 2006 4:13 AM Stefan Roese wrote:
Actually I could make spd_sdram work for my board (our HW guy hooked up DIMM EEPROM connections for me). I stepped through code
using
debugger and found out that EEPROM (address 0x51 on our board) has
been
successfully read, all parameters were sane and SDRAM registers have been updated accordantly.
OK. So far so good.
However sdram0_mcsts register's most significant bit never becomes 1, meaning SDRAM controller cannot complete memory initialization and code stays in endless loop: That looks like HW problem for me.
Yes. I never have seen a board where this did not happen. Did you try
to
skip this test to see, if the DDR interface perhaps works without this
bit
set? (Just a test of course, no solution).
[Leonid] It was first thing I tried - just leave the loop after 1 mln loops even if status is not ready. It fails on memory test in program_tr1() function. I'll work with our HW folks to see what can be done.
Best regards, Leonid.
On Sunday, October 22, 2006 4:13 AM Stefan Roese wrote:
However sdram0_mcsts register's most significant bit never becomes 1, meaning SDRAM controller cannot complete memory initialization and code stays in endless loop: That looks like HW problem for me.
Yes. I never have seen a board where this did not happen.
Now I pass sdram0_mcsts status check (our HW folks said there was some clock marginality they fixed) and proceed to program_tr1() function which fails:
U-Boot 1.1.4 (Oct 23 2006 - 20:28:01)
CPU: AMCC PowerPC 440EP Rev. B at 333.333 MHz (PLB=133, OPB=66, EBC=33 MHz) I2C boot EEPROM disabled Internal PCI arbiter disabled, PCI async ext clock used 32 kB I-Cache 32 kB D-Cache Board: Yosemite - AMCC PPC440EP Evaluation Board I2C: ready DRAM: get_spd_info banks 1 Addresses: 0x51 DIMM: slot 0: populated, bytes 128 size 8 DIMM: slot 0: DDR SDRAM detected DIMM: 0 voltage level supported. DIMM: sdram0_cfg0 0x02000000 -> 0x00000000 -> 0x04000000. DIMM: sdram0_cfg1 0x00000000 -> 0x00000000. DIMM: sdram0_rtr 0x04100000. DIMM: sdram0_tr0 0x00894012 -> 0x00000000 -> 0x408A4012. DIMM: sdram0_b0cr 0x000C6001 base 0x00000000 size 0x10000000. DIMM: sdram0_b1cr 0x100C6001 base 0x10000000 size 0x10000000. DIMM: total size 0x20000000 Starting memory test .... ERROR: Cannot determine a common read delay. ### ERROR ### Please RESET the board ###
I tried several hardcoded tr1 values, it doesn't work either - code crashes on read/write attempts - I must investigate where exactly by reviewing assembler - GDB debugging on C level was not very helpful. But I believe it still same HW problem - situation became better, yet not good enough...
Best regards, Leonid.
On Tuesday 24 October 2006 06:14, Leonid wrote:
Now I pass sdram0_mcsts status check (our HW folks said there was some clock marginality they fixed
Hopefully no further "marginality" problems are hidden.
) and proceed to program_tr1() function which fails:
U-Boot 1.1.4 (Oct 23 2006 - 20:28:01)
CPU: AMCC PowerPC 440EP Rev. B at 333.333 MHz (PLB=133, OPB=66, EBC=33 MHz) I2C boot EEPROM disabled Internal PCI arbiter disabled, PCI async ext clock used 32 kB I-Cache 32 kB D-Cache Board: Yosemite - AMCC PPC440EP Evaluation Board I2C: ready DRAM: get_spd_info banks 1 Addresses: 0x51 DIMM: slot 0: populated, bytes 128 size 8 DIMM: slot 0: DDR SDRAM detected DIMM: 0 voltage level supported. DIMM: sdram0_cfg0 0x02000000 -> 0x00000000 -> 0x04000000. DIMM: sdram0_cfg1 0x00000000 -> 0x00000000. DIMM: sdram0_rtr 0x04100000. DIMM: sdram0_tr0 0x00894012 -> 0x00000000 -> 0x408A4012. DIMM: sdram0_b0cr 0x000C6001 base 0x00000000 size 0x10000000. DIMM: sdram0_b1cr 0x100C6001 base 0x10000000 size 0x10000000. DIMM: total size 0x20000000 Starting memory test .... ERROR: Cannot determine a common read delay. ### ERROR ### Please RESET the board ###
I tried several hardcoded tr1 values, it doesn't work either - code crashes on read/write attempts - I must investigate where exactly by reviewing assembler - GDB debugging on C level was not very helpful. But I believe it still same HW problem - situation became better, yet not good enough...
Did you try different DDR modules? I would suggest to start with small modules, perhaps only with one bank. For bigger modules you have to make sure that enough TLB entries are set up.
Best regards, Stefan
participants (3)
-
Ben Warren -
Leonid -
Stefan Roese