[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata
Since 440EPx/GRx has problems with accessing the last 256 bytes of SDRAM via the Denali DDR/DDR2 controller, we set CONFIG_PRAM to 1 and reserve 1kByte of protected RAM. This way this memory will not get "touched" by U-Boot. And by passing "mem=${mem}" to the Linux kernel, Linux will not use this area either.
This patch adds this workaround for the following 440EPx boards: sequoia, lwmon5. Others should probably follow this example.
Here the description from the AMCC documentation:
CHIP_11: End of memory range area restricted access. Category: 3
Overview: The 440EPx DDR controller does not acknowledge any transaction which is determined to be crossing over the end-of-memory-range boundary, even if the starting address is within valid memory space. Any such transaction from any PLB4 master will result in a PLB time-out on PLB4 bus.
Impact: In case of such misaligned bursts, PLB4 masters will not retrieve any data at all, just the available data up to the end of memory, especially the 440 CPU. For example, if a CPU instruction required an operand located in memory within the last 7 words of memory, the DCU master would burst read 8 words to update the data cache and cross over the end-of-memory-range boundary. Such a DCU read would not be answered by the DDR controller, resulting in a PLB4 time-out and ultimately in a Machine Check interrupt. The data would be inaccessible to the CPU.
Workaround: Forbid any application to access the last 256 bytes of DDR memory. For example, make your operating system believe that the last 256 bytes of DDR memory are absent. AMCC has a patch that does this, available for Linux.
Signed-off-by: Stefan Roese sr@denx.de --- include/configs/lwmon5.h | 7 +++++-- include/configs/sequoia.h | 2 ++ 2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/include/configs/lwmon5.h b/include/configs/lwmon5.h index c3f10c7..62c0e58 100644 --- a/include/configs/lwmon5.h +++ b/include/configs/lwmon5.h @@ -1,5 +1,5 @@ /* - * (C) Copyright 2007 + * (C) Copyright 2007-2008 * Stefan Roese, DENX Software Engineering, sr@denx.de. * * This program is free software; you can redistribute it and/or @@ -145,6 +145,8 @@ #else #define CFG_POST_ECC_ON 0 #endif +#define CONFIG_PRAM 1 /* don't use last 256 bytes */ + /* 440EPx errata CHIP 11 */
/* POST support */ #define CONFIG_POST (CFG_POST_CACHE | \ @@ -205,7 +207,8 @@ "ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}" \ ":${hostname}:${netdev}:off panic=1\0" \ "addtty=setenv bootargs ${bootargs} console=ttyS1,${baudrate}\0"\ - "addmisc=setenv bootargs ${bootargs} rtc-pcf8563.probe=0,0x51\0"\ + "addmisc=setenv bootargs ${bootargs} rtc-pcf8563.probe=0,0x51" \ + " mem=${mem}\0" \ "flash_nfs=run nfsargs addip addtty addmisc;" \ "bootm ${kernel_addr}\0" \ "flash_self=run ramargs addip addtty addmisc;" \ diff --git a/include/configs/sequoia.h b/include/configs/sequoia.h index dfa8779..541f8bc 100644 --- a/include/configs/sequoia.h +++ b/include/configs/sequoia.h @@ -221,6 +221,8 @@ #if !defined(CONFIG_NAND_U_BOOT) && !defined(CONFIG_NAND_SPL) #define CONFIG_DDR_DATA_EYE /* use DDR2 optimization */ #endif +#define CONFIG_PRAM 1 /* don't use last 256 bytes */ + /* 440EPx errata CHIP 11 */
/* * I2C
Hello Stefan,
in message 1206006397-16582-1-git-send-email-sr@denx.de you wrote:
Since 440EPx/GRx has problems with accessing the last 256 bytes of SDRAM via the Denali DDR/DDR2 controller, we set CONFIG_PRAM to 1 and reserve 1kByte of protected RAM. This way this memory will not get "touched" by U-Boot. And by passing "mem=${mem}" to the Linux kernel, Linux will not use this area either.
Note that this workaround is incomplete.
If you enable shared (between U-Boot and Linux) log buffer, then the log buffer will go at the end of memory, and the pRAM area will be allocated below. In this case the problems is not only still present, but also very likely to hit pretty soon.
This patch adds this workaround for the following 440EPx boards: sequoia, lwmon5. Others should probably follow this example.
OK, the default configs for Sequoia doesn't use shared log buffer, and the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in OCM), so these boards will work.
But any changes to the configuration may easily break this again.
I don't like the idea that each affected board has to fix this (probably differently) in it's board config file. I would prefer a fix (common to all affected boards) in one central location.
Best regards,
Wolfgang Denk
Hi Wolfgang,
On Thursday 20 March 2008, Wolfgang Denk wrote:
This patch adds this workaround for the following 440EPx boards: sequoia, lwmon5. Others should probably follow this example.
OK, the default configs for Sequoia doesn't use shared log buffer, and the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in OCM), so these boards will work.
But any changes to the configuration may easily break this again.
Understood.
I don't like the idea that each affected board has to fix this (probably differently) in it's board config file. I would prefer a fix (common to all affected boards) in one central location.
I don't like this either. But I have not come up with a "generic" solution till now. It's not so easy since the 440EPx SDRAM setup code is sometimes common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it gets even more complicated when ECC initialization is needed too. So currently I have no "better" solution as to do it in a board specific way.
As always, patches implementing a "better" more generic solution are welcome.
Thanks.
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
In message 200803201129.06602.sr@denx.de you wrote:
I don't like this either. But I have not come up with a "generic" solution till now. It's not so easy since the 440EPx SDRAM setup code is sometimes common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it gets even more complicated when ECC initialization is needed too. So currently I have no "better" solution as to do it in a board specific way.
As always, patches implementing a "better" more generic solution are welcome.
At the cost of an #ifdef, this could be added to the memory reservation code in "lib_ppc/board.c", i. e. somewhere after
... 427 * Reserve memory at end of RAM for (top down in that order): 428 * - kernel log buffer 429 * - protected RAM 430 * - LCD framebuffer 431 * - monitor code 432 * - board info struct 433 */ 434 len = (ulong)&_end - CFG_MONITOR_BASE; 435 436 addr = CFG_SDRAM_BASE + get_effective_memsize();
Of course you need to make sure that the linux kernel understands this changed behaviour / shifted addresses, too.
Best regards,
Wolfgang Denk
On Mar 20, 2008, at 6:16 AM, Wolfgang Denk wrote:
In message 200803201129.06602.sr@denx.de you wrote:
I don't like this either. But I have not come up with a "generic" solution till now. It's not so easy since the 440EPx SDRAM setup code is sometimes common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it gets even more complicated when ECC initialization is needed too. So currently I have no "better" solution as to do it in a board specific way.
As always, patches implementing a "better" more generic solution are welcome.
At the cost of an #ifdef, this could be added to the memory reservation code in "lib_ppc/board.c", i. e. somewhere after
... 427 * Reserve memory at end of RAM for (top down in that order): 428 * - kernel log buffer 429 * - protected RAM 430 * - LCD framebuffer 431 * - monitor code 432 * - board info struct 433 */ 434 len = (ulong)&_end - CFG_MONITOR_BASE; 435 436 addr = CFG_SDRAM_BASE + get_effective_memsize();
Of course you need to make sure that the linux kernel understands this changed behaviour / shifted addresses, too.
I haven't looked at the exact details but with the new-image branch of u-boot-testing the introduction of using the lmb to keep track of memory regions used might make this a little easier. It clearly will not help in determining when to do a lmb_reserve().
- k
On Thursday 20 March 2008, Wolfgang Denk wrote:
At the cost of an #ifdef, this could be added to the memory reservation code in "lib_ppc/board.c", i. e. somewhere after
... 427 * Reserve memory at end of RAM for (top down in that order): 428 * - kernel log buffer 429 * - protected RAM 430 * - LCD framebuffer 431 * - monitor code 432 * - board info struct 433 */ 434 len = (ulong)&_end - CFG_MONITOR_BASE; 435 436 addr = CFG_SDRAM_BASE + get_effective_memsize();
OK. I'll add it here with an additional #ifdef here. How about something like this:
len = (ulong)&_end - CFG_MONITOR_BASE;
+#ifdef CONFIG_MEM_TOP_HIDE + /* + * Subtract specified amount of memory to hide so that it won't + * get "touched" at all by U-Boot. By fixing up gd->ram_size + * the Linux kernel should now get passed the now "corrected" + * memory size and won't touch it either. This should work + * for arch/ppc and arch/powerpc. Only Linux board ports in + * arch/powerpc with bootwrapper support, that recalculate the + * memory size from the SDRAM controller setup will have to + * get fixed. + */ + gd->ram_size -= CONFIG_MEM_TOP_HIDE; +#endif
addr = CFG_SDRAM_BASE + get_effective_memsize();
Or even better this somewhere above:
#ifndef CONFIG_MEM_TOP_HIDE #define CONFIG_MEM_TOP_HIDE 0 #endif
and we can drop the #ifdef completely.
Of course you need to make sure that the linux kernel understands this changed behaviour / shifted addresses, too.
Right. By fixing up gd->ram_size in the new patch version above, this should get handled for arch/ppc and arch/powerpc kernel versions. So no need for 'mem=xxx' in the bootargs anymore. Or did I miss something here?
Thanks.
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
Stefan Roese wrote:
Hi Wolfgang,
On Thursday 20 March 2008, Wolfgang Denk wrote:
This patch adds this workaround for the following 440EPx boards: sequoia, lwmon5. Others should probably follow this example.
OK, the default configs for Sequoia doesn't use shared log buffer, and the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in OCM), so these boards will work.
But any changes to the configuration may easily break this again.
Understood.
I don't like the idea that each affected board has to fix this (probably differently) in it's board config file. I would prefer a fix (common to all affected boards) in one central location.
I don't like this either. But I have not come up with a "generic" solution till now. It's not so easy since the 440EPx SDRAM setup code is sometimes common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it gets even more complicated when ECC initialization is needed too. So currently I have no "better" solution as to do it in a board specific way.
As always, patches implementing a "better" more generic solution are welcome.
Hi all,
This may not be the "better, more generic solution", but I just added a CONFIG_CHIP_11_ERRATA option and code in lib_ppc/board.c to trim off only 256 bytes before the optional log buffer is carved out.
There's a kernel patch for this errata that trims the memory length. If that patch is applied, does the kernel still need to be made to understand the changed logbuffer location?
Thanks, Dave
On Thursday 20 March 2008, Dave Littell wrote:
This may not be the "better, more generic solution", but I just added a CONFIG_CHIP_11_ERRATA option and code in lib_ppc/board.c to trim off only 256 bytes before the optional log buffer is carved out.
OK. I'll submit a similar patch in a short while. Please give it a try and let me know if you see problems with it.
There's a kernel patch for this errata that trims the memory length. If that patch is applied, does the kernel still need to be made to understand the changed logbuffer location?
I don't think so. With my next patch it should not be necessary anymore. With one exception: Linux kernel ports that have fixed memory sizes or calculate the memory size from the SDRAM controller setup (as done in some arch/powerpc kernels with bootwrapper support).
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
Stefan Roese wrote:
Hi Wolfgang,
On Thursday 20 March 2008, Wolfgang Denk wrote:
This patch adds this workaround for the following 440EPx boards: sequoia, lwmon5. Others should probably follow this example.
OK, the default configs for Sequoia doesn't use shared log buffer, and the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in OCM), so these boards will work.
But any changes to the configuration may easily break this again.
Understood.
I don't like the idea that each affected board has to fix this (probably differently) in it's board config file. I would prefer a fix (common to all affected boards) in one central location.
I don't like this either. But I have not come up with a "generic" solution till now. It's not so easy since the 440EPx SDRAM setup code is sometimes common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it gets even more complicated when ECC initialization is needed too. So currently I have no "better" solution as to do it in a board specific way.
As always, patches implementing a "better" more generic solution are welcome.
Thanks.
Best regards, Stefan
Hi Stefan,
Having multiple implementations of the 440EXp SDRAM setup code is, in itself, less than ideal. One alternative is to have a 440EPX board with on-board SDRAM chips fake an array of SPD bytes describing the chips, and pass it to the "denali_spd_ddr2.c" code. The SPD code then becomes the single location for the technology of how to configure the Denali SDRAM controller.
The additional benefit (for me, anyway :-) ) is having more eyeballs on the SPD code, with more opportunities for testing, bug fixing, and enhancements.
Best regards, Larry
Hi Larry,
On Thursday 20 March 2008, Larry Johnson wrote:
Having multiple implementations of the 440EXp SDRAM setup code is, in itself, less than ideal. One alternative is to have a 440EPX board with on-board SDRAM chips fake an array of SPD bytes describing the chips, and pass it to the "denali_spd_ddr2.c" code. The SPD code then becomes the single location for the technology of how to configure the Denali SDRAM controller.
Generally a good idea. Unfortunately it can't be done. Think about 440EPx boards booting from NAND flash. Here the complete CPU setup code including SDRAM setup needs to fit into 4k. This can only be done with a fixed SDRAM setup. :-(
But your are right. Other boards without such a image size restriction should probably use the SPD code. One reason they aren't doing it right now is, that the SPD code is quite new and didn't exist when those board port were done. But they can be converted to use this code at some time.
The additional benefit (for me, anyway :-) ) is having more eyeballs on the SPD code, with more opportunities for testing, bug fixing, and enhancements.
Ack.
BTW: Can you test your board with ECC modules? We need to change the ECC code in the Denali SPD routines to not touch the last 256 bytes here too. Best would be if you could provide a patch for this. :)
Thanks.
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
Stefan Roese wrote:
[...]
BTW: Can you test your board with ECC modules? We need to change the ECC code in the Denali SPD routines to not touch the last 256 bytes here too. Best would be if you could provide a patch for this. :)
Thanks.
Best regards, Stefan
Yes, we normally use ECC modules in our testing. I've been looking at a patch for "initdram()" Denali SPD, but I've been waiting to see how your "CFG_MEM_TOP_HIDE" patch would turn out.
As things stand now, can I assume that boards using the Denali SPD will also define "CFG_MEM_TOP_HIDE", and therefore initdram() should continue to return the full size of the memory?
The only place that the last 256 bytes of memory are touched is when "dflush()" is called to zero the SDRAM. This does not cause a Machine Check interrupt. I am guessing that all the writes from "dflush()" are aligned, and therefore there are no burst that access beyond the end of of the SDRAM memory space. If so, then my inclination is not to change this part of the code. Does this make sense?
Best regards, Larry
On Thursday 27 March 2008, Larry Johnson wrote:
Yes, we normally use ECC modules in our testing. I've been looking at a patch for "initdram()" Denali SPD, but I've been waiting to see how your "CFG_MEM_TOP_HIDE" patch would turn out.
As things stand now, can I assume that boards using the Denali SPD will also define "CFG_MEM_TOP_HIDE", and therefore initdram() should continue to return the full size of the memory?
Correct.
The only place that the last 256 bytes of memory are touched is when "dflush()" is called to zero the SDRAM.
To be more specific, in the dcbz_area() call.
This does not cause a Machine Check interrupt. I am guessing that all the writes from "dflush()" are aligned, and therefore there are no burst that access beyond the end of of the SDRAM memory space. If so, then my inclination is not to change this part of the code. Does this make sense?
I recommend to change it this way:
/* Zero the memory */ debug("Zeroing SDRAM..."); - dcbz_area(CFG_SDRAM_BASE, dram_size); + dcbz_area(CFG_SDRAM_BASE, dram_size - CFG_MEM_TOP_HIDE); dflush();
And perhaps put something like:
#ifndef CFG_MEM_TOP_HIDE #error "Please define CFG_MEM_TOP_HIDE (see README) in your board config file!" #endif
at the top of the file. What do you think?
Best regards, Stefan
===================================================================== DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: +49-8142-66989-0 Fax: +49-8142-66989-80 Email: office@denx.de =====================================================================
Stefan Roese wrote:
On Thursday 27 March 2008, Larry Johnson wrote:
Yes, we normally use ECC modules in our testing. I've been looking at a patch for "initdram()" Denali SPD, but I've been waiting to see how your "CFG_MEM_TOP_HIDE" patch would turn out.
As things stand now, can I assume that boards using the Denali SPD will also define "CFG_MEM_TOP_HIDE", and therefore initdram() should continue to return the full size of the memory?
Correct.
The only place that the last 256 bytes of memory are touched is when "dflush()" is called to zero the SDRAM.
To be more specific, in the dcbz_area() call.
This does not cause a Machine Check interrupt. I am guessing that all the writes from "dflush()" are aligned, and therefore there are no burst that access beyond the end of of the SDRAM memory space. If so, then my inclination is not to change this part of the code. Does this make sense?
I recommend to change it this way:
/* Zero the memory */ debug("Zeroing SDRAM...");
dcbz_area(CFG_SDRAM_BASE, dram_size);
dcbz_area(CFG_SDRAM_BASE, dram_size - CFG_MEM_TOP_HIDE); dflush();And perhaps put something like:
#ifndef CFG_MEM_TOP_HIDE #error "Please define CFG_MEM_TOP_HIDE (see README) in your board config file!" #endif
at the top of the file. What do you think?
Best regards, Stefan
That sound fine, I'll try to get a patch out soon.
Best regards, Larry
participants (5)
-
Dave Littell -
Kumar Gala -
Larry Johnson -
Stefan Roese -
Wolfgang Denk