From: Stephen Warren swarren@nvidia.com

The return value of query_sdram_size() is assigned directly to gd->ram_size in dram_init(). Adjust the return type to match the field it's assigned to. This has the beneficial effect that on 64-bit systems, the return value can correctly represent large RAM sizes over 4GB.

For similar reasons, change the type of variable size_bytes in the same way.

query_sdram_size() would previously clip the detected RAM size to at most just under 4GB in all cases, since on 32-bit systems, larger values could not be represented. Disable this feature on 64-bit systems since the representation restriction does not exist.

On 64-bit systems, never call get_ram_size() to validate the detected/ calculated RAM size. On any system with a secure OS/... carve-out, RAM may not have a single contiguous usable area, and this can confuse get_ram_size(). Ideally, we'd make this call conditional upon some other flag that indicates specifically that a carve-out is actually in use. At present, building for a 64-bit system is the best indication we have of this fact. In fact, the call to get_ram_size() is not useful by the time U-Boot runs on any system, since U-Boot (and potentially much other early boot software) always runs from RAM on Tegra, so any mistakes in memory controller register programming will already have manifested themselves and prevented U-Boot from running to this point. In the future, we may simply delete the call to get_ram_size() in all cases.

Signed-off-by: Stephen Warren swarren@nvidia.com --- arch/arm/mach-tegra/board.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/arm/mach-tegra/board.c b/arch/arm/mach-tegra/board.c index 40de72dc575f..b00e4b5c1e25 100644 --- a/arch/arm/mach-tegra/board.c +++ b/arch/arm/mach-tegra/board.c @@ -66,10 +66,11 @@ bool tegra_cpu_is_non_secure(void) #endif

/* Read the RAM size directly from the memory controller */ -unsigned int query_sdram_size(void) +static phys_size_t query_sdram_size(void) { struct mc_ctlr *const mc = (struct mc_ctlr *)NV_PA_MC_BASE; - u32 emem_cfg, size_bytes; + u32 emem_cfg; + phys_size_t size_bytes;

emem_cfg = readl(&mc->mc_emem_cfg); #if defined(CONFIG_TEGRA20) @@ -77,6 +78,7 @@ unsigned int query_sdram_size(void) size_bytes = get_ram_size((void *)PHYS_SDRAM_1, emem_cfg * 1024); #else debug("mc->mc_emem_cfg (MEM_SIZE_MB) = 0x%08x\n", emem_cfg); +#ifndef CONFIG_PHYS_64BIT /* * If >=4GB RAM is present, the byte RAM size won't fit into 32-bits * and will wrap. Clip the reported size to the maximum that a 32-bit @@ -84,9 +86,12 @@ unsigned int query_sdram_size(void) */ if (emem_cfg >= 4096) { size_bytes = U32_MAX & ~(0x1000 - 1); - } else { + } else +#endif + { /* RAM size EMC is programmed to. */ - size_bytes = emem_cfg * 1024 * 1024; + size_bytes = (phys_size_t)emem_cfg * 1024 * 1024; +#ifndef CONFIG_ARM64 /* * If all RAM fits within 32-bits, it can be accessed without * LPAE, so go test the RAM size. Otherwise, we can't access @@ -97,6 +102,7 @@ unsigned int query_sdram_size(void) if (emem_cfg <= (0 - PHYS_SDRAM_1) / (1024 * 1024)) size_bytes = get_ram_size((void *)PHYS_SDRAM_1, size_bytes); +#endif } #endif

From: Stephen Warren swarren@nvidia.com

Represent all available RAM in either one or two banks. The first bank describes any RAM below 4GB. The second bank describes any RAM above 4GB.

This split is driven by the following requirements: - The NVIDIA L4T kernel requires separate entries in the DT /memory/reg property for memory below and above the 4GB boundary. The layout of that DT property is directly driven by the entries in the U-Boot bank array. - On systems with RAM beyond a physical address of 4GB, the potential existence of a carve-out at the end of RAM below 4GB can only be represented using multiple banks, since usable RAM is not contiguous.

While making this change, add a lot more comments re: how and why RAM is represented in banks, and implement a few more "semantic" functions that define (and perhaps later detect at run-time) the size of any carve-out.

Signed-off-by: Stephen Warren swarren@nvidia.com --- arch/arm/mach-tegra/board2.c | 120 ++++++++++++++++++++++++++++++++++++----- include/configs/tegra-common.h | 2 +- 2 files changed, 107 insertions(+), 15 deletions(-)

diff --git a/arch/arm/mach-tegra/board2.c b/arch/arm/mach-tegra/board2.c index 37953cfda8a1..8ecc67459a10 100644 --- a/arch/arm/mach-tegra/board2.c +++ b/arch/arm/mach-tegra/board2.c @@ -10,6 +10,7 @@ #include <errno.h> #include <ns16550.h> #include <linux/compiler.h> +#include <linux/sizes.h> #include <asm/io.h> #include <asm/arch/clock.h> #ifdef CONFIG_LCD @@ -287,27 +288,118 @@ void pad_init_mmc(struct mmc_host *host) } #endif /* MMC */

+/* + * In some SW environments, a memory carve-out exists to house a secure + * monitor, a trusted OS, and/or various statically allocated media buffers. + * + * This carveout exists at the highest possible address that is within a + * 32-bit physical address space. + * + * This function returns the total size of this carve-out. At present, the + * returned value is hard-coded for simplicity. In the future, it may be + * possible to determine the carve-out size: + * - By querying some run-time information source, such as: + * - A structure passed to U-Boot by earlier boot software. + * - SoC registers. + * - A call into the secure monitor. + * - In the per-board U-Boot configuration header, based on knowledge of the + * SW environment that U-Boot is being built for. + * + * For now, we support two configurations in U-Boot: + * - 32-bit ports without any form of carve-out. + * - 64 bit ports which are assumed to use a carve-out of a conservatively + * hard-coded size. + */ +static ulong carveout_size(void) +{ #ifdef CONFIG_ARM64 + return SZ_512M; +#else + return 0; +#endif +} + +/* + * Determine the amount of usable RAM below 4GiB, taking into account any + * carve-out that may be assigned. + */ +static ulong usable_ram_size_below_4g(void) +{ + ulong total_size_below_4g; + ulong usable_size_below_4g; + + /* + * The total size of RAM below 4GiB is the lesser address of: + * (a) 2GiB itself (RAM starts at 2GiB, and 4GiB - 2GiB == 2GiB). + * (b) The size RAM physically present in the system. + */ + if (gd->ram_size < SZ_2G) + total_size_below_4g = gd->ram_size; + else + total_size_below_4g = SZ_2G; + + /* Calculate usable RAM by subtracting out any carve-out size */ + usable_size_below_4g = total_size_below_4g - carveout_size(); + + return usable_size_below_4g; +} + +/* + * Represent all available RAM in either one or two banks. + * + * The first bank describes any usable RAM below 4GiB. + * The second bank describes any RAM above 4GiB. + * + * This split is driven by the following requirements: + * - The NVIDIA L4T kernel requires separate entries in the DT /memory/reg + * property for memory below and above the 4GiB boundary. The layout of that + * DT property is directly driven by the entries in the U-Boot bank array. + * - The potential existence of a carve-out at the end of RAM below 4GiB can + * only be represented using multiple banks. + * + * Explicitly removing the carve-out RAM from the bank entries makes the RAM + * layout a bit more obvious, e.g. when running "bdinfo" at the U-Boot + * command-line. + * + * This does mean that the DT U-Boot passes to the Linux kernel will not + * include this RAM in /memory/reg at all. An alternative would be to include + * all RAM in the U-Boot banks (and hence DT), and add a /memreserve/ node + * into DT to stop the kernel from using the RAM. IIUC, I don't /think/ the + * Linux kernel will ever need to access any RAM in* the carve-out via a CPU + * mapping, so either way is acceptable. + * + * On 32-bit systems, we never define a bank for RAM above 4GiB, since the + * start address of that bank cannot be represented in the 32-bit .size + * field. + */ +void dram_init_banksize(void) +{ + gd->bd->bi_dram[0].start = CONFIG_SYS_SDRAM_BASE; + gd->bd->bi_dram[0].size = usable_ram_size_below_4g(); + +#ifdef CONFIG_PHYS_64BIT + if (gd->ram_size > SZ_2G) { + gd->bd->bi_dram[1].start = 0x100000000; + gd->bd->bi_dram[1].size = gd->ram_size - SZ_2G; + } else +#endif + { + gd->bd->bi_dram[1].start = 0; + gd->bd->bi_dram[1].size = 0; + } +} + /* * Most hardware on 64-bit Tegra is still restricted to DMA to the lower * 32-bits of the physical address space. Cap the maximum usable RAM area * at 4 GiB to avoid DMA buffers from being allocated beyond the 32-bit - * boundary that most devices can address. + * boundary that most devices can address. Also, don't let U-Boot use any + * carve-out, as mentioned above. * - * Additionally, ARM64 devices typically run a secure monitor in EL3 and - * U-Boot in EL2, and set up some secure RAM carve-outs to contain the EL3 - * code and data. These carve-outs are located at the top of 32-bit address - * space. Restrict U-Boot's RAM usage to well below the location of those - * carve-outs. Ideally, we would the secure monitor would inform U-Boot of - * exactly which RAM it could use at run-time. However, I'm not sure how to - * do that at present (and even if such a mechanism does exist, it would - * likely not be generic across all forms of secure monitor). + * This function is called before dram_init_banksize(), so we can't simply + * return gd->bd->bi_dram[1].start + gd->bd->bi_dram[1].size. */ ulong board_get_usable_ram_top(ulong total_size) { - if (gd->ram_top > 0xe0000000) - return 0xe0000000; - - return gd->ram_top; + return CONFIG_SYS_SDRAM_BASE + usable_ram_size_below_4g(); } -#endif diff --git a/include/configs/tegra-common.h b/include/configs/tegra-common.h index ffe167e85ac3..5888555743db 100644 --- a/include/configs/tegra-common.h +++ b/include/configs/tegra-common.h @@ -115,7 +115,7 @@ /*----------------------------------------------------------------------- * Physical Memory Map */ -#define CONFIG_NR_DRAM_BANKS 1 +#define CONFIG_NR_DRAM_BANKS 2 #define PHYS_SDRAM_1 NV_PA_SDRC_CS0 #define PHYS_SDRAM_1_SIZE 0x20000000 /* 512M */