[U-Boot] [PATCH v1 0/3] ddr: vybrid: Support for vf610 built-in DDR3 memory calibration
This patch series provides code to perform read leveling - RDLVL, which is adjusting the DQS strobe in relation to the DQ signals so that the strobe edge is centered in the window of valid read data.
The code is based on Vybrid's Reference Manual's: "VFxxx Controller Reference Manual, Rev. 0, 10/2016", page 1600, 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode"
and uses clarification provided by following NXP's community thread: "Vybrid: About DDR leveling feature on DDRMC." https://community.nxp.com/thread/395323
It depends on a BITMAP rework patch: usb: composite: Move bitmap related operations to ./include/linux/bitmap.h http://patchwork.ozlabs.org/patch/1006448/
Lukasz Majewski (3): ddr: vybrid: Add DDRMC calibration related registers (DQS to DQ) ddr: vybrid: Provide code to perform on-boot calibration ddr: vybrid: Add calibration code to memory controler's (DDRMC) setup code
arch/arm/include/asm/arch-vf610/imx-regs.h | 6 + arch/arm/mach-imx/Kconfig | 8 + arch/arm/mach-imx/Makefile | 1 + arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 ++++++++++++++++++++++++++++ arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ arch/arm/mach-imx/ddrmc-vf610.c | 7 + 6 files changed, 417 insertions(+) create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h
This commit provides extra defines needed for DDR memory controller calibration (read leveling performing).
Signed-off-by: Lukasz Majewski lukma@denx.de ---
arch/arm/include/asm/arch-vf610/imx-regs.h | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/arch/arm/include/asm/arch-vf610/imx-regs.h b/arch/arm/include/asm/arch-vf610/imx-regs.h index 08ba8e94f8..f477e414f9 100644 --- a/arch/arm/include/asm/arch-vf610/imx-regs.h +++ b/arch/arm/include/asm/arch-vf610/imx-regs.h @@ -206,11 +206,17 @@ #define DDRMC_CR88_TODTL_CMD(v) (((v) & 0x1f) << 16) #define DDRMC_CR89_AODT_RWSMCS(v) ((v) & 0xf) #define DDRMC_CR91_R2W_SMCSDL(v) (((v) & 0x7) << 16) +#define DDRMC_CR93_SW_LVL_MODE(v) (((v) & 0x3) << 8) +#define DDRMC_CR93_SWLVL_LOAD BIT(16) +#define DDRMC_CR93_SWLVL_START BIT(24) +#define DDRMC_CR93_SWLVL_EXIT BIT(0) +#define DDRMC_CR94_SWLVL_OP_DONE BIT(8) #define DDRMC_CR96_WLMRD(v) (((v) & 0x3f) << 8) #define DDRMC_CR96_WLDQSEN(v) ((v) & 0x3f) #define DDRMC_CR97_WRLVL_EN (1 << 24) #define DDRMC_CR98_WRLVL_DL_0(v) ((v) & 0xffff) #define DDRMC_CR99_WRLVL_DL_1(v) ((v) & 0xffff) +#define DDRMC_CR101_PHY_RDLVL_EDGE BIT(24) #define DDRMC_CR102_RDLVL_GT_REGEN (1 << 16) #define DDRMC_CR102_RDLVL_REG_EN (1 << 8) #define DDRMC_CR105_RDLVL_DL_0(v) (((v) & 0xff) << 8)
On 02.12.2018 21:42, Lukasz Majewski wrote:
This commit provides extra defines needed for DDR memory controller calibration (read leveling performing).
Signed-off-by: Lukasz Majewski lukma@denx.de
arch/arm/include/asm/arch-vf610/imx-regs.h | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/arch/arm/include/asm/arch-vf610/imx-regs.h b/arch/arm/include/asm/arch-vf610/imx-regs.h index 08ba8e94f8..f477e414f9 100644 --- a/arch/arm/include/asm/arch-vf610/imx-regs.h +++ b/arch/arm/include/asm/arch-vf610/imx-regs.h @@ -206,11 +206,17 @@ #define DDRMC_CR88_TODTL_CMD(v) (((v) & 0x1f) << 16) #define DDRMC_CR89_AODT_RWSMCS(v) ((v) & 0xf) #define DDRMC_CR91_R2W_SMCSDL(v) (((v) & 0x7) << 16) +#define DDRMC_CR93_SW_LVL_MODE(v) (((v) & 0x3) << 8) +#define DDRMC_CR93_SWLVL_LOAD BIT(16) +#define DDRMC_CR93_SWLVL_START BIT(24) +#define DDRMC_CR93_SWLVL_EXIT BIT(0)
SWLVL_EXIT is in CR94.
With that fixed, looks good.
Reviewed-by: Stefan Agner stefan.agner@toradex.com
-- Stefan
+#define DDRMC_CR94_SWLVL_OP_DONE BIT(8) #define DDRMC_CR96_WLMRD(v) (((v) & 0x3f) << 8) #define DDRMC_CR96_WLDQSEN(v) ((v) & 0x3f) #define DDRMC_CR97_WRLVL_EN (1 << 24) #define DDRMC_CR98_WRLVL_DL_0(v) ((v) & 0xffff) #define DDRMC_CR99_WRLVL_DL_1(v) ((v) & 0xffff) +#define DDRMC_CR101_PHY_RDLVL_EDGE BIT(24) #define DDRMC_CR102_RDLVL_GT_REGEN (1 << 16) #define DDRMC_CR102_RDLVL_REG_EN (1 << 8) #define DDRMC_CR105_RDLVL_DL_0(v) (((v) & 0xff) << 8)
This patch provides the code to calibrate the DDR's DQS to DQ signals (RDLVL).
It is based on: VFxxx Controller Reference Manual, Rev. 0, 10/2016, page 1600 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode"
and NXP's community thread: "Vybrid: About DDR leveling feature on DDRMC." https://community.nxp.com/thread/395323
Signed-off-by: Lukasz Majewski lukma@denx.de ---
arch/arm/mach-imx/Kconfig | 8 + arch/arm/mach-imx/Makefile | 1 + arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 ++++++++++++++++++++++++++++ arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ 4 files changed, 404 insertions(+) create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h
diff --git a/arch/arm/mach-imx/Kconfig b/arch/arm/mach-imx/Kconfig index a1566cc2ad..ca10ba683f 100644 --- a/arch/arm/mach-imx/Kconfig +++ b/arch/arm/mach-imx/Kconfig @@ -78,3 +78,11 @@ config NXP_BOARD_REVISION NXP boards based on i.MX6/7 contain the board revision information stored in the fuses. Select this option if you want to be able to retrieve the board revision information. + +config DDRMC_VF610_CALIBRATION + bool "Enable DDRMC (DDR3) on-chip calibration" + depends on ARCH_VF610 + help + Vybrid (vf610) SoC provides some on-chip facility to tune the DDR3 + memory parameters. Select this option if you want to calculate them + at boot time. diff --git a/arch/arm/mach-imx/Makefile b/arch/arm/mach-imx/Makefile index 53d9e5f42b..4a07b1ea69 100644 --- a/arch/arm/mach-imx/Makefile +++ b/arch/arm/mach-imx/Makefile @@ -51,6 +51,7 @@ obj-$(CONFIG_SECURE_BOOT) += hab.o endif ifeq ($(SOC),$(filter $(SOC),vf610)) obj-y += ddrmc-vf610.o +obj-$(CONFIG_DDRMC_VF610_CALIBRATION) += ddrmc-vf610-calibration.o endif ifneq ($(CONFIG_SPL_BUILD),y) obj-$(CONFIG_CMD_BMODE) += cmd_bmode.o diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.c b/arch/arm/mach-imx/ddrmc-vf610-calibration.c new file mode 100644 index 0000000000..6ed3f5375b --- /dev/null +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.c @@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * ddrmc DDR3 calibration code for NXP's VF610 + * + * Copyright (C) 2018 DENX Software Engineering + * Lukasz Majewski, DENX Software Engineering, lukma@denx.de + * + */ +/* #define DEBUG */ +#include <common.h> +#include <asm/io.h> +#include <asm/arch/imx-regs.h> +#include <linux/bitmap.h> + +#include "ddrmc-vf610-calibration.h" + +/* + * Documents: + * + * [1] "Vybrid: About DDR leveling feature on DDRMC." + * https://community.nxp.com/thread/395323 + * + * [2] VFxxx Controller Reference Manual, Rev. 0, 10/2016 + * + * + * NOTE + * ==== + * + * NXP recommends setting 'fixed' parameters instead of performing the + * training at each boot. + * + * Use those functions to determine those values on new HW and write + * default values to registers. + * + * SW leveling supported operations - CR93[SW_LVL_MODE]: + * + * - 0x0 (b'00) - No leveling + * + * - 0x1 (b'01) - WRLVL_DL_X - It is not recommended to perform this tuning + * on HW designs utilizing non-flyback topology + * (Single DDR3 with x16). + * Instead the WRLVL_DL_0/1 fields shall be set + * based on trace length differences from their + * layout. + * Mismatches up to 25% or tCK (clock period) are + * allowed, so the value in the filed doesn’t have + * to be very accurate. + * + * - 0x2 (b'10) - RDLVL_DL_0/1 - refers to adjusting the DQS strobe in relation + * to the DQ signals so that the strobe edge is + * centered in the window of valid read data. + * + * - 0x3 (b'11) - RDLVL_GTDL_0/1 - refers to the delay the PHY uses to un-gate + * the Read DQS strobe pad from the time that the + * PHY enables the pad to input the strobe signal. + * + */ +static int ddr_cal_get_first_edge_index(unsigned long *bmap, enum edge e, + int samples, int start, int max) +{ + int i, ret = -1; + + /* + * We look only for the first value (and filter out + * some wrong data) + */ + switch (e) { + case RISING_EDGE: + for (i = start; i <= max - samples; i++) { + if (test_bit(i, bmap)) { + if (!test_bit(i - 1, bmap) && + test_bit(i + 1, bmap) && + test_bit(i + 2, bmap) && + test_bit(i + 3, bmap)) { + return i; + } + } + } + break; + case FALLING_EDGE: + for (i = start; i <= max - samples; i++) { + if (!test_bit(i, bmap)) { + if (test_bit(i - 1, bmap) && + test_bit(i - 2, bmap) && + test_bit(i - 3, bmap)) { + return i; + } + } + } + } + + return ret; +} + +static void bitmap_print(unsigned long *bmap, int max) +{ + int i; + + debug("BITMAP [0x%p]:\n", bmap); + for (i = 0; i <= max; i++) { + debug("%d ", test_bit(i, bmap) ? 1 : 0); + if (i && (i % 32) == (32 - 1)) + debug("\n"); + } + debug("\n"); +} + +#define sw_leveling_op_done \ + while (!(readl(&ddrmr->cr[94]) & DDRMC_CR94_SWLVL_OP_DONE)) + +#define sw_leveling_load_value \ + do { clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SWLVL_LOAD, \ + DDRMC_CR93_SWLVL_LOAD); } while (0) + +#define sw_leveling_start \ + do { clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SWLVL_START, \ + DDRMC_CR93_SWLVL_START); } while (0) + +#define sw_leveling_exit \ + do { clrsetbits_le32(&ddrmr->cr[94], DDRMC_CR93_SWLVL_EXIT, \ + DDRMC_CR93_SWLVL_EXIT); } while (0) + +/* + * RDLVL_DL calibration: + * + * NXP is _NOT_ recommending performing the leveling at each + * boot. Instead - one shall run this procedure on new boards + * and then use hardcoded values. + * + */ +static int ddrmc_cal_dqs_to_dq(struct ddrmr_regs *ddrmr) +{ + DECLARE_BITMAP(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1); + int rdlvl_dl_0_min = -1, rdlvl_dl_0_max = -1; + int rdlvl_dl_1_min = -1, rdlvl_dl_1_max = -1; + int rdlvl_dl_0, rdlvl_dl_1; + u8 swlvl_rsp; + u32 tmp; + int i; + + /* Read defaults */ + u16 rdlvl_dl_0_def = + (readl(&ddrmr->cr[105]) >> RDLVL_DL_O_OFF) & 0xFFFF; + u16 rdlvl_dl_1_def = readl(&ddrmr->cr[110]) & 0xFFFF; + + debug("\nRDLVL: ======================\n"); + debug("RDLVL: DQS to DQ (RDLVL)\n"); + + debug("RDLVL: RDLVL_DL_0_DFL:\t 0x%x\n", rdlvl_dl_0_def); + debug("RDLVL: RDLVL_DL_1_DFL:\t 0x%x\n", rdlvl_dl_1_def); + + /* + * Set/Read setup for calibration + * + * Values necessary for leveling from Vybrid RM [2] - page 1600 + */ + writel(0x40703030, &ddrmr->cr[144]); + writel(0x40, &ddrmr->cr[145]); + writel(0x40, &ddrmr->cr[146]); + + tmp = readl(&ddrmr->cr[144]); + debug("RDLVL: PHY_RDLVL_RES:\t 0x%x\n", (tmp >> 24) & 0xFF);// set 0x40 + debug("RDLVL: PHY_RDLV_LOAD:\t 0x%x\n", (tmp >> 16) & 0xFF);// set 0x70 + debug("RDLVL: PHY_RDLV_DLL:\t 0x%x\n", (tmp >> 8) & 0xFF); // set 0x30 + debug("RDLVL: PHY_RDLV_EN:\t 0x%x\n", tmp & 0xFF); //set 0x30 + + tmp = readl(&ddrmr->cr[145]); + debug("RDLVL: PHY_RDLV_RR:\t 0x%x\n", tmp & 0x3FF); //set 0x40 + + tmp = readl(&ddrmr->cr[146]); + debug("RDLVL: PHY_RDLV_RESP:\t 0x%x\n", tmp); //set 0x40 + + /* + * Program/read the leveling edge RDLVL_EDGE = 0 + * + * 0x00 is the correct output on SWLVL_RSP_X + * If by any chance 1s are visible -> wrong number read + */ + clrbits_le32(&ddrmr->cr[101], DDRMC_CR101_PHY_RDLVL_EDGE); + + tmp = readl(&ddrmr->cr[101]); + debug("RDLVL: PHY_RDLVL_EDGE:\t 0x%x\n", + (tmp >> PHY_RDLVL_EDGE) & 0x1); //set 0 + + /* Program Leveling mode - CR93[SW_LVL_MODE] to ’b10 */ + clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SW_LVL_MODE(0x3), + DDRMC_CR93_SW_LVL_MODE(0x2)); + tmp = readl(&ddrmr->cr[93]); + debug("RDLVL: SW_LVL_MODE:\t 0x%x\n", (tmp >> SW_LVL_MODE) & 0x3); + + /* Start procedure - CR93[SWLVL_START] to ’b1 */ + sw_leveling_start; + + /* Poll CR94[SWLVL_OP_DONE] */ + sw_leveling_op_done; + + /* + * Program delays for RDLVL_DL_0 + * + * The procedure is to increase the delay values from 0 to 0xFF + * and read the response from the DDRMC + */ + debug("\nRDLVL: ---> RDLVL_DL_0\n"); + bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1); + + for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) { + clrsetbits_le32(&ddrmr->cr[105], 0xFFFF << RDLVL_DL_O_OFF, + i << RDLVL_DL_O_OFF); + + /* Load values CR93[SWLVL_LOAD] to ’b1 */ + sw_leveling_load_value; + + /* Poll CR94[SWLVL_OP_DONE] */ + sw_leveling_op_done; + + /* + * Read Responses - SWLVL_RESP_0 + * + * The 0x00 (correct response when PHY_RDLVL_EDGE = 0) + * -> 1 in the bit vector + */ + swlvl_rsp = (readl(&ddrmr->cr[94]) >> SWLVL_RESP_0) & 0xF; + if (swlvl_rsp == 0) + generic_set_bit(i, rdlvl_rsp); + } + + bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY); + + /* + * First test for rising edge 0x0 -> 0x1 in bitmap + */ + rdlvl_dl_0_min = ddr_cal_get_first_edge_index(rdlvl_rsp, RISING_EDGE, + N_SAMPLES, N_SAMPLES, + DDRMC_DQS_DQ_MAX_DELAY); + + /* + * Secondly test for falling edge 0x1 -> 0x0 in bitmap + */ + rdlvl_dl_0_max = ddr_cal_get_first_edge_index(rdlvl_rsp, FALLING_EDGE, + N_SAMPLES, rdlvl_dl_0_min, + DDRMC_DQS_DQ_MAX_DELAY); + + debug("RDLVL: DL_0 min: %d [0x%x] DL_0 max: %d [0x%x]\n", + rdlvl_dl_0_min, rdlvl_dl_0_min, rdlvl_dl_0_max, rdlvl_dl_0_max); + rdlvl_dl_0 = (rdlvl_dl_0_max - rdlvl_dl_0_min) / 2; + + if (rdlvl_dl_0_max == -1 || rdlvl_dl_0_min == -1 || rdlvl_dl_0 <= 0) { + debug("RDLVL: The DQS to DQ delay cannot be found!\n"); + debug("RDLVL: Using default - slice 0: %d!\n", rdlvl_dl_0_def); + rdlvl_dl_0 = rdlvl_dl_0_def; + } + + debug("\nRDLVL: ---> RDLVL_DL_1\n"); + bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1); + + for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) { + clrsetbits_le32(&ddrmr->cr[110], 0xFFFF << RDLVL_DL_1_OFF, + i << RDLVL_DL_1_OFF); + + /* Load values CR93[SWLVL_LOAD] to ’b1 */ + sw_leveling_load_value; + + /* Poll CR94[SWLVL_OP_DONE] */ + sw_leveling_op_done; + + /* + * Read Responses - SWLVL_RESP_1 + * + * The 0x00 (correct response when PHY_RDLVL_EDGE = 0) + * -> 1 in the bit vector + */ + swlvl_rsp = (readl(&ddrmr->cr[95]) >> SWLVL_RESP_1) & 0xF; + if (swlvl_rsp == 0) + generic_set_bit(i, rdlvl_rsp); + } + + bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY); + + /* + * First test for rising edge 0x0 -> 0x1 in bitmap + */ + rdlvl_dl_1_min = ddr_cal_get_first_edge_index(rdlvl_rsp, RISING_EDGE, + N_SAMPLES, N_SAMPLES, + DDRMC_DQS_DQ_MAX_DELAY); + + /* + * Secondly test for falling edge 0x1 -> 0x0 in bitmap + */ + rdlvl_dl_1_max = ddr_cal_get_first_edge_index(rdlvl_rsp, FALLING_EDGE, + N_SAMPLES, rdlvl_dl_1_min, + DDRMC_DQS_DQ_MAX_DELAY); + + debug("RDLVL: DL_1 min: %d [0x%x] DL_1 max: %d [0x%x]\n", + rdlvl_dl_1_min, rdlvl_dl_1_min, rdlvl_dl_1_max, rdlvl_dl_1_max); + rdlvl_dl_1 = (rdlvl_dl_1_max - rdlvl_dl_1_min) / 2; + + if (rdlvl_dl_1_max == -1 || rdlvl_dl_1_min == -1 || rdlvl_dl_1 <= 0) { + debug("RDLVL: The DQS to DQ delay cannot be found!\n"); + debug("RDLVL: Using default - slice 1: %d!\n", rdlvl_dl_1_def); + rdlvl_dl_1 = rdlvl_dl_1_def; + } + + debug("RDLVL: CALIBRATED: rdlvl_dl_0: 0x%x\t rdlvl_dl_1: 0x%x\n", + rdlvl_dl_0, rdlvl_dl_1); + + /* Write new delay values */ + writel((rdlvl_dl_0 << RDLVL_DL_O_OFF), &ddrmr->cr[105]); + writel((rdlvl_dl_1 << RDLVL_DL_1_OFF), &ddrmr->cr[110]); + + sw_leveling_load_value; + sw_leveling_op_done; + + /* Exit procedure - CR94[SWLVL_EXIT] to ’b1 */ + sw_leveling_exit; + + /* Poll CR94[SWLVL_OP_DONE] */ + sw_leveling_op_done; + + return 0; +} + +/* + * WRLVL_DL calibration: + * + * For non-flyback memory architecture - where one have a single DDR3 x16 + * memory - it is NOT necessary to perform "Write Leveling" + * [3] 'Vybrid DDR3 write leveling' https://community.nxp.com/thread/429362 + * + */ + +int ddrmc_calibration(struct ddrmr_regs *ddrmr) +{ + ddrmc_cal_dqs_to_dq(ddrmr); + + return 0; +} diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.h b/arch/arm/mach-imx/ddrmc-vf610-calibration.h new file mode 100644 index 0000000000..7bc53236be --- /dev/null +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * ddrmc DDR3 calibration code for NXP's VF610 + * + * Copyright (C) 2018 DENX Software Engineering + * Lukasz Majewski, DENX Software Engineering, lukma@denx.de + * + */ + +#ifndef __DDRMC_VF610_CALIBRATOIN_H_ +#define __DDRMC_VF610_CALIBRATOIN_H_ + +/* + * Number of "samples" in the calibration bitmap + * to be considered during calibration. + */ +#define N_SAMPLES 3 + +/* + * Constants to indicate if we are looking for a rising or + * falling edge in the calibration bitmap + */ +enum edge { + FALLING_EDGE = 1, + RISING_EDGE +}; + +/* + * The max number of delay elements when DQS to DQ setting + */ +#define DDRMC_DQS_DQ_MAX_DELAY 0xFF + +/* Bits offsets for DDRMC_CR registers */ +/* CR101 */ +#define PHY_RDLVL_EDGE 24 +/* CR93 */ +#define SW_LVL_MODE 8 +/* CR94 */ +#define SWLVL_RESP_0 24 +/* CR95 */ +#define SWLVL_RESP_1 0 +/* CR105 */ +#define RDLVL_DL_O_OFF 8 +/* CR110 */ +#define RDLVL_DL_1_OFF 0 + +/** + * ddrmc_calibration - Vybrid's (VF610) DDR3 calibration code + * + * This function is calculating proper memory controller values + * during run time. + * + * @param ddrmr_regs - memory controller registers + * + * @return 0 on success, otherwise error code + */ +int ddrmc_calibration(struct ddrmr_regs *ddrmr); + +#endif /* __DDRMC_VF610_CALIBRATOIN_H_ */
On 02.12.2018 21:42, Lukasz Majewski wrote:
This patch provides the code to calibrate the DDR's DQS to DQ signals (RDLVL).
It is based on: VFxxx Controller Reference Manual, Rev. 0, 10/2016, page 1600 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode"
and NXP's community thread: "Vybrid: About DDR leveling feature on DDRMC." https://community.nxp.com/thread/395323
Signed-off-by: Lukasz Majewski lukma@denx.de
Thanks for looking into this! We actually tried to use the official DDRV tool, but were not able to get really good results with it!
In a quick test the calibration worked here on a Colibri VF50. Will do some more testing later this week.
Some minor things below:
arch/arm/mach-imx/Kconfig | 8 + arch/arm/mach-imx/Makefile | 1 + arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 ++++++++++++++++++++++++++++ arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ 4 files changed, 404 insertions(+) create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h
diff --git a/arch/arm/mach-imx/Kconfig b/arch/arm/mach-imx/Kconfig index a1566cc2ad..ca10ba683f 100644 --- a/arch/arm/mach-imx/Kconfig +++ b/arch/arm/mach-imx/Kconfig @@ -78,3 +78,11 @@ config NXP_BOARD_REVISION NXP boards based on i.MX6/7 contain the board revision information stored in the fuses. Select this option if you want to be able to retrieve the board revision information.
+config DDRMC_VF610_CALIBRATION
- bool "Enable DDRMC (DDR3) on-chip calibration"
- depends on ARCH_VF610
- help
Vybrid (vf610) SoC provides some on-chip facility to tune the DDR3memory parameters. Select this option if you want to calculate themat boot time.diff --git a/arch/arm/mach-imx/Makefile b/arch/arm/mach-imx/Makefile index 53d9e5f42b..4a07b1ea69 100644 --- a/arch/arm/mach-imx/Makefile +++ b/arch/arm/mach-imx/Makefile @@ -51,6 +51,7 @@ obj-$(CONFIG_SECURE_BOOT) += hab.o endif ifeq ($(SOC),$(filter $(SOC),vf610)) obj-y += ddrmc-vf610.o +obj-$(CONFIG_DDRMC_VF610_CALIBRATION) += ddrmc-vf610-calibration.o endif ifneq ($(CONFIG_SPL_BUILD),y) obj-$(CONFIG_CMD_BMODE) += cmd_bmode.o diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.c b/arch/arm/mach-imx/ddrmc-vf610-calibration.c new file mode 100644 index 0000000000..6ed3f5375b --- /dev/null +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.c @@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0+ +/*
- ddrmc DDR3 calibration code for NXP's VF610
- Copyright (C) 2018 DENX Software Engineering
- Lukasz Majewski, DENX Software Engineering, lukma@denx.de
- */
+/* #define DEBUG */ +#include <common.h> +#include <asm/io.h> +#include <asm/arch/imx-regs.h> +#include <linux/bitmap.h>
+#include "ddrmc-vf610-calibration.h"
+/*
- Documents:
- [1] "Vybrid: About DDR leveling feature on DDRMC."
- [2] VFxxx Controller Reference Manual, Rev. 0, 10/2016
- NOTE
- ====
- NXP recommends setting 'fixed' parameters instead of performing the
- training at each boot.
Can you also add this information to the Kconfig? That is what people usually read to decide whether to enable an option.
- Use those functions to determine those values on new HW and write
- default values to registers.
I guess the idea is to add it back to the board specific DDR settings? Can you be a bit more clear, e.g.
Use those functions to determine those values on new HW, read the calculated value from registers and add them to the board specific struct ddrmc_cr_setting.
- SW leveling supported operations - CR93[SW_LVL_MODE]:
- 0x0 (b'00) - No leveling
- 0x1 (b'01) - WRLVL_DL_X - It is not recommended to perform this tuning
on HW designs utilizing non-flyback topology
(Single DDR3 with x16).
Instead the WRLVL_DL_0/1 fields shall be set
based on trace length differences from their
layout.
Mismatches up to 25% or tCK (clock period) are
allowed, so the value in the filed doesn’t have
to be very accurate.
- 0x2 (b'10) - RDLVL_DL_0/1 - refers to adjusting the DQS strobe in relation
to the DQ signals so that the strobe edge is
centered in the window of valid read data.
- 0x3 (b'11) - RDLVL_GTDL_0/1 - refers to the delay the PHY uses to un-gate
the Read DQS strobe pad from the time that the
PHY enables the pad to input the strobe signal.
- */
+static int ddr_cal_get_first_edge_index(unsigned long *bmap, enum edge e,
int samples, int start, int max)+{
- int i, ret = -1;
- /*
* We look only for the first value (and filter out* some wrong data)*/- switch (e) {
- case RISING_EDGE:
for (i = start; i <= max - samples; i++) {if (test_bit(i, bmap)) {if (!test_bit(i - 1, bmap) &&test_bit(i + 1, bmap) &&test_bit(i + 2, bmap) &&test_bit(i + 3, bmap)) {return i;}}}break;- case FALLING_EDGE:
for (i = start; i <= max - samples; i++) {if (!test_bit(i, bmap)) {if (test_bit(i - 1, bmap) &&test_bit(i - 2, bmap) &&test_bit(i - 3, bmap)) {return i;}}}- }
- return ret;
+}
+static void bitmap_print(unsigned long *bmap, int max) +{
- int i;
- debug("BITMAP [0x%p]:\n", bmap);
- for (i = 0; i <= max; i++) {
debug("%d ", test_bit(i, bmap) ? 1 : 0);if (i && (i % 32) == (32 - 1))debug("\n");- }
- debug("\n");
+}
+#define sw_leveling_op_done \
- while (!(readl(&ddrmr->cr[94]) & DDRMC_CR94_SWLVL_OP_DONE))
+#define sw_leveling_load_value \
- do { clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SWLVL_LOAD, \
DDRMC_CR93_SWLVL_LOAD); } while (0)+#define sw_leveling_start \
- do { clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SWLVL_START, \
DDRMC_CR93_SWLVL_START); } while (0)+#define sw_leveling_exit \
- do { clrsetbits_le32(&ddrmr->cr[94], DDRMC_CR93_SWLVL_EXIT, \
DDRMC_CR93_SWLVL_EXIT); } while (0)+/*
- RDLVL_DL calibration:
- NXP is _NOT_ recommending performing the leveling at each
- boot. Instead - one shall run this procedure on new boards
- and then use hardcoded values.
- */
+static int ddrmc_cal_dqs_to_dq(struct ddrmr_regs *ddrmr) +{
- DECLARE_BITMAP(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1);
- int rdlvl_dl_0_min = -1, rdlvl_dl_0_max = -1;
- int rdlvl_dl_1_min = -1, rdlvl_dl_1_max = -1;
- int rdlvl_dl_0, rdlvl_dl_1;
- u8 swlvl_rsp;
- u32 tmp;
- int i;
- /* Read defaults */
- u16 rdlvl_dl_0_def =
(readl(&ddrmr->cr[105]) >> RDLVL_DL_O_OFF) & 0xFFFF;- u16 rdlvl_dl_1_def = readl(&ddrmr->cr[110]) & 0xFFFF;
- debug("\nRDLVL: ======================\n");
- debug("RDLVL: DQS to DQ (RDLVL)\n");
- debug("RDLVL: RDLVL_DL_0_DFL:\t 0x%x\n", rdlvl_dl_0_def);
- debug("RDLVL: RDLVL_DL_1_DFL:\t 0x%x\n", rdlvl_dl_1_def);
- /*
* Set/Read setup for calibration** Values necessary for leveling from Vybrid RM [2] - page 1600*/- writel(0x40703030, &ddrmr->cr[144]);
- writel(0x40, &ddrmr->cr[145]);
- writel(0x40, &ddrmr->cr[146]);
- tmp = readl(&ddrmr->cr[144]);
- debug("RDLVL: PHY_RDLVL_RES:\t 0x%x\n", (tmp >> 24) & 0xFF);// set 0x40
- debug("RDLVL: PHY_RDLV_LOAD:\t 0x%x\n", (tmp >> 16) & 0xFF);// set 0x70
- debug("RDLVL: PHY_RDLV_DLL:\t 0x%x\n", (tmp >> 8) & 0xFF); // set 0x30
- debug("RDLVL: PHY_RDLV_EN:\t 0x%x\n", tmp & 0xFF); //set 0x30
- tmp = readl(&ddrmr->cr[145]);
- debug("RDLVL: PHY_RDLV_RR:\t 0x%x\n", tmp & 0x3FF); //set 0x40
- tmp = readl(&ddrmr->cr[146]);
- debug("RDLVL: PHY_RDLV_RESP:\t 0x%x\n", tmp); //set 0x40
- /*
* Program/read the leveling edge RDLVL_EDGE = 0** 0x00 is the correct output on SWLVL_RSP_X* If by any chance 1s are visible -> wrong number read*/- clrbits_le32(&ddrmr->cr[101], DDRMC_CR101_PHY_RDLVL_EDGE);
- tmp = readl(&ddrmr->cr[101]);
- debug("RDLVL: PHY_RDLVL_EDGE:\t 0x%x\n",
(tmp >> PHY_RDLVL_EDGE) & 0x1); //set 0- /* Program Leveling mode - CR93[SW_LVL_MODE] to ’b10 */
- clrsetbits_le32(&ddrmr->cr[93], DDRMC_CR93_SW_LVL_MODE(0x3),
DDRMC_CR93_SW_LVL_MODE(0x2));- tmp = readl(&ddrmr->cr[93]);
- debug("RDLVL: SW_LVL_MODE:\t 0x%x\n", (tmp >> SW_LVL_MODE) & 0x3);
- /* Start procedure - CR93[SWLVL_START] to ’b1 */
- sw_leveling_start;
- /* Poll CR94[SWLVL_OP_DONE] */
- sw_leveling_op_done;
- /*
* Program delays for RDLVL_DL_0** The procedure is to increase the delay values from 0 to 0xFF* and read the response from the DDRMC*/- debug("\nRDLVL: ---> RDLVL_DL_0\n");
- bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1);
- for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) {
clrsetbits_le32(&ddrmr->cr[105], 0xFFFF << RDLVL_DL_O_OFF,i << RDLVL_DL_O_OFF);/* Load values CR93[SWLVL_LOAD] to ’b1 */sw_leveling_load_value;/* Poll CR94[SWLVL_OP_DONE] */sw_leveling_op_done;/** Read Responses - SWLVL_RESP_0** The 0x00 (correct response when PHY_RDLVL_EDGE = 0)* -> 1 in the bit vector*/swlvl_rsp = (readl(&ddrmr->cr[94]) >> SWLVL_RESP_0) & 0xF;if (swlvl_rsp == 0)generic_set_bit(i, rdlvl_rsp);- }
- bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY);
- /*
* First test for rising edge 0x0 -> 0x1 in bitmap*/- rdlvl_dl_0_min = ddr_cal_get_first_edge_index(rdlvl_rsp, RISING_EDGE,
N_SAMPLES, N_SAMPLES,DDRMC_DQS_DQ_MAX_DELAY);- /*
* Secondly test for falling edge 0x1 -> 0x0 in bitmap*/- rdlvl_dl_0_max = ddr_cal_get_first_edge_index(rdlvl_rsp, FALLING_EDGE,
N_SAMPLES, rdlvl_dl_0_min,DDRMC_DQS_DQ_MAX_DELAY);- debug("RDLVL: DL_0 min: %d [0x%x] DL_0 max: %d [0x%x]\n",
rdlvl_dl_0_min, rdlvl_dl_0_min, rdlvl_dl_0_max, rdlvl_dl_0_max);- rdlvl_dl_0 = (rdlvl_dl_0_max - rdlvl_dl_0_min) / 2;
- if (rdlvl_dl_0_max == -1 || rdlvl_dl_0_min == -1 || rdlvl_dl_0 <= 0) {
debug("RDLVL: The DQS to DQ delay cannot be found!\n");debug("RDLVL: Using default - slice 0: %d!\n", rdlvl_dl_0_def);rdlvl_dl_0 = rdlvl_dl_0_def;- }
- debug("\nRDLVL: ---> RDLVL_DL_1\n");
- bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1);
- for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) {
clrsetbits_le32(&ddrmr->cr[110], 0xFFFF << RDLVL_DL_1_OFF,i << RDLVL_DL_1_OFF);/* Load values CR93[SWLVL_LOAD] to ’b1 */sw_leveling_load_value;/* Poll CR94[SWLVL_OP_DONE] */sw_leveling_op_done;/** Read Responses - SWLVL_RESP_1** The 0x00 (correct response when PHY_RDLVL_EDGE = 0)* -> 1 in the bit vector*/swlvl_rsp = (readl(&ddrmr->cr[95]) >> SWLVL_RESP_1) & 0xF;if (swlvl_rsp == 0)generic_set_bit(i, rdlvl_rsp);- }
- bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY);
- /*
* First test for rising edge 0x0 -> 0x1 in bitmap*/- rdlvl_dl_1_min = ddr_cal_get_first_edge_index(rdlvl_rsp, RISING_EDGE,
N_SAMPLES, N_SAMPLES,DDRMC_DQS_DQ_MAX_DELAY);- /*
* Secondly test for falling edge 0x1 -> 0x0 in bitmap*/- rdlvl_dl_1_max = ddr_cal_get_first_edge_index(rdlvl_rsp, FALLING_EDGE,
N_SAMPLES, rdlvl_dl_1_min,DDRMC_DQS_DQ_MAX_DELAY);- debug("RDLVL: DL_1 min: %d [0x%x] DL_1 max: %d [0x%x]\n",
rdlvl_dl_1_min, rdlvl_dl_1_min, rdlvl_dl_1_max, rdlvl_dl_1_max);- rdlvl_dl_1 = (rdlvl_dl_1_max - rdlvl_dl_1_min) / 2;
- if (rdlvl_dl_1_max == -1 || rdlvl_dl_1_min == -1 || rdlvl_dl_1 <= 0) {
debug("RDLVL: The DQS to DQ delay cannot be found!\n");debug("RDLVL: Using default - slice 1: %d!\n", rdlvl_dl_1_def);rdlvl_dl_1 = rdlvl_dl_1_def;- }
- debug("RDLVL: CALIBRATED: rdlvl_dl_0: 0x%x\t rdlvl_dl_1: 0x%x\n",
rdlvl_dl_0, rdlvl_dl_1);- /* Write new delay values */
- writel((rdlvl_dl_0 << RDLVL_DL_O_OFF), &ddrmr->cr[105]);
- writel((rdlvl_dl_1 << RDLVL_DL_1_OFF), &ddrmr->cr[110]);
- sw_leveling_load_value;
- sw_leveling_op_done;
- /* Exit procedure - CR94[SWLVL_EXIT] to ’b1 */
- sw_leveling_exit;
- /* Poll CR94[SWLVL_OP_DONE] */
- sw_leveling_op_done;
- return 0;
+}
+/*
- WRLVL_DL calibration:
- For non-flyback memory architecture - where one have a single DDR3 x16
- memory - it is NOT necessary to perform "Write Leveling"
- [3] 'Vybrid DDR3 write leveling' https://community.nxp.com/thread/429362
- */
+int ddrmc_calibration(struct ddrmr_regs *ddrmr) +{
- ddrmc_cal_dqs_to_dq(ddrmr);
- return 0;
+} diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.h b/arch/arm/mach-imx/ddrmc-vf610-calibration.h new file mode 100644 index 0000000000..7bc53236be --- /dev/null +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/*
- ddrmc DDR3 calibration code for NXP's VF610
- Copyright (C) 2018 DENX Software Engineering
- Lukasz Majewski, DENX Software Engineering, lukma@denx.de
- */
+#ifndef __DDRMC_VF610_CALIBRATOIN_H_ +#define __DDRMC_VF610_CALIBRATOIN_H_
+/*
- Number of "samples" in the calibration bitmap
- to be considered during calibration.
- */
+#define N_SAMPLES 3
+/*
- Constants to indicate if we are looking for a rising or
- falling edge in the calibration bitmap
- */
+enum edge {
- FALLING_EDGE = 1,
- RISING_EDGE
+};
+/*
- The max number of delay elements when DQS to DQ setting
- */
+#define DDRMC_DQS_DQ_MAX_DELAY 0xFF
+/* Bits offsets for DDRMC_CR registers */ +/* CR101 */ +#define PHY_RDLVL_EDGE 24 +/* CR93 */ +#define SW_LVL_MODE 8 +/* CR94 */ +#define SWLVL_RESP_0 24 +/* CR95 */ +#define SWLVL_RESP_1 0 +/* CR105 */ +#define RDLVL_DL_O_OFF 8 +/* CR110 */ +#define RDLVL_DL_1_OFF 0
Should we not move those defines to arch/arm/include/asm/arch-vf610/imx-regs.h too?
-- Stefan
+/**
- ddrmc_calibration - Vybrid's (VF610) DDR3 calibration code
- This function is calculating proper memory controller values
- during run time.
- @param ddrmr_regs - memory controller registers
- @return 0 on success, otherwise error code
- */
+int ddrmc_calibration(struct ddrmr_regs *ddrmr);
+#endif /* __DDRMC_VF610_CALIBRATOIN_H_ */
Hi Stefan,
On 02.12.2018 21:42, Lukasz Majewski wrote:
This patch provides the code to calibrate the DDR's DQS to DQ signals (RDLVL).
It is based on: VFxxx Controller Reference Manual, Rev. 0, 10/2016, page 1600 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode"
and NXP's community thread: "Vybrid: About DDR leveling feature on DDRMC." https://community.nxp.com/thread/395323
Signed-off-by: Lukasz Majewski lukma@denx.de
Thanks for looking into this! We actually tried to use the official DDRV tool, but were not able to get really good results with it!
I was only able to use the SW on-chip calibration.
In a quick test the calibration worked here on a Colibri VF50. Will do some more testing later this week.
There is also ongoing discussion regarding the gate training, on which I've encountered some issues:
https://community.nxp.com/thread/490391
Some minor things below:
arch/arm/mach-imx/Kconfig | 8 + arch/arm/mach-imx/Makefile | 1 + arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 ++++++++++++++++++++++++++++ arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ 4 files changed, 404 insertions(+) create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h
diff --git a/arch/arm/mach-imx/Kconfig b/arch/arm/mach-imx/Kconfig index a1566cc2ad..ca10ba683f 100644 --- a/arch/arm/mach-imx/Kconfig +++ b/arch/arm/mach-imx/Kconfig @@ -78,3 +78,11 @@ config NXP_BOARD_REVISION NXP boards based on i.MX6/7 contain the board revision information stored in the fuses. Select this option if you want to be able to retrieve the board revision information.
+config DDRMC_VF610_CALIBRATION
- bool "Enable DDRMC (DDR3) on-chip calibration"
- depends on ARCH_VF610
- help
Vybrid (vf610) SoC provides some on-chip facility totune the DDR3
memory parameters. Select this option if you want tocalculate them
at boot time.diff --git a/arch/arm/mach-imx/Makefile b/arch/arm/mach-imx/Makefile index 53d9e5f42b..4a07b1ea69 100644 --- a/arch/arm/mach-imx/Makefile +++ b/arch/arm/mach-imx/Makefile @@ -51,6 +51,7 @@ obj-$(CONFIG_SECURE_BOOT) += hab.o endif ifeq ($(SOC),$(filter $(SOC),vf610)) obj-y += ddrmc-vf610.o +obj-$(CONFIG_DDRMC_VF610_CALIBRATION) += ddrmc-vf610-calibration.o endif ifneq ($(CONFIG_SPL_BUILD),y) obj-$(CONFIG_CMD_BMODE) += cmd_bmode.o diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.c b/arch/arm/mach-imx/ddrmc-vf610-calibration.c new file mode 100644 index 0000000000..6ed3f5375b --- /dev/null +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.c @@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0+ +/*
- ddrmc DDR3 calibration code for NXP's VF610
- Copyright (C) 2018 DENX Software Engineering
- Lukasz Majewski, DENX Software Engineering, lukma@denx.de
- */
+/* #define DEBUG */ +#include <common.h> +#include <asm/io.h> +#include <asm/arch/imx-regs.h> +#include <linux/bitmap.h>
+#include "ddrmc-vf610-calibration.h"
+/*
- Documents:
- [1] "Vybrid: About DDR leveling feature on DDRMC."
- [2] VFxxx Controller Reference Manual, Rev. 0, 10/2016
- NOTE
- ====
- NXP recommends setting 'fixed' parameters instead of performing
the
- training at each boot.
Can you also add this information to the Kconfig? That is what people usually read to decide whether to enable an option.
Yes, I will add it. Accoridng to NXP, this code shall be only run on board during validation, not production. And due to "simple" DDR3 connection on most boards (single x16 DDR IC), this should be enough to get it reliably working.
- Use those functions to determine those values on new HW and
write
- default values to registers.
I guess the idea is to add it back to the board specific DDR settings? Can you be a bit more clear, e.g.
Use those functions to determine those values on new HW, read the calculated value from registers and add them to the board specific struct ddrmc_cr_setting.
Ok. I will add such "big, fat note".
- SW leveling supported operations - CR93[SW_LVL_MODE]:
- 0x0 (b'00) - No leveling
- 0x1 (b'01) - WRLVL_DL_X - It is not recommended to perform
this tuning
on HW designs utilizing non-flybacktopology
(Single DDR3 with x16).
Instead the WRLVL_DL_0/1 fieldsshall be set
based on trace length differencesfrom their
layout.
Mismatches up to 25% or tCK (clockperiod) are
allowed, so the value in the fileddoesn’t have
to be very accurate.
- 0x2 (b'10) - RDLVL_DL_0/1 - refers to adjusting the DQS
strobe in relation
to the DQ signals so that thestrobe edge is
centered in the window of validread data.
- 0x3 (b'11) - RDLVL_GTDL_0/1 - refers to the delay the PHY
uses to un-gate
the Read DQS strobe pad from thetime that the
PHY enables the pad to input thestrobe signal.
- */
+static int ddr_cal_get_first_edge_index(unsigned long *bmap, enum edge e,
int samples, int start,int max) +{
- int i, ret = -1;
- /*
* We look only for the first value (and filter out* some wrong data)*/- switch (e) {
- case RISING_EDGE:
for (i = start; i <= max - samples; i++) {if (test_bit(i, bmap)) {if (!test_bit(i - 1, bmap) &&test_bit(i + 1, bmap) &&test_bit(i + 2, bmap) &&test_bit(i + 3, bmap)) {return i;}}}break;- case FALLING_EDGE:
for (i = start; i <= max - samples; i++) {if (!test_bit(i, bmap)) {if (test_bit(i - 1, bmap) &&test_bit(i - 2, bmap) &&test_bit(i - 3, bmap)) {return i;}}}- }
- return ret;
+}
+static void bitmap_print(unsigned long *bmap, int max) +{
- int i;
- debug("BITMAP [0x%p]:\n", bmap);
- for (i = 0; i <= max; i++) {
debug("%d ", test_bit(i, bmap) ? 1 : 0);if (i && (i % 32) == (32 - 1))debug("\n");- }
- debug("\n");
+}
+#define sw_leveling_op_done \
- while (!(readl(&ddrmr->cr[94]) & DDRMC_CR94_SWLVL_OP_DONE))
+#define sw_leveling_load_value \
- do { clrsetbits_le32(&ddrmr->cr[93],
DDRMC_CR93_SWLVL_LOAD, \
DDRMC_CR93_SWLVL_LOAD); } while (0)+#define sw_leveling_start \
- do { clrsetbits_le32(&ddrmr->cr[93],
DDRMC_CR93_SWLVL_START, \
DDRMC_CR93_SWLVL_START); } while (0)+#define sw_leveling_exit \
- do { clrsetbits_le32(&ddrmr->cr[94],
DDRMC_CR93_SWLVL_EXIT, \
DDRMC_CR93_SWLVL_EXIT); } while (0)+/*
- RDLVL_DL calibration:
- NXP is _NOT_ recommending performing the leveling at each
- boot. Instead - one shall run this procedure on new boards
- and then use hardcoded values.
- */
+static int ddrmc_cal_dqs_to_dq(struct ddrmr_regs *ddrmr) +{
- DECLARE_BITMAP(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1);
- int rdlvl_dl_0_min = -1, rdlvl_dl_0_max = -1;
- int rdlvl_dl_1_min = -1, rdlvl_dl_1_max = -1;
- int rdlvl_dl_0, rdlvl_dl_1;
- u8 swlvl_rsp;
- u32 tmp;
- int i;
- /* Read defaults */
- u16 rdlvl_dl_0_def =
(readl(&ddrmr->cr[105]) >> RDLVL_DL_O_OFF) &0xFFFF;
- u16 rdlvl_dl_1_def = readl(&ddrmr->cr[110]) & 0xFFFF;
- debug("\nRDLVL: ======================\n");
- debug("RDLVL: DQS to DQ (RDLVL)\n");
- debug("RDLVL: RDLVL_DL_0_DFL:\t 0x%x\n", rdlvl_dl_0_def);
- debug("RDLVL: RDLVL_DL_1_DFL:\t 0x%x\n", rdlvl_dl_1_def);
- /*
* Set/Read setup for calibration** Values necessary for leveling from Vybrid RM [2] - page1600
*/- writel(0x40703030, &ddrmr->cr[144]);
- writel(0x40, &ddrmr->cr[145]);
- writel(0x40, &ddrmr->cr[146]);
- tmp = readl(&ddrmr->cr[144]);
- debug("RDLVL: PHY_RDLVL_RES:\t 0x%x\n", (tmp >> 24) &
0xFF);// set 0x40
- debug("RDLVL: PHY_RDLV_LOAD:\t 0x%x\n", (tmp >> 16) &
0xFF);// set 0x70
- debug("RDLVL: PHY_RDLV_DLL:\t 0x%x\n", (tmp >> 8) &
0xFF); // set 0x30
- debug("RDLVL: PHY_RDLV_EN:\t 0x%x\n", tmp & 0xFF); //set
0x30 +
- tmp = readl(&ddrmr->cr[145]);
- debug("RDLVL: PHY_RDLV_RR:\t 0x%x\n", tmp & 0x3FF); //set
0x40 +
- tmp = readl(&ddrmr->cr[146]);
- debug("RDLVL: PHY_RDLV_RESP:\t 0x%x\n", tmp); //set 0x40
- /*
* Program/read the leveling edge RDLVL_EDGE = 0** 0x00 is the correct output on SWLVL_RSP_X* If by any chance 1s are visible -> wrong number read*/- clrbits_le32(&ddrmr->cr[101], DDRMC_CR101_PHY_RDLVL_EDGE);
- tmp = readl(&ddrmr->cr[101]);
- debug("RDLVL: PHY_RDLVL_EDGE:\t 0x%x\n",
(tmp >> PHY_RDLVL_EDGE) & 0x1); //set 0- /* Program Leveling mode - CR93[SW_LVL_MODE] to ’b10 */
- clrsetbits_le32(&ddrmr->cr[93],
DDRMC_CR93_SW_LVL_MODE(0x3),
DDRMC_CR93_SW_LVL_MODE(0x2));- tmp = readl(&ddrmr->cr[93]);
- debug("RDLVL: SW_LVL_MODE:\t 0x%x\n", (tmp >> SW_LVL_MODE)
& 0x3); +
- /* Start procedure - CR93[SWLVL_START] to ’b1 */
- sw_leveling_start;
- /* Poll CR94[SWLVL_OP_DONE] */
- sw_leveling_op_done;
- /*
* Program delays for RDLVL_DL_0** The procedure is to increase the delay values from 0 to0xFF
* and read the response from the DDRMC*/- debug("\nRDLVL: ---> RDLVL_DL_0\n");
- bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1);
- for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) {
clrsetbits_le32(&ddrmr->cr[105], 0xFFFF <<RDLVL_DL_O_OFF,
i << RDLVL_DL_O_OFF);/* Load values CR93[SWLVL_LOAD] to ’b1 */sw_leveling_load_value;/* Poll CR94[SWLVL_OP_DONE] */sw_leveling_op_done;/** Read Responses - SWLVL_RESP_0** The 0x00 (correct response when PHY_RDLVL_EDGE= 0)
* -> 1 in the bit vector*/swlvl_rsp = (readl(&ddrmr->cr[94]) >>SWLVL_RESP_0) & 0xF;
if (swlvl_rsp == 0)generic_set_bit(i, rdlvl_rsp);- }
- bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY);
- /*
* First test for rising edge 0x0 -> 0x1 in bitmap*/- rdlvl_dl_0_min = ddr_cal_get_first_edge_index(rdlvl_rsp,
RISING_EDGE,
N_SAMPLES,N_SAMPLES,
DDRMC_DQS_DQ_MAX_DELAY); +
- /*
* Secondly test for falling edge 0x1 -> 0x0 in bitmap*/- rdlvl_dl_0_max = ddr_cal_get_first_edge_index(rdlvl_rsp,
FALLING_EDGE,
N_SAMPLES,rdlvl_dl_0_min,
DDRMC_DQS_DQ_MAX_DELAY); +
- debug("RDLVL: DL_0 min: %d [0x%x] DL_0 max: %d [0x%x]\n",
rdlvl_dl_0_min, rdlvl_dl_0_min, rdlvl_dl_0_max,rdlvl_dl_0_max);
- rdlvl_dl_0 = (rdlvl_dl_0_max - rdlvl_dl_0_min) / 2;
- if (rdlvl_dl_0_max == -1 || rdlvl_dl_0_min == -1 ||
rdlvl_dl_0 <= 0) {
debug("RDLVL: The DQS to DQ delay cannot befound!\n");
debug("RDLVL: Using default - slice 0: %d!\n",rdlvl_dl_0_def);
rdlvl_dl_0 = rdlvl_dl_0_def;- }
- debug("\nRDLVL: ---> RDLVL_DL_1\n");
- bitmap_zero(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY + 1);
- for (i = 0; i <= DDRMC_DQS_DQ_MAX_DELAY; i++) {
clrsetbits_le32(&ddrmr->cr[110], 0xFFFF <<RDLVL_DL_1_OFF,
i << RDLVL_DL_1_OFF);/* Load values CR93[SWLVL_LOAD] to ’b1 */sw_leveling_load_value;/* Poll CR94[SWLVL_OP_DONE] */sw_leveling_op_done;/** Read Responses - SWLVL_RESP_1** The 0x00 (correct response when PHY_RDLVL_EDGE= 0)
* -> 1 in the bit vector*/swlvl_rsp = (readl(&ddrmr->cr[95]) >>SWLVL_RESP_1) & 0xF;
if (swlvl_rsp == 0)generic_set_bit(i, rdlvl_rsp);- }
- bitmap_print(rdlvl_rsp, DDRMC_DQS_DQ_MAX_DELAY);
- /*
* First test for rising edge 0x0 -> 0x1 in bitmap*/- rdlvl_dl_1_min = ddr_cal_get_first_edge_index(rdlvl_rsp,
RISING_EDGE,
N_SAMPLES,N_SAMPLES,
DDRMC_DQS_DQ_MAX_DELAY); +
- /*
* Secondly test for falling edge 0x1 -> 0x0 in bitmap*/- rdlvl_dl_1_max = ddr_cal_get_first_edge_index(rdlvl_rsp,
FALLING_EDGE,
N_SAMPLES,rdlvl_dl_1_min,
DDRMC_DQS_DQ_MAX_DELAY); +
- debug("RDLVL: DL_1 min: %d [0x%x] DL_1 max: %d [0x%x]\n",
rdlvl_dl_1_min, rdlvl_dl_1_min, rdlvl_dl_1_max,rdlvl_dl_1_max);
- rdlvl_dl_1 = (rdlvl_dl_1_max - rdlvl_dl_1_min) / 2;
- if (rdlvl_dl_1_max == -1 || rdlvl_dl_1_min == -1 ||
rdlvl_dl_1 <= 0) {
debug("RDLVL: The DQS to DQ delay cannot befound!\n");
debug("RDLVL: Using default - slice 1: %d!\n",rdlvl_dl_1_def);
rdlvl_dl_1 = rdlvl_dl_1_def;- }
- debug("RDLVL: CALIBRATED: rdlvl_dl_0: 0x%x\t rdlvl_dl_1:
0x%x\n",
rdlvl_dl_0, rdlvl_dl_1);- /* Write new delay values */
- writel((rdlvl_dl_0 << RDLVL_DL_O_OFF), &ddrmr->cr[105]);
- writel((rdlvl_dl_1 << RDLVL_DL_1_OFF), &ddrmr->cr[110]);
- sw_leveling_load_value;
- sw_leveling_op_done;
- /* Exit procedure - CR94[SWLVL_EXIT] to ’b1 */
- sw_leveling_exit;
- /* Poll CR94[SWLVL_OP_DONE] */
- sw_leveling_op_done;
- return 0;
+}
+/*
- WRLVL_DL calibration:
- For non-flyback memory architecture - where one have a single
DDR3 x16
- memory - it is NOT necessary to perform "Write Leveling"
- [3] 'Vybrid DDR3 write leveling'
https://community.nxp.com/thread/429362
- */
+int ddrmc_calibration(struct ddrmr_regs *ddrmr) +{
- ddrmc_cal_dqs_to_dq(ddrmr);
- return 0;
+} diff --git a/arch/arm/mach-imx/ddrmc-vf610-calibration.h b/arch/arm/mach-imx/ddrmc-vf610-calibration.h new file mode 100644 index 0000000000..7bc53236be --- /dev/null +++ b/arch/arm/mach-imx/ddrmc-vf610-calibration.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/*
- ddrmc DDR3 calibration code for NXP's VF610
- Copyright (C) 2018 DENX Software Engineering
- Lukasz Majewski, DENX Software Engineering, lukma@denx.de
- */
+#ifndef __DDRMC_VF610_CALIBRATOIN_H_ +#define __DDRMC_VF610_CALIBRATOIN_H_
+/*
- Number of "samples" in the calibration bitmap
- to be considered during calibration.
- */
+#define N_SAMPLES 3
+/*
- Constants to indicate if we are looking for a rising or
- falling edge in the calibration bitmap
- */
+enum edge {
- FALLING_EDGE = 1,
- RISING_EDGE
+};
+/*
- The max number of delay elements when DQS to DQ setting
- */
+#define DDRMC_DQS_DQ_MAX_DELAY 0xFF
+/* Bits offsets for DDRMC_CR registers */ +/* CR101 */ +#define PHY_RDLVL_EDGE 24 +/* CR93 */ +#define SW_LVL_MODE 8 +/* CR94 */ +#define SWLVL_RESP_0 24 +/* CR95 */ +#define SWLVL_RESP_1 0 +/* CR105 */ +#define RDLVL_DL_O_OFF 8 +/* CR110 */ +#define RDLVL_DL_1_OFF 0
Should we not move those defines to arch/arm/include/asm/arch-vf610/imx-regs.h too?
Ok, I will add them to imx-regs.h
Thanks for the review.
-- Stefan
+/**
- ddrmc_calibration - Vybrid's (VF610) DDR3 calibration code
- This function is calculating proper memory controller values
- during run time.
- @param ddrmr_regs - memory controller registers
- @return 0 on success, otherwise error code
- */
+int ddrmc_calibration(struct ddrmr_regs *ddrmr);
+#endif /* __DDRMC_VF610_CALIBRATOIN_H_ */
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
This patch extends the vf610 DDR memory controller code to support SW leveling.
Signed-off-by: Lukasz Majewski lukma@denx.de
---
arch/arm/mach-imx/ddrmc-vf610.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/arch/arm/mach-imx/ddrmc-vf610.c b/arch/arm/mach-imx/ddrmc-vf610.c index ea6a49e0fa..8474023fae 100644 --- a/arch/arm/mach-imx/ddrmc-vf610.c +++ b/arch/arm/mach-imx/ddrmc-vf610.c @@ -10,6 +10,9 @@ #include <asm/arch/imx-regs.h> #include <asm/arch/iomux-vf610.h> #include <asm/arch/ddrmc-vf610.h> +#ifdef CONFIG_DDRMC_VF610_CALIBRATION +#include "ddrmc-vf610-calibration.h" +#endif
void ddrmc_setup_iomux(const iomux_v3_cfg_t *pads, int pads_count) { @@ -233,4 +236,8 @@ void ddrmc_ctrl_init_ddr3(struct ddr3_jedec_timings const *timings,
while (!(readl(&ddrmr->cr[80]) & 0x100)) udelay(10); + +#ifdef CONFIG_DDRMC_VF610_CALIBRATION + ddrmc_calibration(ddrmr); +#endif }
On 02.12.2018 21:42, Lukasz Majewski wrote:
This patch extends the vf610 DDR memory controller code to support SW leveling.
Signed-off-by: Lukasz Majewski lukma@denx.de
arch/arm/mach-imx/ddrmc-vf610.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/arch/arm/mach-imx/ddrmc-vf610.c b/arch/arm/mach-imx/ddrmc-vf610.c index ea6a49e0fa..8474023fae 100644 --- a/arch/arm/mach-imx/ddrmc-vf610.c +++ b/arch/arm/mach-imx/ddrmc-vf610.c @@ -10,6 +10,9 @@ #include <asm/arch/imx-regs.h> #include <asm/arch/iomux-vf610.h> #include <asm/arch/ddrmc-vf610.h> +#ifdef CONFIG_DDRMC_VF610_CALIBRATION +#include "ddrmc-vf610-calibration.h" +#endif
Is this ifdef needed? I think it should be fine to always include...
Otherwise:
Reviewed-by: Stefan Agner stefan.agner@toradex.com
-- Stefan
void ddrmc_setup_iomux(const iomux_v3_cfg_t *pads, int pads_count) { @@ -233,4 +236,8 @@ void ddrmc_ctrl_init_ddr3(struct ddr3_jedec_timings const *timings,
while (!(readl(&ddrmr->cr[80]) & 0x100)) udelay(10);
+#ifdef CONFIG_DDRMC_VF610_CALIBRATION
- ddrmc_calibration(ddrmr);
+#endif }
On Mon, 03 Dec 2018 16:55:13 +0100 Stefan Agner stefan@agner.ch wrote:
On 02.12.2018 21:42, Lukasz Majewski wrote:
This patch extends the vf610 DDR memory controller code to support SW leveling.
Signed-off-by: Lukasz Majewski lukma@denx.de
arch/arm/mach-imx/ddrmc-vf610.c | 7 +++++++ 1 file changed, 7 insertions(+)
diff --git a/arch/arm/mach-imx/ddrmc-vf610.c b/arch/arm/mach-imx/ddrmc-vf610.c index ea6a49e0fa..8474023fae 100644 --- a/arch/arm/mach-imx/ddrmc-vf610.c +++ b/arch/arm/mach-imx/ddrmc-vf610.c @@ -10,6 +10,9 @@ #include <asm/arch/imx-regs.h> #include <asm/arch/iomux-vf610.h> #include <asm/arch/ddrmc-vf610.h> +#ifdef CONFIG_DDRMC_VF610_CALIBRATION +#include "ddrmc-vf610-calibration.h" +#endif
Is this ifdef needed? I think it should be fine to always include...
Ok, I will remove it - indeed it is not necessary.
Otherwise:
Reviewed-by: Stefan Agner stefan.agner@toradex.com
-- Stefan
void ddrmc_setup_iomux(const iomux_v3_cfg_t *pads, int pads_count) { @@ -233,4 +236,8 @@ void ddrmc_ctrl_init_ddr3(struct ddr3_jedec_timings const *timings,
while (!(readl(&ddrmr->cr[80]) & 0x100)) udelay(10);
+#ifdef CONFIG_DDRMC_VF610_CALIBRATION
- ddrmc_calibration(ddrmr);
+#endif }
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
[Adding Stefan and Marcel in case they could help reviewing/testing this series]
On Sun, Dec 2, 2018 at 6:42 PM Lukasz Majewski lukma@denx.de wrote:
This patch series provides code to perform read leveling - RDLVL, which is adjusting the DQS strobe in relation to the DQ signals so that the strobe edge is centered in the window of valid read data.
The code is based on Vybrid's Reference Manual's: "VFxxx Controller Reference Manual, Rev. 0, 10/2016", page 1600, 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode"
and uses clarification provided by following NXP's community thread: "Vybrid: About DDR leveling feature on DDRMC." https://community.nxp.com/thread/395323
It depends on a BITMAP rework patch: usb: composite: Move bitmap related operations to ./include/linux/bitmap.h http://patchwork.ozlabs.org/patch/1006448/
Lukasz Majewski (3): ddr: vybrid: Add DDRMC calibration related registers (DQS to DQ) ddr: vybrid: Provide code to perform on-boot calibration ddr: vybrid: Add calibration code to memory controler's (DDRMC) setup code
arch/arm/include/asm/arch-vf610/imx-regs.h | 6 + arch/arm/mach-imx/Kconfig | 8 + arch/arm/mach-imx/Makefile | 1 + arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 ++++++++++++++++++++++++++++ arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ arch/arm/mach-imx/ddrmc-vf610.c | 7 + 6 files changed, 417 insertions(+) create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h
-- 2.11.0
Hi Fabio,
[Adding Stefan and Marcel in case they could help reviewing/testing this series]
There is also a started thread regarding GTDL: https://community.nxp.com/thread/490391
On which I'm now also working.
On Sun, Dec 2, 2018 at 6:42 PM Lukasz Majewski lukma@denx.de wrote:
This patch series provides code to perform read leveling - RDLVL, which is adjusting the DQS strobe in relation to the DQ signals so that the strobe edge is centered in the window of valid read data.
The code is based on Vybrid's Reference Manual's: "VFxxx Controller Reference Manual, Rev. 0, 10/2016", page 1600, 10.1.6.16.4.1 "Software Read Leveling in MC Evaluation Mode"
and uses clarification provided by following NXP's community thread: "Vybrid: About DDR leveling feature on DDRMC." https://community.nxp.com/thread/395323
It depends on a BITMAP rework patch: usb: composite: Move bitmap related operations to ./include/linux/bitmap.h http://patchwork.ozlabs.org/patch/1006448/
Lukasz Majewski (3): ddr: vybrid: Add DDRMC calibration related registers (DQS to DQ) ddr: vybrid: Provide code to perform on-boot calibration ddr: vybrid: Add calibration code to memory controler's (DDRMC) setup code
arch/arm/include/asm/arch-vf610/imx-regs.h | 6 + arch/arm/mach-imx/Kconfig | 8 + arch/arm/mach-imx/Makefile | 1 + arch/arm/mach-imx/ddrmc-vf610-calibration.c | 336 ++++++++++++++++++++++++++++ arch/arm/mach-imx/ddrmc-vf610-calibration.h | 59 +++++ arch/arm/mach-imx/ddrmc-vf610.c | 7 + 6 files changed, 417 insertions(+) create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.c create mode 100644 arch/arm/mach-imx/ddrmc-vf610-calibration.h
-- 2.11.0
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
participants (3)
-
Fabio Estevam -
Lukasz Majewski -
Stefan Agner