On Sun, Apr 03, 2016 at 07:11:21PM +0300, Siarhei Siamashka wrote:
Hi Piotr,
Hi Siarhei, thanks for comprehensive reply.
Other question is how to proceed from this point ? I have datasheet for DDR3, but number of possible parameters to tweak seems to be overwhelming. Also as sunxi wiki claim timing from datasheet have to be converted to number of cycles
- I'm not sure how to approach that. Is there any guide where to start ?
Well, the standard JEDEC timings are already tabulated in
http://git.denx.de/?p=u-boot.git;a=blob;f=board/sunxi/dram_timings_sun4i.hSome DRAM chips may have better timings than mandated by the standard, but this is more like performance tuning. Either way, these timings are a "digital" thing, they specify how many cycles to wait between sending different commands to the DRAM chip. And they are rather unlikely to be related to your reliability problems.
It is much more likely that you have "analog" problems and the signal gets distorted too badly on the way from the DRAM controller in the SoC to the DRAM chips. From what I heard, the DDR3 tracks routing is a major PITA, one needs to get both the tracks length and the tracks impedance matched within certain tolerance levels. As you are dealing with a custom hardware design, then maybe the DDR3 routing is a little bit screwed up?
You're right. I think this is the reason, because of that first thing I tried when I saw corrupted serial output was decreasing DRAM_CLK.
The DRAM controller configuration registers allow to configure the delays and drive/termination impedance to correct the PCB imperfection to some extent. In practice, all the "professionally" designed Allwinner devices work with the same generic Allwinner blessed DRAM settings, because the PCB routing is apparently usually done and tested to work fine with these default settings. Also reducing the DRAM clock speed usually improves reliability. It was believed that any A10/A13/A20 device should work stable if we clock the DRAM down to 360MHz.
On the hardware side, you can maybe check the ZQ and RZQ resistors (one connected to the SoC and also one extra resistor for each DRAM chip). They should at least exist and be preferably 240 ohm. Some board manufacturers tune the nominal value of these resistors, apparently as a way to fine tune the reliability. That's the configuration knob that is easily available to board manufacturers (they don't dare to touch the DRAM settings in U-boot and probably don't know how to do this).
I have schematics and confirmed that there is 240 ohm (%1 tolerance) resistor although I cannot be sure about its quality. So there are resistors on ZQ from DRAM die an SZQ (AA7) from SoC side. If I will get to wall I will probably check if those have good enough quality and maybe replace it. Is there any guide what can be used or I should just try slightly higher/lower and see if situation change ?
One more thing is the VDD-DLL voltage. It is served from DCDC3 (AXP209 PMIC) and is usually set to 1.250V by default. Increasing this voltage usually improves the DRAM reliability and allows it to be clocked higher. You can try to increase this voltage to 1.300V to check if this helps. Don't go beyond 1.325V on Allwinner A10 though.
sunxi wiki describe calibration, when it is possible to run system, but what can be done in situation when U-Boot cannot reach that state ?
The whole idea of this calibration process is to find good ZQ and TPR3 settings. The settings are assumed to be good if the board is able to pass the lima-memtester reliability test at a higher DRAM clock speed without failing. We exploit the fact that lima-memster is by far more sensitive to DRAM misconfiguration than the usual software. So the borderline unreliable DRAM settings may not exhibit any visible bad effects (the system is able to boot, the usual linux software works fine too), but easily detected by lima-memtester. Because the system is sufficiently usable at such borderline unstable DRAM settings, we can run automation scripts to do a bruteforce search, which is probing different TPR3 values without human supervision. That's what is explained at:
https://linux-sunxi.org/A10_DRAM_Controller_CalibrationBut in your case, you can try different ZQ and TPR3 settings manually in the U-Boot defconfig and instead of running lima-memtester (which first needs a bootable userland) just monitor how far the system is able to boot as an alternative selection criteria for picking better configuration values. If you manage to boot the system up to a semi-working userland, then you can switch to using lima-memtester for the final tuning and validation.
I applied above suggestions those were very helpful. In addition, for further reference, to tweak ZQ for A10 configuration requires enabling ODT. So I applied those options:
CONFIG_AXP_DCDC3_VOLT=1300 CONFIG_DRAM_ZQ=123 CONFIG_DRAM_ODT_EN=y
And now I sporadically can start kernel. Unfortunately sunxi-3.4 crash on:
mm/page_alloc.c:2317 __alloc_pages_nodemask+0x374/0x7b8()
and there are also couple BUGs:
BUG: Bad page state in process swapper pfn:4ebb7
I wonder if tweaking ZQ can help with that. I will try and let you know. If you have any suggestions related to above
If anyone more experienced can advise what can be done at this point I would appreciate.
Maybe ask the hardware people if they have done the DDR3 routing right? Also if it is just a single board, then it might be not misconfigured, but defective. Trying another sample could be a good idea.
It would be hard to verify hardware people answer. I think all points to wrong DRAM wiring. Hardware people obviously didn't have previous experience with Allwinner (ie. IDDET for USB OTG was not connected and other minor problems). Are there any guides for DRAM wiring for Allwinner ?
I have 2 boards and tried both although I need more statistics to gain confidence in results.
[1] http://www.micron.com/~/media/Documents/Products/Data%20Sheet/DRAM/DDR3/1Gb_... [2] http://git.denx.de/?p=u-boot.git;a=blob;f=lib/crc32.c;h=97592124867abb815d57... [3] https://gist.github.com/pietrushnic/ea41a4ae38b7af8a6fa456113ce19af8 [4] http://www.denx.de/wiki/view/DULG/UBootCrashAfterRelocation [5] https://gist.github.com/pietrushnic/d6a3d6c0b4a20a3226dfb67c4d129142
-- Best regards, Siarhei Siamashka