Hi,
I have a custom board using the LS2088A device. We are seeing problems regarding the DPAA2 network since the beginning (2016). The problem occurs usually after the first link is established after booting Linux.
Example:
[ 7.755443] fsl_dpaa2_eth dpni.0 ni0: Link Event: state up [ 7.791804] BUG: failure at /home/ralf/LS2088A-SDK-20160719-yocto/build_ls2088a-lemans/tmp/work-shared/l s2088a-lemans/kernel-source/drivers/staging/fsl-mc/bus/dpio/dpio_service.c:7 21/dpaa2_io_store_next()! [ 7.810074] Kernel panic - not syncing: BUG! [ 7.814341] CPU: 5 PID: 2701 Comm: run-parts Not tainted 4.1.8+g0a32d65 #15 [ 7.821301] Hardware name: IMAGO VisionBox LeMans (DT) [ 7.826435] Call trace: [ 7.828883] [<ffff800000089958>] dump_backtrace+0x0/0x11c [ 7.834279] [<ffff800000089a84>] show_stack+0x10/0x1c [ 7.839328] [<ffff800000938214>] dump_stack+0x88/0xc8 [ 7.844376] [<ffff800000936f30>] panic+0xe0/0x220 [ 7.849077] [<ffff8000007e48fc>] dpaa2_io_store_next+0xe4/0xe8 [ 7.854908] [<ffff800000801d88>] consume_frames+0x2c/0x94 [ 7.860304] [<ffff8000008042c8>] dpaa2_eth_poll+0x50/0x108 [ 7.865786] [<ffff80000083acf8>] net_rx_action+0x1d0/0x2b0 [ 7.871269] [<ffff8000000b6f88>] __do_softirq+0xfc/0x238 [ 7.876577] [<ffff8000000b7338>] irq_exit+0x90/0xd0 [ 7.881452] [<ffff8000000ee170>] __handle_domain_irq+0x5c/0xb0 [ 7.887281] [<ffff8000000824f4>] gic_handle_irq+0x80/0xe4 [ 7.892676] Exception stack(0xffff808126943ad0 to 0xffff808126943bf0) [ 7.899114] 3ac0: a5d52003 00000081 25eb58a4 ffff8081 [ 7.907291] 3ae0: 26943c20 ffff8081 00944af0 ffff8000 60000145 00000000 0014065c ffff8000 [ 7.915469] 3b00: 25eb58a4 ffff8081 80000000 00000000 00000000 00000000 00000000 00000000 [ 7.923646] 3b20: ffffffc8 ffffffff 00000000 00000000 26943a50 ffff8081 00be4a60 ffff8000 [ 7.931823] 3b40: 00be4a48 ffff8000 00be4a30 ffff8000 c69754c0 ffff7c01 6ab1b6a0 ffff8000 [ 7.940000] 3b60: 00000004 00000000 374da550 ffff8081 374da550 ffff8081 a80b9588 0000ffff [ 7.948177] 3b80: 000b3134 ffff8000 a8011140 0000ffff f7b64440 0000ffff a5d52003 00000081 [ 7.956354] 3ba0: 25eb58a4 ffff8081 25e4c000 ffff8081 00000000 00000000 00000000 00000000 [ 7.964531] 3bc0: 6a93f178 ffff8000 00e15000 ffff8000 26940000 ffff8081 6a93f178 ffff8000 [ 7.972708] 3be0: 00413000 00000000 26943c20 ffff8081 [ 7.977755] [<ffff8000000855a4>] el1_irq+0x64/0xd8 [ 7.982543] [<ffff80000015fbfc>] copy_page_range+0x8ac/0x91c [ 7.988199] [<ffff8000000b24c8>] copy_process.part.42+0xba8/0x13f0 [ 7.994376] [<ffff8000000b2e7c>] do_fork+0xb4/0x2e8 [ 7.999249] [<ffff8000000b3144>] SyS_clone+0x10/0x1c [ 8.004212] CPU0: stopping [ 8.006917] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.1.8+g0a32d65 #15 [ 8.013616] Hardware name: IMAGO VisionBox LeMans (DT) [ 8.018750] Call trace: [ 8.021195] [<ffff800000089958>] dump_backtrace+0x0/0x11c [ 8.026592] [<ffff800000089a84>] show_stack+0x10/0x1c [ 8.031641] [<ffff800000938214>] dump_stack+0x88/0xc8 [ 8.036689] [<ffff8000000906f8>] handle_IPI+0x18c/0x19c
The problem doesn't always occur and also not with every board we have. I have tried different versions of the Management Complex Firmware, U-Boot and the Linux kernel since the beginning. All show the same behavior.
Now, I figured out that initialization of MC memory before starting the MC within U-Boot makes a difference. Everything works fine if I initialize the memory with zero first:
int mc_init(u64 mc_fw_addr, u64 mc_dpc_addr) { ... /* Clear MC RAM first: */ memset((void *)mc_ram_addr, 0, mc_ram_size);
/* * Load the MC FW at the beginning of the MC private DRAM block: */ mc_copy_image("MC Firmware", (u64)raw_image_addr, raw_image_size, mc_ram_addr); ...
If I initialize the memory with 0xff instead, the problem always occurs.
Is there any known problem for using uninitialized memory for the MC? I couldn't find anything in the NXP documents I have.
Please note that I'm not using DDR4 ECC memory. ECC memory is normally initialized with the value of CONFIG_MEM_INIT_VALUE (0xdeadbeef) by the memory controller. Initializing MC memory with the value 0xdeadbeef manually also seems to fix the problem for me.
Thanks, Ralf