Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c index 6eac5ef..5cc09ba 100644 --- a/arch/arm/cpu/armv7/cpu.c +++ b/arch/arm/cpu/armv7/cpu.c @@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags) */ dcache_disable(); v7_outer_cache_disable(); + ISB;

/* * After D-cache is flushed and before it is disabled there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable ==>flush_dcache_all() ==>set_cr() disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val) { if (is_hyp()) asm volatile("mcr p15, 4, %0, c1, c0, 0 @ set CR" : : "r" (val) : "cc"); else asm volatile("mcr p15, 0, %0, c1, c0, 0 @ set CR" : : "r" (val) : "cc"); isb(); }

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it. @Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

On 2016年07月25日 23:00, Sandy Patterson wrote:

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <xzy.xu@rock-chips.com mailto:xzy.xu@rock-chips.com> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find
out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
    printf("Boot reached stage %d\n", progress);

}


On 2016年07月25日 22:12, Ziyuan Xu wrote:

    Hi All,

    I'm sorry to tell you that I failed to boot linux kernel with
    the mainline u-boot on rk3288 board(both evb-rk3288 &
    fennec-rk3288). It was stuck in cleanup_before_linux() before
    jumping to linux, and the boot_stage_flag is
    BOOTSTAGE_ID_BOOTM_HANDOFF.

    ## Current stack ends at 0x7df638b0 *  kernel: cmdline image
    address = 0x02000000
    ## No init Ramdisk
       ramdisk start = 0x00000000, ramdisk end = 0x00000000
    ## No Flattened Device Tree
    Continuing to boot without FDT
    Initial value for argc=3
    Final value for argc=3
    using: ATAGS
    ## Transferring control to Linux (at address 02000000)...

    Starting kernel ...

    With the further investigation, it never returnned back from
    invalidate_dcache_all(). I mean that I can't reach stage 4 as
    below.

    cleanup_before_linux
        ==>cleanup_before_linux_select
            ==>1.disable_interrupts
            ==>2.dcache_disable
            ==>3.invalidate_dcache_all
            ==>4.icache_disable

    Debug further, invalidate_dcache_all  invalidate all cache
    one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or
    UNIFIED. And invalidate way from one set to another set in
    order. The problem is that the PC ran away in invalidate way
    loop  [cache level L1!!!].

    I add some serial output code in __v7_flush_dcache_all to
    figure out the bog.
    I expect:
    Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d.....
    0x01, 0x00 (hex)
    In fact, print the value of r9 in sequence at first, but print
    unexpected value afterwards.   e.g 0x7f, 0x7e, 0x7d, ..,0x73,
    0x40, 0x85,....

    ENTRY(__v7_flush_dcache_all)
    [snip]
    loop1:
        mov    r9, r7                @ create working copy of max
    index
    loop2:
    +      stmfd    sp!, {r0}
    +      ldr     r0, =0xff690000
    +      str     r9, [r0]
    +      ldmfd    sp!, {r0}
     ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and
    cache number into r11
     THUMB(    lsl    r6, r4, r5        )
     THUMB(    orr    r11, r10, r6        )    @ factor way and
    cache number into r11
     ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index
    number into r11
     THUMB(    lsl    r6, r9, r2        )
     THUMB(    orr    r11, r11, r6        )    @ factor index
    number into r11
        mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate
    by set/way
        subs    r9, r9, #1            @ decrement the index
        bge    loop2
        subs    r4, r4, #1            @ decrement the way
        bge    loop1
    skip:
    ENDPROC(__v7_flush_dcache_all)

    I don't have the jtag, hence I can't address the current pc. I
    have no doubt that if any glitches in __v7_flush_dcache_all, I
    reviewed several times, also copy kernel's implement to here.
    :-( No effect.
    A more interesting thing is that Sandy had report it. He and I
    have similar problem. Everything work fine after I applied his
    patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
    @Stany, I'm sorry that I disable dcache during hack.. That was
    a mistake:-(

    @Simon & hieko,
    Can you boot linux with the mainline u-boot? have a try?


    _______________________________________________
    U-Boot mailing list
    U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
    http://lists.denx.de/mailman/listinfo/u-boot

+ Simon and heiko

On 2016年07月26日 14:30, Ziyuan Xu wrote:

Dear All,

I add the ISB operation after dcache_disable(), and I can jump to linux kernel entry.:-)

diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c index 6eac5ef..5cc09ba 100644 --- a/arch/arm/cpu/armv7/cpu.c +++ b/arch/arm/cpu/armv7/cpu.c @@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags) */ dcache_disable(); v7_outer_cache_disable();

•           ISB;
  
            /*
            * After D-cache is flushed and before it is disabled 
  

there may

Sounds crazy. In fact, there is already an 'ISB' operation in set_cr() which to disable dcache and MMU. But it just tell processor that memory had change, not real ISB for armv7.

dcache_disable ==>flush_dcache_all() ==>set_cr() disable dcache and MMU.

#define isb() __asm__ __volatile__ ("" : : : "memory") static inline void set_cr(unsigned int val) { if (is_hyp()) asm volatile("mcr p15, 4, %0, c1, c0, 0 @ set CR" : : "r" (val) : "cc"); else asm volatile("mcr p15, 0, %0, c1, c0, 0 @ set CR" : : "r" (val) : "cc"); isb(); }

I also find something in ARM cortex-A17 TRM. ==>"Disable the MMU from the each processor followed by an ISB to ensure the MMU disable operation is complete, then followed by a DSB to drain previous memory transactions."

In my humble opinion, maybe instructions tlb had alter, and cause running away??? I'm not sure about it. @Sandy, could you have a try with my update?

@Simon, did you hit this glitch? I hope you can help ...:-)

On 2016年07月25日 23:00, Sandy Patterson wrote:

...

Ah, thanks. Your debugging looks the same as what I've seen from printf debugging. I'll try to verify though.

On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu <xzy.xu@rock-chips.com mailto:xzy.xu@rock-chips.com> wrote:

hi Stany,

The difference is that you print out assertion log.

Reset not supported on this platform
### ERROR ### Please RESET the board ###

You can add show_boot_progress() function in your bsp file to find
out something.
#define CONFIG_SHOW_BOOT_PROGRESS
void show_boot_progress(int progress)
{
    printf("Boot reached stage %d\n", progress);

}


On 2016年07月25日 22:12, Ziyuan Xu wrote:

    Hi All,

    I'm sorry to tell you that I failed to boot linux kernel with
    the mainline u-boot on rk3288 board(both evb-rk3288 &
    fennec-rk3288). It was stuck in cleanup_before_linux() before
    jumping to linux, and the boot_stage_flag is
    BOOTSTAGE_ID_BOOTM_HANDOFF.

    ## Current stack ends at 0x7df638b0 *  kernel: cmdline image
    address = 0x02000000
    ## No init Ramdisk
       ramdisk start = 0x00000000, ramdisk end = 0x00000000
    ## No Flattened Device Tree
    Continuing to boot without FDT
    Initial value for argc=3
    Final value for argc=3
    using: ATAGS
    ## Transferring control to Linux (at address 02000000)...

    Starting kernel ...

    With the further investigation, it never returnned back from
    invalidate_dcache_all(). I mean that I can't reach stage 4 as
    below.

    cleanup_before_linux
        ==>cleanup_before_linux_select
            ==>1.disable_interrupts
            ==>2.dcache_disable
            ==>3.invalidate_dcache_all
            ==>4.icache_disable

    Debug further, invalidate_dcache_all  invalidate all cache
    one-by-one which cache type is DATA_ONLY, INSTRUCTION_DATA or
    UNIFIED. And invalidate way from one set to another set in
    order. The problem is that the PC ran away in invalidate way
    loop  [cache level L1!!!].

    I add some serial output code in __v7_flush_dcache_all to
    figure out the bog.
    I expect:
    Print the value of r9 in sequence, e.g 0x7f, 0x7e, 0x7d.....
    0x01, 0x00 (hex)
    In fact, print the value of r9 in sequence at first, but print
    unexpected value afterwards.   e.g 0x7f, 0x7e, 0x7d, ..,0x73,
    0x40, 0x85,....

    ENTRY(__v7_flush_dcache_all)
    [snip]
    loop1:
        mov    r9, r7                @ create working copy of max
    index
    loop2:
    +      stmfd    sp!, {r0}
    +      ldr     r0, =0xff690000
    +      str     r9, [r0]
    +      ldmfd    sp!, {r0}
     ARM(    orr    r11, r10, r4, lsl r5    )    @ factor way and
    cache number into r11
     THUMB(    lsl    r6, r4, r5        )
     THUMB(    orr    r11, r10, r6        )    @ factor way and
    cache number into r11
     ARM(    orr    r11, r11, r9, lsl r2    )    @ factor index
    number into r11
     THUMB(    lsl    r6, r9, r2        )
     THUMB(    orr    r11, r11, r6        )    @ factor index
    number into r11
        mcr    p15, 0, r11, c7, c14, 2        @ clean & invalidate
    by set/way
        subs    r9, r9, #1            @ decrement the index
        bge    loop2
        subs    r4, r4, #1            @ decrement the way
        bge    loop1
    skip:
    ENDPROC(__v7_flush_dcache_all)

    I don't have the jtag, hence I can't address the current pc. I
    have no doubt that if any glitches in __v7_flush_dcache_all, I
    reviewed several times, also copy kernel's implement to here.
    :-( No effect.
    A more interesting thing is that Sandy had report it. He and I
    have similar problem. Everything work fine after I applied his
    patches, or disable dcache(active CONFIG_SYS_DCACHE_OFF).
    @Stany, I'm sorry that I disable dcache during hack.. That was
    a mistake:-(

    @Simon & hieko,
    Can you boot linux with the mainline u-boot? have a try?


    _______________________________________________
    U-Boot mailing list
    U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
    http://lists.denx.de/mailman/listinfo/u-boot

---

U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot

Hi Sandy,

I'm pleased to learn that it does fix for you.

On 2016年07月26日 21:16, Sandy Patterson wrote:

Hi Ziyuan, good work!

This does in fact fix it for me.

I followed through your notes. What is isr() supposed to do? it looks like it just tells the compiler that the memory cache has been invalidated. But in the

isb()? I'm not very enrich knowledgeable in cache, this issue I'm doing also is a big learning experience for me. I catch something from TRM, and review u-boot again and again in order to figure out the root cause. I suspect the former disable_dcache/mmu operation didn't complete, or branch prediction was abnormal? I'm not really sure.

TRM it says we need to actually do an ISB.

Would fixing the dcache flush so it does an ISB when it disables the MMU maybe be an alternative solution? (I couldn't figure out which macro I should be using. isb() is defined for arm64 to do the actual op.

Anyway, the following also fixes for me.

diff --git a/arch/arm/lib/cache-cp15.c b/arch/arm/lib/cache-cp15.c index 1121dc3..3494a5c 100644 --- a/arch/arm/lib/cache-cp15.c +++ b/arch/arm/lib/cache-cp15.c @@ -217,6 +217,8 @@ static void cache_disable(uint32_t cache_bit) if (cache_bit == (CR_C | CR_M)) flush_dcache_all(); set_cr(reg & ~cache_bit);

•   if(cache_bit & CR_M)
  

•           asm volatile ("isb" : : : "memory");
  

} #endif

I'm not sure what kinds of side effects for other boards these changes would have.

Let's listen to the opinions of others.

On Tue, Jul 26, 2016 at 8:15 AM, Ziyuan Xu <xzy.xu@rock-chips.com mailto:xzy.xu@rock-chips.com> wrote:

+ Simon and heiko

On 2016年07月26日 14:30, Ziyuan Xu wrote:

    Dear All,

    I add the ISB operation after dcache_disable(), and I can jump
    to linux kernel entry.:-)

    diff --git a/arch/arm/cpu/armv7/cpu.c b/arch/arm/cpu/armv7/cpu.c
    index 6eac5ef..5cc09ba 100644
    --- a/arch/arm/cpu/armv7/cpu.c
    +++ b/arch/arm/cpu/armv7/cpu.c
    @@ -43,6 +43,7 @@ int cleanup_before_linux_select(int flags)
                    */
                    dcache_disable();
                    v7_outer_cache_disable();
    +               ISB;

                    /*
                    * After D-cache is flushed and before it is
    disabled there may

    Sounds crazy. In fact, there is already an 'ISB' operation in
    set_cr() which to disable dcache and MMU. But it just tell
    processor that  memory had change, not real ISB for armv7.

    dcache_disable
            ==>flush_dcache_all()
            ==>set_cr()   disable dcache and MMU.

    #define isb() __asm__ __volatile__ ("" : : : "memory")
    static inline void set_cr(unsigned int val)
    {
        if (is_hyp())
            asm volatile("mcr p15, 4, %0, c1, c0, 0    @ set CR" :
                                      : "r" (val)
                                      : "cc");
        else
            asm volatile("mcr p15, 0, %0, c1, c0, 0    @ set CR" :
                                      : "r" (val)
                                      : "cc");
        isb();
    }

    I also find something in ARM cortex-A17 TRM. ==>"Disable the
    MMU from the each processor followed by an ISB to ensure the
    MMU disable operation is complete, then followed by a DSB to
    drain previous memory transactions."

    In my humble opinion, maybe instructions tlb had alter, and
    cause running away??? I'm not sure about it.
    @Sandy, could you have a try with my update?

    @Simon,  did you hit this glitch? I hope you can help ...:-)

    On 2016年07月25日 23:00, Sandy Patterson wrote:

        Ah, thanks. Your debugging looks the same as what I've
        seen from printf debugging. I'll try to verify though.

        On Mon, Jul 25, 2016 at 10:58 AM, Ziyuan Xu
        <xzy.xu@rock-chips.com <mailto:xzy.xu@rock-chips.com>
        <mailto:xzy.xu@rock-chips.com
        <mailto:xzy.xu@rock-chips.com>>> wrote:

            hi Stany,

            The difference is that you print out assertion log.

            Reset not supported on this platform
            ### ERROR ### Please RESET the board ###

            You can add show_boot_progress() function in your bsp
        file to find
            out something.
            #define CONFIG_SHOW_BOOT_PROGRESS
            void show_boot_progress(int progress)
            {
                printf("Boot reached stage %d\n", progress);

            }


            On 2016年07月25日 22:12, Ziyuan Xu wrote:

                Hi All,

                I'm sorry to tell you that I failed to boot linux
        kernel with
                the mainline u-boot on rk3288 board(both evb-rk3288 &
                fennec-rk3288). It was stuck in
        cleanup_before_linux() before
                jumping to linux, and the boot_stage_flag is
                BOOTSTAGE_ID_BOOTM_HANDOFF.

                ## Current stack ends at 0x7df638b0 * kernel:
        cmdline image
                address = 0x02000000
                ## No init Ramdisk
                   ramdisk start = 0x00000000, ramdisk end =
        0x00000000
                ## No Flattened Device Tree
                Continuing to boot without FDT
                Initial value for argc=3
                Final value for argc=3
                using: ATAGS
                ## Transferring control to Linux (at address
        02000000)...

                Starting kernel ...

                With the further investigation, it never returnned
        back from
                invalidate_dcache_all(). I mean that I can't reach
        stage 4 as
                below.

                cleanup_before_linux
                    ==>cleanup_before_linux_select
                        ==>1.disable_interrupts
                        ==>2.dcache_disable
                        ==>3.invalidate_dcache_all
                        ==>4.icache_disable

                Debug further, invalidate_dcache_all invalidate
        all cache
                one-by-one which cache type is DATA_ONLY,
        INSTRUCTION_DATA or
                UNIFIED. And invalidate way from one set to
        another set in
                order. The problem is that the PC ran away in
        invalidate way
                loop  [cache level L1!!!].

                I add some serial output code in
        __v7_flush_dcache_all to
                figure out the bog.
                I expect:
                Print the value of r9 in sequence, e.g 0x7f, 0x7e,
        0x7d.....
                0x01, 0x00 (hex)
                In fact, print the value of r9 in sequence at
        first, but print
                unexpected value afterwards.   e.g 0x7f, 0x7e,
        0x7d, ..,0x73,
                0x40, 0x85,....

                ENTRY(__v7_flush_dcache_all)
                [snip]
                loop1:
                    mov    r9, r7                @ create working
        copy of max
                index
                loop2:
                +      stmfd    sp!, {r0}
                +      ldr     r0, =0xff690000
                +      str     r9, [r0]
                +      ldmfd    sp!, {r0}
                 ARM(    orr    r11, r10, r4, lsl r5    ) @ factor
        way and
                cache number into r11
                 THUMB(    lsl    r6, r4, r5        )
                 THUMB(    orr    r11, r10, r6        )    @
        factor way and
                cache number into r11
                 ARM(    orr    r11, r11, r9, lsl r2    ) @ factor
        index
                number into r11
                 THUMB(    lsl    r6, r9, r2        )
                 THUMB(    orr    r11, r11, r6        )    @
        factor index
                number into r11
                    mcr    p15, 0, r11, c7, c14, 2        @ clean
        & invalidate
                by set/way
                    subs    r9, r9, #1            @ decrement the
        index
                    bge    loop2
                    subs    r4, r4, #1            @ decrement the way
                    bge    loop1
                skip:
                ENDPROC(__v7_flush_dcache_all)

                I don't have the jtag, hence I can't address the
        current pc. I
                have no doubt that if any glitches in
        __v7_flush_dcache_all, I
                reviewed several times, also copy kernel's
        implement to here.
                :-( No effect.
                A more interesting thing is that Sandy had report
        it. He and I
                have similar problem. Everything work fine after I
        applied his
                patches, or disable dcache(active
        CONFIG_SYS_DCACHE_OFF).
                @Stany, I'm sorry that I disable dcache during
        hack.. That was
                a mistake:-(

                @Simon & hieko,
                Can you boot linux with the mainline u-boot? have
        a try?


        _______________________________________________
                U-Boot mailing list
        U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
        <mailto:U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>>
        http://lists.denx.de/mailman/listinfo/u-boot









    _______________________________________________
    U-Boot mailing list
    U-Boot@lists.denx.de <mailto:U-Boot@lists.denx.de>
    http://lists.denx.de/mailman/listinfo/u-boot