On 9/10/20 2:39 AM, Rick Chen wrote:
Hi Sean
On 9/9/20 3:50 AM, Rick Chen wrote:
Hi Sean
Clearing MIP doesn't do anything. Whoops. The following commits should tackle this problem in a more robust manner.
I still not catch your points about that this commit 947263033 really help to fix pending IPIs not clean problem on k210 platform at that time, but you just said it do nothing and remove it here currently.
After refactoring the original smp patch to remove the null check, I neglected to re-test with a debug uart enabled. Without doing that test, it is impossible to catch the pending IPI bug. The secondary hart will crash before the serial driver is initialized. I didn't do that test at the time, because I was mostly worried about the secondary hart corrupting the device tree and causing the boot to fail. That failure mode was fixed by 40686c394. So I saw that and thought that the pending IPI issue was solved as well.
Can you describe more clearly why with a debug uart enabled will trigger the pending IPI bug ?
It doesn't trigger the bug, it always happens. However, if there is no debug uart nothing gets printed, because it occurs before the serial driver is initialized.
And why the pending IPI be raised and not clear before jump to U-Boot ?
I don't know. Presumably the boot rom raises it to signal to jump to U-Boot and never clears it.
Why the csrc MODE_PREFIX(ip), t0 don't help to fix this bug ?
The problem is that MSIP in MIP is not necessarily writable.
Each lower privilege level has a separate software interrupt-pending bit (HSIP, SSIP, USIP), which can be both read and written by CSR accesses from code running on the local hart at the associated or any higher privilege level. The machine-level MSIP bits are written by accesses to memory- mapped control registers, which are used by remote harts to provide machine-mode interprocessor interrupts. Interprocessor interrupts for lower privilege levels are implemented through ABI, SBI or HBI calls to the AEE, SEE or HEE respectively, which might ultimately result in a machine- mode write to the receiving hart’s MSIP bit. A hart can write its own MSIP bit using the same memory-mapped control register.
Contrast for example with SSIP in SIP
Three types of interrupts are defined: software interrupts, timer interrupts, and external interrupts. A supervisor-level software interrupt is triggered on the current hart by writing 1 to its supervisor software interrupt-pending (SSIP) bit in the sip register. A pending supervisor-level software interrupt can be cleared by writing 0 to the SSIP bit in sip. Supervisor-level software interrupts are disabled when the SSIE bit in the sie register is clear.
I originally added this at your suggestion, but I never ended up testing it separately from the IPI cleanup patch.
--Sean