[U-Boot] Question re HYP mode and IRQ/FIQ stack setting
(Cc:ing Andre and Christoffer as they have discussed HYP on the ML.)
Hello,
I am working on changing the way IRQ/FIQ stacks are set up, from "on-the-fly in a hurry while in the handler" to "during init, so that when entering the handler, the stack is already correct".
Setting the stack then requires switching from the current mode (in most cases, SVC32, 0x13) to IRQ (0x11) or FIQ (0x12) mode, in order to set the right banked SP, then back into the original mode.
However, in the first lines of reset in arch/arm/cpu/armv7/start.S, the possibility of U-Boot being started in HYP mode (0x1A) is considered and, if in HYP mode, no switch to SVC32 is performed.
I understand that the problem here is, if we drop from HYP to SVC32, then we cannot go back to HYP, and we want to be able to remain in HYP.
Does this also apply to dropping from HYP to IRQ or FIQ mode, i.e., if we do such a drop, are we prevented from rising back from IRQ or FIQ mode to HYP? I seem to remember such an issue, but I am no specialist in HYP, so any help is welcome.
Amicalement,
On 12 November 2013 03:41, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
(Cc:ing Andre and Christoffer as they have discussed HYP on the ML.)
Hello,
I am working on changing the way IRQ/FIQ stacks are set up, from "on-the-fly in a hurry while in the handler" to "during init, so that when entering the handler, the stack is already correct".
Setting the stack then requires switching from the current mode (in most cases, SVC32, 0x13) to IRQ (0x11) or FIQ (0x12) mode, in order to set the right banked SP, then back into the original mode.
However, in the first lines of reset in arch/arm/cpu/armv7/start.S, the possibility of U-Boot being started in HYP mode (0x1A) is considered and, if in HYP mode, no switch to SVC32 is performed.
I understand that the problem here is, if we drop from HYP to SVC32, then we cannot go back to HYP, and we want to be able to remain in HYP.
correct (not without setting up a trap handler in Hyp mode and trapping to Hyp mode)
Does this also apply to dropping from HYP to IRQ or FIQ mode, i.e., if we do such a drop, are we prevented from rising back from IRQ or FIQ mode to HYP? I seem to remember such an issue, but I am no specialist in HYP, so any help is welcome.
Yes, it also applies. Hyp is strictly more privileged (PL2) than all the PL1 modes (SVC, SYS, IRQ, FIQ, ABT, UND) and therefore requires a trap to go from PL1 to PL2 (basically this is how hardware protection works - just like with syscalls from user mode to PL1).
You can use MSR and MRS instructions to access the IRQ and FIQ registers directly from Hyp mode though.
-Christoffer
On 11/12/2013 05:28 PM, Christoffer Dall wrote:
On 12 November 2013 03:41, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
(Cc:ing Andre and Christoffer as they have discussed HYP on the ML.)
Hello,
I am working on changing the way IRQ/FIQ stacks are set up, from "on-the-fly in a hurry while in the handler" to "during init, so that when entering the handler, the stack is already correct".
Setting the stack then requires switching from the current mode (in most cases, SVC32, 0x13) to IRQ (0x11) or FIQ (0x12) mode, in order to set the right banked SP, then back into the original mode.
However, in the first lines of reset in arch/arm/cpu/armv7/start.S, the possibility of U-Boot being started in HYP mode (0x1A) is considered and, if in HYP mode, no switch to SVC32 is performed.
I understand that the problem here is, if we drop from HYP to SVC32, then we cannot go back to HYP, and we want to be able to remain in HYP.
Right, that is to keep the HYP mode in case the firmware already enabled it. This is for instance the case on the new Calxeda Midway. Actually this approach will become more widespread, since it is required to provide proper PSCI support (which needs to run in secure state, so requires an even higher privilege level than HYP: EL3 in the new ARM speak).
correct (not without setting up a trap handler in Hyp mode and trapping to Hyp mode)
Does this also apply to dropping from HYP to IRQ or FIQ mode, i.e., if we do such a drop, are we prevented from rising back from IRQ or FIQ mode to HYP? I seem to remember such an issue, but I am no specialist in HYP, so any help is welcome.
Yes, it also applies. Hyp is strictly more privileged (PL2) than all the PL1 modes (SVC, SYS, IRQ, FIQ, ABT, UND) and therefore requires a trap to go from PL1 to PL2 (basically this is how hardware protection works - just like with syscalls from user mode to PL1).
Thanks Christoffer for clarifying this, I wasn't sure about FIQ, but of course your explanation (EL1 vs. EL2) makes totally sense.
But I wonder what happens when we enter FIQ or IRQ due to an actual interrupt. Will the CPU return into HYP mode when the handler returns? That is subject to some HYP mode register configuration, right?
You can use MSR and MRS instructions to access the IRQ and FIQ registers directly from Hyp mode though.
Albert, so does "msr sp_{fiq,irq}, r<n>" fix your problem? Or do you still need to actually go into one of these modes for further setup?
Regards, Andre.
On 12 November 2013 08:53, Andre Przywara andre.przywara@linaro.org wrote:
On 11/12/2013 05:28 PM, Christoffer Dall wrote:
On 12 November 2013 03:41, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
(Cc:ing Andre and Christoffer as they have discussed HYP on the ML.)
Hello,
I am working on changing the way IRQ/FIQ stacks are set up, from "on-the-fly in a hurry while in the handler" to "during init, so that when entering the handler, the stack is already correct".
Setting the stack then requires switching from the current mode (in most cases, SVC32, 0x13) to IRQ (0x11) or FIQ (0x12) mode, in order to set the right banked SP, then back into the original mode.
However, in the first lines of reset in arch/arm/cpu/armv7/start.S, the possibility of U-Boot being started in HYP mode (0x1A) is considered and, if in HYP mode, no switch to SVC32 is performed.
I understand that the problem here is, if we drop from HYP to SVC32, then we cannot go back to HYP, and we want to be able to remain in HYP.
Right, that is to keep the HYP mode in case the firmware already enabled it. This is for instance the case on the new Calxeda Midway. Actually this approach will become more widespread, since it is required to provide proper PSCI support (which needs to run in secure state, so requires an even higher privilege level than HYP: EL3 in the new ARM speak).
What Andre is referring to here is that the PSCI specs mandates the CPUs that a are turned on using PSCI go through the highest privilege level implemented in the non-secure state. I couldn't actually find the place in the PSCI specs where this is also true for the boot CPU, but granted, anything else will be quite strange.
So to stay in 32-bit ARMv7 terminology here, a likely case for systems with PSCI support in the firmware is that all cores will enter U-Boot in non-secure PL2 (hyp) mode and power management is done by calling SMC calls to the firmware that sits in the secure monitor.
correct (not without setting up a trap handler in Hyp mode and trapping to Hyp mode)
Does this also apply to dropping from HYP to IRQ or FIQ mode, i.e., if we do such a drop, are we prevented from rising back from IRQ or FIQ mode to HYP? I seem to remember such an issue, but I am no specialist in HYP, so any help is welcome.
Yes, it also applies. Hyp is strictly more privileged (PL2) than all the PL1 modes (SVC, SYS, IRQ, FIQ, ABT, UND) and therefore requires a trap to go from PL1 to PL2 (basically this is how hardware protection works - just like with syscalls from user mode to PL1).
Thanks Christoffer for clarifying this, I wasn't sure about FIQ, but of course your explanation (EL1 vs. EL2) makes totally sense.
But I wonder what happens when we enter FIQ or IRQ due to an actual interrupt. Will the CPU return into HYP mode when the handler returns? That is subject to some HYP mode register configuration, right?
Not quite, interrupts in Hyp mode will always be taken to Hyp mode (unless it's secure interrupts, which are taken to monitor mode). In fact, when you have the virtualization extensions, you also have the security extensions, and I think all FIQs go to the secure mode here, so you can't ever see an FIQ in Hyp mode. I may be mistaken here, someone should look it up and verify.
Now, you can configure Hyp mode to trap interrupts raised while in any PL1 or PL0 mode to Hyp mode.
I suspect that if you are in Hyp mode, you should not worry about FIQ/IRQ mode, but just make sure to configure Hyp mode properly to handle interrupts. (it's a separate entry in the exception vector and you probably need to look at the HSR register whn you've taken an interrupt). So, as Andre suggests below, it depends on your use case.
You can use MSR and MRS instructions to access the IRQ and FIQ registers directly from Hyp mode though.
Albert, so does "msr sp_{fiq,irq}, r<n>" fix your problem? Or do you still need to actually go into one of these modes for further setup?
Regards, Andre.
Hi Christoffer,
On Tue, 12 Nov 2013 09:09:23 -0800, Christoffer Dall christoffer.dall@linaro.org wrote:
I suspect that if you are in Hyp mode, you should not worry about FIQ/IRQ mode, but just make sure to configure Hyp mode properly to handle interrupts. (it's a separate entry in the exception vector and you probably need to look at the HSR register whn you've taken an interrupt). So, as Andre suggests below, it depends on your use case.
Ok, so les me try to sum that up from my perspective (IRQ and FIQ stacks in U-Boot):
Some ARM U-Boot targets run in SVC32 mode, and some in HYP mode.
For targets which run in SVC32 mode: aborts execute in abort mode, IRQs in IRQ mode, FIQs in FIQ mode, etc., each mode having its own stack.
For targets which run in HYP mode: aborts, FIQ and IRQs all run in HYP mode, using always the same stack.
In both types of targers: prefetch and data aborts, FIQs and IRQs execute through the usual vectors; in SVC32 mode, because there's no other way; in HYP mode, because exceptions occurring while in HYP mode use the normal vector, not the HYP vector.
Correct?
If so, then U-Boot stack requirements for SVC32 and HYP mode are as follows: in SVC32, we need separate IRQ and FIQ stacks, and the main (SVC32) stack does not have to accommodate for interrupt handler context storage. In HYP mode, the only stack is the HYP one and exception handlers will use it too, so it has to accommodate for their context storage.
Still correct?
Amicalement,
On 12 November 2013 13:29, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
Hi Christoffer,
On Tue, 12 Nov 2013 09:09:23 -0800, Christoffer Dall christoffer.dall@linaro.org wrote:
I suspect that if you are in Hyp mode, you should not worry about FIQ/IRQ mode, but just make sure to configure Hyp mode properly to handle interrupts. (it's a separate entry in the exception vector and you probably need to look at the HSR register whn you've taken an interrupt). So, as Andre suggests below, it depends on your use case.
Ok, so les me try to sum that up from my perspective (IRQ and FIQ stacks in U-Boot):
Some ARM U-Boot targets run in SVC32 mode, and some in HYP mode.
For targets which run in SVC32 mode: aborts execute in abort mode, IRQs in IRQ mode, FIQs in FIQ mode, etc., each mode having its own stack.
correct
For targets which run in HYP mode: aborts, FIQ and IRQs all run in HYP mode, using always the same stack.
correct.
In both types of targers: prefetch and data aborts, FIQs and IRQs execute through the usual vectors; in SVC32 mode, because there's no other way; in HYP mode, because exceptions occurring while in HYP mode use the normal vector, not the HYP vector.
Correct?
no, this sounds fishy. Look at the ARM ARMv7 (DDI 0406C.b) page B1-1166. Table B1-3 will tell you what you need to know.
Hyp mode has its own vectors, pointed to by the HVBAR control register.
I strongly suggest you familiarize yourself with these parts of the ARM ARM before writing code to that effect.
If so, then U-Boot stack requirements for SVC32 and HYP mode are as follows: in SVC32, we need separate IRQ and FIQ stacks, and the main (SVC32) stack does not have to accommodate for interrupt handler context storage. In HYP mode, the only stack is the HYP one and exception handlers will use it too, so it has to accommodate for their context storage.
That is correct, your Hyp mode stack should always be valid (have a page or so, like the kernel) and then you can always push things on there, even when taking an exception.
-Christoffer
Hi Christoffer,
On Tue, 12 Nov 2013 14:34:00 -0800, Christoffer Dall christoffer.dall@linaro.org wrote:
On 12 November 2013 13:29, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
Hi Christoffer,
On Tue, 12 Nov 2013 09:09:23 -0800, Christoffer Dall christoffer.dall@linaro.org wrote:
I suspect that if you are in Hyp mode, you should not worry about FIQ/IRQ mode, but just make sure to configure Hyp mode properly to handle interrupts. (it's a separate entry in the exception vector and you probably need to look at the HSR register whn you've taken an interrupt). So, as Andre suggests below, it depends on your use case.
Ok, so les me try to sum that up from my perspective (IRQ and FIQ stacks in U-Boot):
Some ARM U-Boot targets run in SVC32 mode, and some in HYP mode.
For targets which run in SVC32 mode: aborts execute in abort mode, IRQs in IRQ mode, FIQs in FIQ mode, etc., each mode having its own stack.
correct
For targets which run in HYP mode: aborts, FIQ and IRQs all run in HYP mode, using always the same stack.
correct.
In both types of targers: prefetch and data aborts, FIQs and IRQs execute through the usual vectors; in SVC32 mode, because there's no other way; in HYP mode, because exceptions occurring while in HYP mode use the normal vector, not the HYP vector.
Correct?
no, this sounds fishy. Look at the ARM ARMv7 (DDI 0406C.b) page B1-1166. Table B1-3 will tell you what you need to know.
Hyp mode has its own vectors, pointed to by the HVBAR control register.
I strongly suggest you familiarize yourself with these parts of the ARM ARM before writing code to that effect.
To clarify my meaning: I wasn't speaking about the exceptions base address, but about the exception vector offset. IOW, I did not mean to say that in HYP mode, aborts, FIQs, IRQs etc use the SVC32 table of vectors rather than the HYP table. I meant to say that e.g. an IRQ will fire at offset 0x18, whether it fires while from SVC32 into IRQ mode, or from HYP mode into HYP mode. This seemed consistent with figure B1-8 on page B1-1179 of DDI 0406C.b, respectively first and sixth "exit box" on the right.
Is this clearer and more correct?
If so, then U-Boot stack requirements for SVC32 and HYP mode are as follows: in SVC32, we need separate IRQ and FIQ stacks, and the main (SVC32) stack does not have to accommodate for interrupt handler context storage. In HYP mode, the only stack is the HYP one and exception handlers will use it too, so it has to accommodate for their context storage.
That is correct, your Hyp mode stack should always be valid (have a page or so, like the kernel) and then you can always push things on there, even when taking an exception.
Thanks Christoffer! This should allow me to refactor interrupt stack settings as I intended.
-Christoffer
Amicalement,
On 12 November 2013 22:22, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
Hi Christoffer,
On Tue, 12 Nov 2013 14:34:00 -0800, Christoffer Dall christoffer.dall@linaro.org wrote:
On 12 November 2013 13:29, Albert ARIBAUD albert.u.boot@aribaud.net wrote:
Hi Christoffer,
On Tue, 12 Nov 2013 09:09:23 -0800, Christoffer Dall christoffer.dall@linaro.org wrote:
I suspect that if you are in Hyp mode, you should not worry about FIQ/IRQ mode, but just make sure to configure Hyp mode properly to handle interrupts. (it's a separate entry in the exception vector and you probably need to look at the HSR register whn you've taken an interrupt). So, as Andre suggests below, it depends on your use case.
Ok, so les me try to sum that up from my perspective (IRQ and FIQ stacks in U-Boot):
Some ARM U-Boot targets run in SVC32 mode, and some in HYP mode.
For targets which run in SVC32 mode: aborts execute in abort mode, IRQs in IRQ mode, FIQs in FIQ mode, etc., each mode having its own stack.
correct
For targets which run in HYP mode: aborts, FIQ and IRQs all run in HYP mode, using always the same stack.
correct.
In both types of targers: prefetch and data aborts, FIQs and IRQs execute through the usual vectors; in SVC32 mode, because there's no other way; in HYP mode, because exceptions occurring while in HYP mode use the normal vector, not the HYP vector.
Correct?
no, this sounds fishy. Look at the ARM ARMv7 (DDI 0406C.b) page B1-1166. Table B1-3 will tell you what you need to know.
Hyp mode has its own vectors, pointed to by the HVBAR control register.
I strongly suggest you familiarize yourself with these parts of the ARM ARM before writing code to that effect.
To clarify my meaning: I wasn't speaking about the exceptions base address, but about the exception vector offset. IOW, I did not mean to say that in HYP mode, aborts, FIQs, IRQs etc use the SVC32 table of vectors rather than the HYP table. I meant to say that e.g. an IRQ will fire at offset 0x18, whether it fires while from SVC32 into IRQ mode, or from HYP mode into HYP mode. This seemed consistent with figure B1-8 on page B1-1179 of DDI 0406C.b, respectively first and sixth "exit box" on the right.
Is this clearer and more correct?
yes, now I understand. It's correct.
If so, then U-Boot stack requirements for SVC32 and HYP mode are as follows: in SVC32, we need separate IRQ and FIQ stacks, and the main (SVC32) stack does not have to accommodate for interrupt handler context storage. In HYP mode, the only stack is the HYP one and exception handlers will use it too, so it has to accommodate for their context storage.
That is correct, your Hyp mode stack should always be valid (have a page or so, like the kernel) and then you can always push things on there, even when taking an exception.
Thanks Christoffer! This should allow me to refactor interrupt stack settings as I intended.
-Christoffer
Amicalement,
Albert.
participants (3)
-
Albert ARIBAUD -
Andre Przywara -
Christoffer Dall