[U-Boot] [RFC PATCH] arm: arm926ejs: flush cache before disable it
flush cache before disable it
Signed-off-by: Bo Shen voice.shen@gmail.com --- arch/arm/cpu/arm926ejs/cpu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm/cpu/arm926ejs/cpu.c b/arch/arm/cpu/arm926ejs/cpu.c index 626384c..10aa165 100644 --- a/arch/arm/cpu/arm926ejs/cpu.c +++ b/arch/arm/cpu/arm926ejs/cpu.c @@ -46,15 +46,14 @@ int cleanup_before_linux (void)
disable_interrupts ();
+ /* flush I/D-cache */ + cache_flush();
/* turn off I/D-cache */ icache_disable(); dcache_disable(); l2_cache_disable();
- /* flush I/D-cache */ - cache_flush(); - return 0; }
Hi Bo,
On Tue, 2 Jul 2013 12:35:54 +0000, Bo Shen voice.shen@gmail.com wrote:
flush cache before disable it
Signed-off-by: Bo Shen voice.shen@gmail.com
arch/arm/cpu/arm926ejs/cpu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm/cpu/arm926ejs/cpu.c b/arch/arm/cpu/arm926ejs/cpu.c index 626384c..10aa165 100644 --- a/arch/arm/cpu/arm926ejs/cpu.c +++ b/arch/arm/cpu/arm926ejs/cpu.c @@ -46,15 +46,14 @@ int cleanup_before_linux (void)
disable_interrupts ();
/* flush I/D-cache */
cache_flush();
/* turn off I/D-cache */ icache_disable(); dcache_disable(); l2_cache_disable();
- /* flush I/D-cache */
- cache_flush();
- return 0;
}
What is this change supposed to fix? There is no need to flush before disabling, and actually, flushing before disabling runs the risk that between the two, some cache lines be dirtied again so that cache and memory won't be coherent any more.
Amicalement,
Hi Albert,
于 7/6/2013 5:02 AM, Albert ARIBAUD 写道:
Hi Bo,
On Tue, 2 Jul 2013 12:35:54 +0000, Bo Shen voice.shen@gmail.com wrote:
flush cache before disable it
Signed-off-by: Bo Shen voice.shen@gmail.com
arch/arm/cpu/arm926ejs/cpu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm/cpu/arm926ejs/cpu.c b/arch/arm/cpu/arm926ejs/cpu.c index 626384c..10aa165 100644 --- a/arch/arm/cpu/arm926ejs/cpu.c +++ b/arch/arm/cpu/arm926ejs/cpu.c @@ -46,15 +46,14 @@ int cleanup_before_linux (void)
disable_interrupts ();
/* flush I/D-cache */
cache_flush();
/* turn off I/D-cache */ icache_disable(); dcache_disable(); l2_cache_disable();
- /* flush I/D-cache */
- cache_flush();
- return 0; }
What is this change supposed to fix?
Actually, this is not a issue fix. Maybe my understanding wrong. I think this is just a logic issue. If the cache is disable, then we flush it, will the contents in the cache corrupted?
There is no need to flush before disabling, and actually, flushing before disabling runs the risk that between the two, some cache lines be dirtied again so that cache and memory won't be coherent any more.
I am not fully understand this. In my mind, I think flush cache (writing the dirty data back to memory) is used to keep the coherence. If my understanding is not correct, please help give more explaination or some reference document to me for understanding.
Thanks.
Amicalement,
Best Regards, Bo Shen
Hi Bo,
On Mon, 08 Jul 2013 07:33:18 +0800, Bo Shen voice.shen@gmail.com wrote:
Hi Albert,
于 7/6/2013 5:02 AM, Albert ARIBAUD 写道:
Hi Bo,
On Tue, 2 Jul 2013 12:35:54 +0000, Bo Shen voice.shen@gmail.com wrote:
flush cache before disable it
Signed-off-by: Bo Shen voice.shen@gmail.com
arch/arm/cpu/arm926ejs/cpu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/arch/arm/cpu/arm926ejs/cpu.c b/arch/arm/cpu/arm926ejs/cpu.c index 626384c..10aa165 100644 --- a/arch/arm/cpu/arm926ejs/cpu.c +++ b/arch/arm/cpu/arm926ejs/cpu.c @@ -46,15 +46,14 @@ int cleanup_before_linux (void)
disable_interrupts ();
/* flush I/D-cache */
cache_flush();
/* turn off I/D-cache */ icache_disable(); dcache_disable(); l2_cache_disable();
- /* flush I/D-cache */
- cache_flush();
- return 0; }
What is this change supposed to fix?
Actually, this is not a issue fix. Maybe my understanding wrong. I think this is just a logic issue. If the cache is disable, then we flush it, will the contents in the cache corrupted?
There is no need to flush before disabling, and actually, flushing before disabling runs the risk that between the two, some cache lines be dirtied again so that cache and memory won't be coherent any more.
I am not fully understand this. In my mind, I think flush cache (writing the dirty data back to memory) is used to keep the coherence. If my understanding is not correct, please help give more explaination or some reference document to me for understanding.
Maybe you're thinking that "disabling" a cache means turning it off, losing all its content? Actually, disabling does not affect the cache lines' content or state; it only 'bypasses' the cache until enabled again. While the cache is disabled, all dirty lines remain dirty, and all non-empty lines remain non-empty.
That being said, yes, flushing is used to keep cache and memory coherent, and this is precisely why the flush must happen after the cache is disabled, not before.
It you flush first then disable, you leave a time window between the two where a write to the cache can happen (either because your code does one, or because the compiler optimized one in). If it happens, then you disable a cache which is still dirty -- IOW, your flushing has failed its mission, and your cache and memory are still not coherent.
Now, if you disable then flush, then any write between the two will go straight to memory without dirtying the cache, and once it is flushed, you end up with coherent cache and memory.
Note that the same time window reason, the cache must be invalidated before it is enabled, not after; invalidating after enabling could cause reads to take old, stale values from the cache rather than fetch the current, fresh ones from memory and caching them.
Thanks.
You're welcome.
Best Regards, Bo Shen
Amicalement,
hi Albert, On Mon Jul 08, 2013 at 12:22:57PM +0200, Albert ARIBAUD wrote:
<snip>
It you flush first then disable, you leave a time window between the two where a write to the cache can happen (either because your code does one, or because the compiler optimized one in). If it happens, then you disable a cache which is still dirty -- IOW, your flushing has failed its mission, and your cache and memory are still not coherent.
Since this is specific to arm926ejs, can we not flush *and* invalidate the dcache before disabling it -- since the arm926ejs cache uses a read allocate policy, flushing and invalidating a cache before disabling it would not result in the cache getting written to in the window that you refer to. Also, flushing and cleaning is an atomic operation.
Now, if you disable then flush, then any write between the two will go straight to memory without dirtying the cache, and once it is flushed, you end up with coherent cache and memory.
I had a question here. Would the same logic not apply to the case you mention -- in case the cache is disabled and subsequently flushed, could there not be a scenario where there is a valid(updated) data that gets written to the memory, which then gets overwritten by the cache flush. Or am i missing something here.
sughosh
Hi Sughosh,
On Mon, 8 Jul 2013 17:38:46 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert, On Mon Jul 08, 2013 at 12:22:57PM +0200, Albert ARIBAUD wrote:
<snip>
It you flush first then disable, you leave a time window between the two where a write to the cache can happen (either because your code does one, or because the compiler optimized one in). If it happens, then you disable a cache which is still dirty -- IOW, your flushing has failed its mission, and your cache and memory are still not coherent.
Since this is specific to arm926ejs, can we not flush *and* invalidate the dcache before disabling it -- since the arm926ejs cache uses a read allocate policy, flushing and invalidating a cache before disabling it would not result in the cache getting written to in the window that you refer to. Also, flushing and cleaning is an atomic operation.
Invalidating the cache in addition to flushing it would not prevent further writes from dirtying the cache lines if they happen before the cache is disabled.
Now, if you disable then flush, then any write between the two will go straight to memory without dirtying the cache, and once it is flushed, you end up with coherent cache and memory.
I had a question here. Would the same logic not apply to the case you mention -- in case the cache is disabled and subsequently flushed, could there not be a scenario where there is a valid(updated) data that gets written to the memory, which then gets overwritten by the cache flush. Or am i missing something here.
There could be such a risk indeed, in the following scenario:
- an address gets written into, causing the new value to be cached; - the cache is disabled; - the same address is written into again, directly into memory; - the flush occurs, overwriting the second value with the first.
This scenario requires two subsequent writes of different values to the same address, which is less likely than the failure scenario of flushing before disabling, which only requires writing a new value once for any address:
- the flush occurs; - an address gets written into, causing the new value to be cached; - the cache is disabled; - the value is lost as the cache will be invalidated before being re-enabled.
I'll grant you that the current code is not zero-risk, if we ever have code that double-writes two different values in the same location. But the proposed RFC increases the risks.
sughosh
Amicalement,
hi Albert,
On Mon Jul 08, 2013 at 02:32:16PM +0200, Albert ARIBAUD wrote:
Hi Sughosh,
On Mon, 8 Jul 2013 17:38:46 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert, On Mon Jul 08, 2013 at 12:22:57PM +0200, Albert ARIBAUD wrote:
<snip>
It you flush first then disable, you leave a time window between the two where a write to the cache can happen (either because your code does one, or because the compiler optimized one in). If it happens, then you disable a cache which is still dirty -- IOW, your flushing has failed its mission, and your cache and memory are still not coherent.
Since this is specific to arm926ejs, can we not flush *and* invalidate the dcache before disabling it -- since the arm926ejs cache uses a read allocate policy, flushing and invalidating a cache before disabling it would not result in the cache getting written to in the window that you refer to. Also, flushing and cleaning is an atomic operation.
Invalidating the cache in addition to flushing it would not prevent further writes from dirtying the cache lines if they happen before the cache is disabled.
I have a doubt on this. The arm926ejs uses a read-allocate policy, wherein a new cache line is allocated only on a read miss -- a write to an address not present in the cache gets written to memory. So if the cache line is invalidated, how will data get written to the cache.
-sughosh
Hi Sughosh,
On Mon, 8 Jul 2013 19:37:22 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert,
On Mon Jul 08, 2013 at 02:32:16PM +0200, Albert ARIBAUD wrote:
Hi Sughosh,
On Mon, 8 Jul 2013 17:38:46 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert, On Mon Jul 08, 2013 at 12:22:57PM +0200, Albert ARIBAUD wrote:
<snip>
It you flush first then disable, you leave a time window between the two where a write to the cache can happen (either because your code does one, or because the compiler optimized one in). If it happens, then you disable a cache which is still dirty -- IOW, your flushing has failed its mission, and your cache and memory are still not coherent.
Since this is specific to arm926ejs, can we not flush *and* invalidate the dcache before disabling it -- since the arm926ejs cache uses a read allocate policy, flushing and invalidating a cache before disabling it would not result in the cache getting written to in the window that you refer to. Also, flushing and cleaning is an atomic operation.
Invalidating the cache in addition to flushing it would not prevent further writes from dirtying the cache lines if they happen before the cache is disabled.
I have a doubt on this. The arm926ejs uses a read-allocate policy, wherein a new cache line is allocated only on a read miss -- a write to an address not present in the cache gets written to memory. So if the cache line is invalidated, how will data get written to the cache.
The arm926ej-s data cache does not have a single fixed policy, and does not have a bypass-on-write policy, only write-through and copy-back.
Other, more complex, policies may be defined, but at the MMU, not cache, level, and those are not constant for all arm926ej-s based SoCs; not even constant for a given SoC as they are configurable at run-time to fit the chosen system addressing map.
(Besides, bypassing the cache for writes and not reads is of little interest for plain DDR caching.)
-sughosh
Amicalement,
hi Albert,
On Mon Jul 08, 2013 at 09:55:51PM +0200, Albert ARIBAUD wrote:
<snip>
Invalidating the cache in addition to flushing it would not prevent further writes from dirtying the cache lines if they happen before the cache is disabled.
I have a doubt on this. The arm926ejs uses a read-allocate policy, wherein a new cache line is allocated only on a read miss -- a write to an address not present in the cache gets written to memory. So if the cache line is invalidated, how will data get written to the cache.
The arm926ej-s data cache does not have a single fixed policy, and does not have a bypass-on-write policy, only write-through and copy-back.
Other, more complex, policies may be defined, but at the MMU, not cache, level, and those are not constant for all arm926ej-s based SoCs; not even constant for a given SoC as they are configurable at run-time to fit the chosen system addressing map.
Can you please elucidate on these policies. Based on my reading of the arm developers manual and the arm926ejs trm, the mmu makes a particular region cacheable and/or write bufferable. I did not find mention of any other policies. Maybe pointers or links to the documents would help.
(Besides, bypassing the cache for writes and not reads is of little interest for plain DDR caching.)
Again, this is independent of the target interface that is being cached(if i've missed something, can you please point me to the document). Thanks.
-sughosh
hi Albert,
On Mon Jul 08, 2013 at 09:55:51PM +0200, Albert ARIBAUD wrote:
<snip>
Invalidating the cache in addition to flushing it would not prevent further writes from dirtying the cache lines if they happen before the cache is disabled.
I have a doubt on this. The arm926ejs uses a read-allocate policy, wherein a new cache line is allocated only on a read miss -- a write to an address not present in the cache gets written to memory. So if the cache line is invalidated, how will data get written to the cache.
The arm926ej-s data cache does not have a single fixed policy, and does not have a bypass-on-write policy, only write-through and copy-back.
Other, more complex, policies may be defined, but at the MMU, not cache, level, and those are not constant for all arm926ej-s based SoCs; not even constant for a given SoC as they are configurable at run-time to fit the chosen system addressing map.
Can you please elucidate on these policies. Based on my reading of the arm developers manual and the arm926ejs trm, the mmu makes a particular region cacheable and/or write bufferable. I did not find mention of any other policies. Maybe pointers or links to the documents would help.
(Besides, bypassing the cache for writes and not reads is of little interest for plain DDR caching.)
Again, afaik this is independent of the target interface that is being cached(if i've missed something, can you please point me to the document). Thanks.
-sughosh
Hi Sughosh,
On Tue, 9 Jul 2013 11:41:34 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert,
On Mon Jul 08, 2013 at 09:55:51PM +0200, Albert ARIBAUD wrote:
<snip>
Invalidating the cache in addition to flushing it would not prevent further writes from dirtying the cache lines if they happen before the cache is disabled.
I have a doubt on this. The arm926ejs uses a read-allocate policy, wherein a new cache line is allocated only on a read miss -- a write to an address not present in the cache gets written to memory. So if the cache line is invalidated, how will data get written to the cache.
The arm926ej-s data cache does not have a single fixed policy, and does not have a bypass-on-write policy, only write-through and copy-back.
Other, more complex, policies may be defined, but at the MMU, not cache, level, and those are not constant for all arm926ej-s based SoCs; not even constant for a given SoC as they are configurable at run-time to fit the chosen system addressing map.
Can you please elucidate on these policies. Based on my reading of the arm developers manual and the arm926ejs trm, the mmu makes a particular region cacheable and/or write bufferable. I did not find mention of any other policies. Maybe pointers or links to the documents would help.
You are correct re the other policies of the DDI0198E (ARM926EJ-S TRM) MMU -- page 3-11, bits 3-2 of the section descriptor. Note however that you may have to refer to your specific SoC's TRM or equivalent, as the SoC designer may have defined its own system-level cache and MMU architecture.
Note in any case that none of the policies mentioned in DDI0198E is described as read-allocate (let alone "read-allocate only" where writes would bypass the enabled cache); on the contrary, the only cache policies mentioned are write-through and write-back, both of which contradict cache bypass on write.
(Besides, bypassing the cache for writes and not reads is of little interest for plain DDR caching.)
Again, afaik this is independent of the target interface that is being cached(if i've missed something, can you please point me to the document). Thanks.
Sorry, I don't understand this last comment of yours wrt my point on the (lack of) interest of bypassing cache for DDR caching.
-sughosh
Amicalement,
hi Albert,
On Tue Jul 09, 2013 at 10:28:13AM +0200, Albert ARIBAUD wrote:
The arm926ej-s data cache does not have a single fixed policy, and does not have a bypass-on-write policy, only write-through and copy-back.
Other, more complex, policies may be defined, but at the MMU, not cache, level, and those are not constant for all arm926ej-s based SoCs; not even constant for a given SoC as they are configurable at run-time to fit the chosen system addressing map.
Can you please elucidate on these policies. Based on my reading of the arm developers manual and the arm926ejs trm, the mmu makes a particular region cacheable and/or write bufferable. I did not find mention of any other policies. Maybe pointers or links to the documents would help.
You are correct re the other policies of the DDI0198E (ARM926EJ-S TRM) MMU -- page 3-11, bits 3-2 of the section descriptor. Note however that you may have to refer to your specific SoC's TRM or equivalent, as the SoC designer may have defined its own system-level cache and MMU architecture.
Note in any case that none of the policies mentioned in DDI0198E is described as read-allocate (let alone "read-allocate only" where writes would bypass the enabled cache); on the contrary, the only cache policies mentioned are write-through and write-back, both of which contradict cache bypass on write.
I was referring to the cache allocation policy mentioned in section 4.1 in the DDI0198E document -- this is also mentioned in table 12.1 in chapter 12 of the arm developers guide.
(Besides, bypassing the cache for writes and not reads is of little interest for plain DDR caching.)
Again, afaik this is independent of the target interface that is being cached(if i've missed something, can you please point me to the document). Thanks.
Sorry, I don't understand this last comment of yours wrt my point on the (lack of) interest of bypassing cache for DDR caching.
What i meant to state was that i did not find any mention that the cache real allocate policy did not apply for DDR caching.
-sughosh
Hi Sughosh,
On Wed, 10 Jul 2013 15:35:10 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert,
On Tue Jul 09, 2013 at 10:28:13AM +0200, Albert ARIBAUD wrote:
The arm926ej-s data cache does not have a single fixed policy, and does not have a bypass-on-write policy, only write-through and copy-back.
Other, more complex, policies may be defined, but at the MMU, not cache, level, and those are not constant for all arm926ej-s based SoCs; not even constant for a given SoC as they are configurable at run-time to fit the chosen system addressing map.
Can you please elucidate on these policies. Based on my reading of the arm developers manual and the arm926ejs trm, the mmu makes a particular region cacheable and/or write bufferable. I did not find mention of any other policies. Maybe pointers or links to the documents would help.
You are correct re the other policies of the DDI0198E (ARM926EJ-S TRM) MMU -- page 3-11, bits 3-2 of the section descriptor. Note however that you may have to refer to your specific SoC's TRM or equivalent, as the SoC designer may have defined its own system-level cache and MMU architecture.
Note in any case that none of the policies mentioned in DDI0198E is described as read-allocate (let alone "read-allocate only" where writes would bypass the enabled cache); on the contrary, the only cache policies mentioned are write-through and write-back, both of which contradict cache bypass on write.
I was referring to the cache allocation policy mentioned in section 4.1 in the DDI0198E document -- this is also mentioned in table 12.1 in chapter 12 of the arm developers guide.
Can you please quote the exact part of 4.1 which describes the cache policy and then explain what you think it means exactly?
-sughosh
Amicalement,
hi Albert,
On Wed Jul 10, 2013 at 02:30:30PM +0200, Albert ARIBAUD wrote:
You are correct re the other policies of the DDI0198E (ARM926EJ-S TRM) MMU -- page 3-11, bits 3-2 of the section descriptor. Note however that you may have to refer to your specific SoC's TRM or equivalent, as the SoC designer may have defined its own system-level cache and MMU architecture.
Note in any case that none of the policies mentioned in DDI0198E is described as read-allocate (let alone "read-allocate only" where writes would bypass the enabled cache); on the contrary, the only cache policies mentioned are write-through and write-back, both of which contradict cache bypass on write.
I was referring to the cache allocation policy mentioned in section 4.1 in the DDI0198E document -- this is also mentioned in table 12.1 in chapter 12 of the arm developers guide.
Can you please quote the exact part of 4.1 which describes the cache policy and then explain what you think it means exactly?
I was referring to this particular point in section 4.1 "Allocate on read-miss is supported. The caches perform critical-word first cache refilling."
Based on the cache line allocation policies described in section 12.3.3 in the arm developers guide, my interpretation of 'allocate on read-miss' was as above.
-sughosh
Hi Sughosh,
On Wed, 10 Jul 2013 23:04:45 +0530, Sughosh Ganu urwithsughosh@gmail.com wrote:
hi Albert,
On Wed Jul 10, 2013 at 02:30:30PM +0200, Albert ARIBAUD wrote:
You are correct re the other policies of the DDI0198E (ARM926EJ-S TRM) MMU -- page 3-11, bits 3-2 of the section descriptor. Note however that you may have to refer to your specific SoC's TRM or equivalent, as the SoC designer may have defined its own system-level cache and MMU architecture.
Note in any case that none of the policies mentioned in DDI0198E is described as read-allocate (let alone "read-allocate only" where writes would bypass the enabled cache); on the contrary, the only cache policies mentioned are write-through and write-back, both of which contradict cache bypass on write.
I was referring to the cache allocation policy mentioned in section 4.1 in the DDI0198E document -- this is also mentioned in table 12.1 in chapter 12 of the arm developers guide.
Can you please quote the exact part of 4.1 which describes the cache policy and then explain what you think it means exactly?
I was referring to this particular point in section 4.1 "Allocate on read-miss is supported. The caches perform critical-word first cache refilling."
This is not equivalent to "the cache policy is read-allocate".
This paragraph does not specify the cache's general policy, only a read policy, and only the *read allocation* policy. Its write policies are not defined or even constrained by this paragraph.
Based on the cache line allocation policies described in section 12.3.3 in the arm developers guide, my interpretation of 'allocate on read-miss' was as above.
The ARM926EJ-S is "allocate on read-miss" as well as "allocate on write-miss"; and none of its write policies bypasses the cache.
-sughosh
Amicalement,
hi Albert, On Mon Jul 08, 2013 at 12:22:57PM +0200, Albert ARIBAUD wrote:
<snip>
It you flush first then disable, you leave a time window between the two where a write to the cache can happen (either because your code does one, or because the compiler optimized one in). If it happens, then you disable a cache which is still dirty -- IOW, your flushing has failed its mission, and your cache and memory are still not coherent.
Since this is specific to arm926ejs, can we not flush *and* invalidate the dcache before disabling it -- since the arm926ejs cache uses a read allocate policy, flushing and invalidating a cache before disabling it would not result in the cache getting written to in the window that you refer to. Also, flushing and invalidating is an atomic operation.
Now, if you disable then flush, then any write between the two will go straight to memory without dirtying the cache, and once it is flushed, you end up with coherent cache and memory.
I had a question here. Would the same logic not apply to the case you mention -- in case the cache is disabled and subsequently flushed, could there not be a scenario where there is a valid(updated) data that gets written to the memory, which then gets overwritten by the cache flush. Or am i missing something here.
-sughosh
participants (3)
-
Albert ARIBAUD -
Bo Shen -
Sughosh Ganu