[U-Boot] OMAP3 performance regression in 2011.12
Commit "armv7: disable L2 cache in cleanup_before_linux()" on 6th Dec 2011 by Aneesh V adds the following:
arch/arm/cpu/armv7/cpu.c:cleanup_before_linux()
... v7_out_cache_disable(); ...
The commit message implies this change was to make booting reliable on OMAP4 by disabling L2 cache before jumping to Linux.
However, when running with a stock 3.2 Linux kernel on an OMAP3 it has the effect of massively reducing system performance (when running using an OMAP3- only 3.2 Linux Kernel on a GUSMTIX Overo OMAP3530).
Therefore, I assume this means that the kernel isn't turning the L2 cache back on for an OMAP3 (at least with my kernel build)!
So, my question is...
Are there any Kconfig options in Linux that will re-enable the L2 cache (something obvious that I've missed), or is this commit just bad-news for OMAP3?
Cheers, Joe
On Mon, Jan 9, 2012 at 3:27 AM, Joe Woodward jw@terrafix.co.uk wrote:
Commit "armv7: disable L2 cache in cleanup_before_linux()" on 6th Dec 2011 by Aneesh V adds the following:
arch/arm/cpu/armv7/cpu.c:cleanup_before_linux()
... v7_out_cache_disable(); ...
The commit message implies this change was to make booting reliable on OMAP4 by disabling L2 cache before jumping to Linux.
However, when running with a stock 3.2 Linux kernel on an OMAP3 it has the effect of massively reducing system performance (when running using an OMAP3- only 3.2 Linux Kernel on a GUSMTIX Overo OMAP3530).
Therefore, I assume this means that the kernel isn't turning the L2 cache back on for an OMAP3 (at least with my kernel build)!
So, my question is...
Are there any Kconfig options in Linux that will re-enable the L2 cache (something obvious that I've missed), or is this commit just bad-news for OMAP3?
Are you certain that this is the commit that's causing your problem? The kernel is responsible for turning the cache back on and has for a long time, iirc.
I'm fairly certain...
If I take the 2011.12 uBoot release the kernel takes about twice the time to boot (compared to 2011.09), and the device is noticably slower.
Then if I comment out the v7_out_cache_disable() line in cpu.c and rebuild uBoot then everything speeds up again.
I thought the kernel would turn on the cache again too...
Is there any easy way from userspace to determine if the cache is on?
I did a bit of Googling and found: http://www.spinics.net/lists/arm-kernel/msg50083.html
It may be that the kernel is re-enabling the L1 cache, but expecting L2 to be on? Or the way v7_out_cache_disable() disables L2 is not compatible with the way the kernel expects to re-enable it?
Cheers, Joe
-----Original Message----- From: Tom Rini tom.rini@gmail.com To: Joe Woodward jw@terrafix.co.uk Cc: u-boot@lists.denx.de Date: Mon, 9 Jan 2012 08:11:07 -0700 Subject: Re: [U-Boot] OMAP3 performance regression in 2011.12
On Mon, Jan 9, 2012 at 3:27 AM, Joe Woodward jw@terrafix.co.uk wrote:
Commit "armv7: disable L2 cache in cleanup_before_linux()" on 6th Dec
2011 by Aneesh V adds the following:
arch/arm/cpu/armv7/cpu.c:cleanup_before_linux()
... v7_out_cache_disable(); ...
The commit message implies this change was to make booting reliable
on OMAP4 by disabling L2 cache before jumping to Linux.
However, when running with a stock 3.2 Linux kernel on an OMAP3 it
has the effect of massively reducing system performance (when running using an OMAP3-
only 3.2 Linux Kernel on a GUSMTIX Overo OMAP3530).
Therefore, I assume this means that the kernel isn't turning the L2
cache back on for an OMAP3 (at least with my kernel build)!
So, my question is...
Are there any Kconfig options in Linux that will re-enable the L2
cache (something obvious that I've missed), or is this commit just bad-news for OMAP3?
Are you certain that this is the commit that's causing your problem? The kernel is responsible for turning the cache back on and has for a long time, iirc.
-- Tom
-----Original Message----- From: Tom Rini tom.rini@gmail.com To: Joe Woodward jw@terrafix.co.uk Cc: u-boot@lists.denx.de Date: Mon, 9 Jan 2012 08:11:07 -0700 Subject: Re: [U-Boot] OMAP3 performance regression in 2011.12
On Mon, Jan 9, 2012 at 3:27 AM, Joe Woodward jw@terrafix.co.uk
wrote:
Commit "armv7: disable L2 cache in cleanup_before_linux()" on 6th
Dec
2011 by Aneesh V adds the following:
arch/arm/cpu/armv7/cpu.c:cleanup_before_linux()
... v7_out_cache_disable(); ...
The commit message implies this change was to make booting reliable
on OMAP4 by disabling L2 cache before jumping to Linux.
However, when running with a stock 3.2 Linux kernel on an OMAP3 it
has the effect of massively reducing system performance (when running using an OMAP3-
only 3.2 Linux Kernel on a GUSMTIX Overo OMAP3530).
Therefore, I assume this means that the kernel isn't turning the L2
cache back on for an OMAP3 (at least with my kernel build)!
So, my question is...
Are there any Kconfig options in Linux that will re-enable the L2
cache (something obvious that I've missed), or is this commit just bad-news for OMAP3?
Are you certain that this is the commit that's causing your problem? The kernel is responsible for turning the cache back on and has for a long time, iirc.
-- Tom
(apologies for previous top posting, wasn't paying attention to what I was doing!)
I'm fairly certain...
If I take the 2011.12 uBoot release the kernel takes about twice the time to boot (compared to 2011.09), and the device is noticably slower.
Then if I comment out the v7_out_cache_disable() line in cpu.c and rebuild uBoot then everything speeds up again.
I thought the kernel would turn on the cache again too...
Is there any easy way from userspace to determine if the cache is on?
I did a bit of Googling and found: http://www.spinics.net/lists/arm-kernel/msg50064.html http://www.spinics.net/lists/arm-kernel/msg50083.html
It may be that the kernel is re-enabling the L1 cache, but expecting L2 to be on? Or the way v7_out_cache_disable() disables L2 is not compatible with the way the kernel expects to re-enable it?
Also, in the Linux there seem to be OMAP4 specific functions for re-enabling the L2 cache (omap4-common.c:omap_l2_cache_init()), but none for OMAP3. I'm assuming this is because up to now OMAP3 is assumed to have the L2 left enabled? Either that, or there is some generic Cortex-A8 method for enabling the L2 cache in the kernel soures that I've not found...
Cheers, Joe
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Hi Joe,
On Monday 09 January 2012 09:18 PM, Joe Woodward wrote:
<snip>
(apologies for previous top posting, wasn't paying attention to what I was doing!)
I'm fairly certain...
If I take the 2011.12 uBoot release the kernel takes about twice the time to boot (compared to 2011.09), and the device is noticably slower.
Then if I comment out the v7_out_cache_disable() line in cpu.c and rebuild uBoot then everything speeds up again.
I thought the kernel would turn on the cache again too...
Is there any easy way from userspace to determine if the cache is on?
It won't be easy I believe, unless you have a debugger that allows you to see CP15 registers.
I did a bit of Googling and found: http://www.spinics.net/lists/arm-kernel/msg50064.html http://www.spinics.net/lists/arm-kernel/msg50083.html
It may be that the kernel is re-enabling the L1 cache, but expecting L2 to be on?
Ideally kernel should be enabling L2 too. But looks like L2 was enabled by ROM code itself in OMAP3, but D-cache disabled globally. So, it gets enabled as soon as the D-cache is enabled globally.
L2$ in OMAP3 is a bit tricky. The cache is known to ARM but enabling/disabling it and affecting secure entries needs ROM assistance. So, while ARMv7 generic code can flush L2, we need OMAP specific code to enable/disable it.
So, before my patch and before cache-support came to U-boot L2$ always remained enabled. Since de-compressor's flush indeed flushed L2$ we didn't get into any coherency problems.
Let the discussion in l-o conclude. I think fixing it in kernel is the right thing to do. But if that's not possible, we can have a patch in U-boot to work around it.
br, Aneesh
Aneesh V aneesh@ti.com writes:
Hi Joe,
On Monday 09 January 2012 09:18 PM, Joe Woodward wrote:
If I take the 2011.12 uBoot release the kernel takes about twice the time to boot (compared to 2011.09), and the device is noticably slower.
Then if I comment out the v7_out_cache_disable() line in cpu.c and rebuild uBoot then everything speeds up again.
I thought the kernel would turn on the cache again too... [...] I did a bit of Googling and found: http://www.spinics.net/lists/arm-kernel/msg50064.html http://www.spinics.net/lists/arm-kernel/msg50083.html
It may be that the kernel is re-enabling the L1 cache, but expecting L2 to be on?
Ideally kernel should be enabling L2 too. But looks like L2 was enabled by ROM code itself in OMAP3, but D-cache disabled globally. So, it gets enabled as soon as the D-cache is enabled globally.
L2$ in OMAP3 is a bit tricky. The cache is known to ARM but enabling/disabling it and affecting secure entries needs ROM assistance. So, while ARMv7 generic code can flush L2, we need OMAP specific code to enable/disable it.
On OMAP3 ES2 and later, the L2EN bit in the auxiliary control register is banked between secure and non-secure modes, so there is no need for ROM calls to enable/disable the L2$ on these chips.
On Tuesday 17 January 2012 08:21 PM, Måns Rullgård wrote:
Aneesh Vaneesh@ti.com writes:
Hi Joe,
On Monday 09 January 2012 09:18 PM, Joe Woodward wrote:
If I take the 2011.12 uBoot release the kernel takes about twice the time to boot (compared to 2011.09), and the device is noticably slower.
Then if I comment out the v7_out_cache_disable() line in cpu.c and rebuild uBoot then everything speeds up again.
I thought the kernel would turn on the cache again too... [...] I did a bit of Googling and found: http://www.spinics.net/lists/arm-kernel/msg50064.html http://www.spinics.net/lists/arm-kernel/msg50083.html
It may be that the kernel is re-enabling the L1 cache, but expecting L2 to be on?
Ideally kernel should be enabling L2 too. But looks like L2 was enabled by ROM code itself in OMAP3, but D-cache disabled globally. So, it gets enabled as soon as the D-cache is enabled globally.
L2$ in OMAP3 is a bit tricky. The cache is known to ARM but enabling/disabling it and affecting secure entries needs ROM assistance. So, while ARMv7 generic code can flush L2, we need OMAP specific code to enable/disable it.
On OMAP3 ES2 and later, the L2EN bit in the auxiliary control register is banked between secure and non-secure modes, so there is no need for ROM calls to enable/disable the L2$ on these chips.
Yes, but IIRC, there was an erratum around it in some ARM revisions (I think 3430 ES2 was affected) and the workaround was to keep both the banked bits at the same value always. So, to keep things simple and working for all revisions, I try to change both together. In the U-Boot code where this is done I have added a comment on this.
participants (4)
-
Aneesh V -
Joe Woodward -
Måns Rullgård -
Tom Rini