[RESEND PATCH] kconfig: Proposed language extension for multiple builds
(I am sending this again to get more feedback)
In the case of Linux, only one build is produced so there is only a single configuration. For other projects, such as U-Boot and Zephyr, the same code is used to produce multiple builds, each with related (but different) options enabled.
This can be handled with the existing kconfig language, but it is quite verbose, somewhat tedious and very error-prone, since there is a lot of duplication. The result is hard to maintain.
Describe an extension to the Kconfig language to support easier handling of this use case.
Signed-off-by: Simon Glass sjg@chromium.org ---
Documentation/kbuild/kconfig-language.rst | 134 ++++++++++++++++++++++ 1 file changed, 134 insertions(+)
diff --git a/Documentation/kbuild/kconfig-language.rst b/Documentation/kbuild/kconfig-language.rst index 858ed5d80defe..73fb016a5533f 100644 --- a/Documentation/kbuild/kconfig-language.rst +++ b/Documentation/kbuild/kconfig-language.rst @@ -228,6 +228,24 @@ applicable everywhere (see syntax). enables the third modular state for all config symbols. At most one symbol may have the "modules" option set.
+- phase declaration: "defphase" + This defines a new build phase. See `Build Phases`_. + +- default phase: "phasedefault" + This indicates the default build phase. See `Build Phases`_. + +- add entries for phases: "addphases" + This creates new phase-specific entries based on a template entry and adds + the same attributes to it. See `Build Phases`_. + +- set entries for phases: "setphases" + This sets the phases which need an entry. This allows creating an entry that + only has a primary phase. See `Build Phases`_. + +- indicate a phase-specific attribute: "forphases" + This marks an attribute as being applicable only to a particular phase or + group of phases. See `Build Phases`_. + Menu dependencies -----------------
@@ -319,6 +337,119 @@ MODVERSIONS directly depends on MODULES, this means it's only visible if MODULES is different from 'n'. The comment on the other hand is only visible when MODULES is set to 'n'.
+Build Phases +------------ + +Some projects use Kconfig to control multiple build phases, each phase +resulting in a separate set of object files and executable. This is the +case in U-Boot [12]_. Zephyr OS seems to be heading this way too [13]_. + +Generally the phases are related, so that enabling an entry in the primary +phase also enables it by default in the others. But in some cases it may +be desirable to use separate conditions for each phase. + +All phases have a phase name, for example `SPL`. This name is used as a +prefix to each entry used in that phase, with an underscore in between. +So if FOO is the primary entry, the equivalent entry for the SPL phase +is SPL_FOO. The primary phase is marked with a "phasedefault" entry. + +Phases are declared like any other menu entry except that also have a +"defphase" keyword. Phase entries are normally hidden so do not have a +prompt:: + + config PPL + bool + defphase "Primary Program Loader" + phasedefault + help + This is the primary bootloader. + + config SPL + bool + defphase "Secondary Program Loader" + help + This is used to set up memory and load the primary bootloader. + +The default phase (here PPL) is assumed for all entries, in the sense that +all entries are present in PPL by default and no prefix is needed on these +entries. So FOO means that it applies to PPL. There must be exactly one +default phase. + +The resulting menu entries can be used normally throughout the Kconfig. With +this technique, the different build phases can be fully and individually +controlled from Kconfig. + +However it is not ideal. Often the secondary phases have far fewer entries than +the primary phase, since they offer fewer features. Even so, each FOO that is +needed in a phase must have an SPL_FOO, etc. To avoid an explosion of entries, +it is possible to indicate which are enabled, as a shortcut for creating new +entries:: + + config FOO + bool "Enable foo feature" + addphases SPL + depends on %BAR + depends on QUX + forphases SPL depends on FIZZ + +Note that "%" expands to the phase, so this is equivalent to (ignoring BAR):: + + config FOO + bool "Enable foo feature" + depends on BAR + depends on QUX + + config SPL_FOO # Phase is prepended + bool "Enable foo feature (SPL)" # Suffix is added + depends on SPL_BAR # "%" dependency is expanded + depends on QUX + depends on FIZZ # Added only for SPL + depends on SPL # Added automatically + +Attributes declared in the primary symbol FOO (such as "depends on BAR") also +apply to the secondary ones. + +An entry without any 'addphases' attribute applies to all phases. Individual +phase entries are not available in that case. If the entry is enabled, then +it is enabled for all phases. Only one entry appears in the resulting Kconfig. + +In the case where an entry should apply only to the primary phase (or a +particular set of phases), you can uses "setphases" instead of "addphases":: + + config FOO + bool "Enable foo feature" + setphases PPL + +This means that even if the option is enabled, it will not be active outside +the primary-phase build, here named "PPL". + +Internally, phases are implemented simply by creating new entries. These +appear in the Kconfig as per normal. It would be possible for a Kconfig +editor to show the entries just for a particular phase, leaving out the +entries not applicable to that phase. + +When phases are used, the Kconfig tool outputs separate auto.conf files for +each phase (e.g. auto_spl.conf), so that if SPL_FOO is enabled, then +`CONFIG_FOO=y` is present in the file. This makes it easy for the build system +to build the correct code, use IS_ENABLED(), etc. + +To ensure that the correct options are enabled for each build, in addition to +the normal CONFIG_FOO option in the file, phase symbols are added too. For +example, if FOO is enabled in the SPL phase, then auto_spl.conf contains:: + + CONFIG_FOO=y + CONFIG_SPL_FOO=y + +The phase-specific line (CONFIG_SPL_FOO) is seldom needed, but it allows one +phase to access the symbols from another phase. For example, if the primary +phase needs to receive boot timings from SPL, then support for boot timings must +be added in both phases. If the timings are at a fixed address, this can be in a +shared symbol (like CONFIG_SPL_TIMING_BASE) that both phases can access. + +Technical note: the grammar definition of <symbol> in this documeent does not +include the "%" prefix at present. It can be used with any attribute. It cannot +be used with any top-level items, like "config", "menuconfig", "if" and "menu". +
Kconfig syntax -------------- @@ -744,3 +875,6 @@ https://kernelnewbies.org/KernelProjects/kconfig-sat .. [9] https://www4.cs.fau.de/Publications/2011/tartler_11_eurosys.pdf .. [10] https://paulgazzillo.com/papers/esecfse21.pdf .. [11] https://github.com/paulgazz/kmax + +.. [12] https://u-boot.readthedocs.io/en/latest/develop/spl.html +.. [13] https://docs.zephyrproject.org/latest/build/sysbuild/index.html
Hi--
On 3/10/23 18:37, Simon Glass wrote:
(I am sending this again to get more feedback)
In the case of Linux, only one build is produced so there is only a single configuration. For other projects, such as U-Boot and Zephyr, the same code is used to produce multiple builds, each with related (but different) options enabled.
This can be handled with the existing kconfig language, but it is quite verbose, somewhat tedious and very error-prone, since there is a lot of duplication. The result is hard to maintain.
Describe an extension to the Kconfig language to support easier handling of this use case.
Signed-off-by: Simon Glass sjg@chromium.org
IMO Masahiro has already answered this multiple times and I agree with his answers.
For others, the full previous thread is at https://lore.kernel.org/all/20230219145453.1.Idaaf79c3e768b85750d5a7eb732052...
Documentation/kbuild/kconfig-language.rst | 134 ++++++++++++++++++++++ 1 file changed, 134 insertions(+)
On Fri, Mar 10, 2023 at 09:39:15PM -0800, Randy Dunlap wrote:
Hi--
On 3/10/23 18:37, Simon Glass wrote:
(I am sending this again to get more feedback)
In the case of Linux, only one build is produced so there is only a single configuration. For other projects, such as U-Boot and Zephyr, the same code is used to produce multiple builds, each with related (but different) options enabled.
This can be handled with the existing kconfig language, but it is quite verbose, somewhat tedious and very error-prone, since there is a lot of duplication. The result is hard to maintain.
Describe an extension to the Kconfig language to support easier handling of this use case.
Signed-off-by: Simon Glass sjg@chromium.org
IMO Masahiro has already answered this multiple times and I agree with his answers.
For others, the full previous thread is at https://lore.kernel.org/all/20230219145453.1.Idaaf79c3e768b85750d5a7eb732052...
Well, I think what was unclear, or maybe we just wanted to confirm the answer was "none at all", was this. As good community neighbors, we see a generic issue in the Kconfig language, a tool used frequently outside of just the Linux kernel, and would like to contribute back. Ideally without having first gone off, designed and implemented something, and then been told it's all wrong and to rewrite it first. So what level of interest is there in this?
As I pointed out in that thread, I believe barebox has examples where some keyword like we're proposing here would help them (and yes, there's only a dozen or so symbols so it's also manageable without anything special), and Simon believes Zephyr will be in a similar situation soon enough (which doesn't use the kernel's implementation of the language). Frankly, I keep going back to "tristate" is just the original example of what we're talking about here (CONFIG_FOO=n, CONFIG_FOO_MODULE=y), not that I'm suggesting we would remove the tristate word. So we would really like to make sure as many people and projects are aware, as possible.
And as Simon asked in the thread, what about code refactoring that makes further maintenance easier? Clearly, such patches would need to be against the current appropriate tree.
Tom Rini trini@konsulko.com writes:
On Fri, Mar 10, 2023 at 09:39:15PM -0800, Randy Dunlap wrote:
Hi--
On 3/10/23 18:37, Simon Glass wrote:
(I am sending this again to get more feedback)
In the case of Linux, only one build is produced so there is only a single configuration. For other projects, such as U-Boot and Zephyr, the same code is used to produce multiple builds, each with related (but different) options enabled.
This can be handled with the existing kconfig language, but it is quite verbose, somewhat tedious and very error-prone, since there is a lot of duplication. The result is hard to maintain.
Describe an extension to the Kconfig language to support easier handling of this use case.
Signed-off-by: Simon Glass sjg@chromium.org
IMO Masahiro has already answered this multiple times and I agree with his answers.
For others, the full previous thread is at https://lore.kernel.org/all/20230219145453.1.Idaaf79c3e768b85750d5a7eb732052...
So what level of interest is there in this?
Unlike Masahiro & co I am interested in generalizing Kconfig to be usable outside of the Linux kernel (for example, I've integrated it into the build2 build system[1]). However, in this case, I tend to agree with Randy and Masahiro: this feels like a very niche use-case (which I am still not 100% clear on, after reading the description 3 times) that would add quite a bit of complexity.
One thing that did cross my mind during those 3 reads is that maybe the essence of the feature you are looking for here is to be able to use a value from the previous phase as "initial" (i.e., stronger than Kconfig default but weaker than user-specified) when configuring the next phase. It probably won't allow you to solve your problem in an auto-magical way like your proposal, but perhaps you could get close enough while not complicating the Kconfig language significantly.
And as Simon asked in the thread, what about code refactoring that makes further maintenance easier?
From my experience, there is little interest in patches that make Kconfig
more general or improve the compatibility of the implementation. As a result, I am maintaining my patch set[2] out of tree.
[1] https://build2.org/libbuild2-kconfig/doc/build2-kconfig-manual.xhtml [2] https://github.com/build2-packaging/kconfig/tree/upstream-5.15-rc7
On Sun, Mar 12, 2023 at 1:55 AM Tom Rini trini@konsulko.com wrote:
On Fri, Mar 10, 2023 at 09:39:15PM -0800, Randy Dunlap wrote:
Hi--
On 3/10/23 18:37, Simon Glass wrote:
(I am sending this again to get more feedback)
In the case of Linux, only one build is produced so there is only a single configuration. For other projects, such as U-Boot and Zephyr, the same code is used to produce multiple builds, each with related (but different) options enabled.
This can be handled with the existing kconfig language, but it is quite verbose, somewhat tedious and very error-prone, since there is a lot of duplication. The result is hard to maintain.
Describe an extension to the Kconfig language to support easier handling of this use case.
Signed-off-by: Simon Glass sjg@chromium.org
IMO Masahiro has already answered this multiple times and I agree with his answers.
For others, the full previous thread is at https://lore.kernel.org/all/20230219145453.1.Idaaf79c3e768b85750d5a7eb732052...
Well, I think what was unclear, or maybe we just wanted to confirm the answer was "none at all", was this. As good community neighbors, we see a generic issue in the Kconfig language, a tool used frequently outside of just the Linux kernel, and would like to contribute back. Ideally without having first gone off, designed and implemented something, and then been told it's all wrong and to rewrite it first. So what level of interest is there in this?
Sorry, no interest. If you want to get a clear answer, NACK.
As I pointed out in that thread, I believe barebox has examples where some keyword like we're proposing here would help them (and yes, there's only a dozen or so symbols so it's also manageable without anything special),
Barebox keeps PBL in very limited, ad-hoc implementation. PBL has no more than 10 user-configurable options. Sascha Hauer designed it this way.
Linux kernel also has a small loader (a.k.a decompressor) in arch/*/boot/decompress/.
For example, CONFIG_KERNEL_GZIP is a CONFIG option for the decompressor instead of the main kernel.
In this sense, you could apply your theory, "Linux kernel is also multi build-phases, so Kconfig should have this extension to move CONFIG_KERNEL_GZIP to another build phase". No, no. The main kernel and the decompressor are well separated and the latter is small and simple.
Barebox is the same - the main Barebox and PBL are well separated and PBL is really small and simple.
The problems you are suffering from do not exist in Barebox.
and Simon believes Zephyr will be in a similar situation soon enough (which doesn't use the kernel's implementation of the language).
Zephyr does not share any Kconfig code with Linux. They use Python implementation, a.k.a. Kconfiglib. It is up to the Zephyr community, but this requires extra effort.
Frankly, I keep going back to "tristate" is just the original example of what we're talking about here (CONFIG_FOO=n, CONFIG_FOO_MODULE=y), not that I'm suggesting we would remove the tristate word. So we would really like to make sure as many people and projects are aware, as possible.
This is on the boundary. We can make the tristate optional if it does not make the code too ugly.
But, if you do not add CONFIG_MODULES in your Kconfig file, users will not see 'm' in the first place.
I know some help messages still mention 'm', but is this the problem you want to solve?
And as Simon asked in the thread, what about code refactoring that makes further maintenance easier? Clearly, such patches would need to be against the current appropriate tree.
If such patches clean up the code, they will be appreciated.
participants (5)
-
Boris Kolpackov -
Masahiro Yamada -
Randy Dunlap -
Simon Glass -
Tom Rini