[U-Boot] [RFC] Kbuild support for ARM FIT images
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
Currently the following is what I envision the way it will work: - New "uImage-fit" target in $(boot)/ which is made to depend on "dtbs" target - Changes to dtbs target to output a $(srctree)/.tmp_list_dtbs listing the full paths to all the dtbs built - A new script "mkfitsrc.sh" that spits out an ITS to $(srctree)/.tmp_its . The above tmp file is passed - For each dtb, a new "device tree" node is created in the ITS - For each dtb, a new "configuration" node is created in the ITS - mkimage is called on the ITS sources to output the kernel image. (TODO: to use mkuboot.sh)
I have yet to do some boot time tests, as I don't have my board now and don't want to post something that doesn't work. But all this has been build-tested.
Any comments on this approach? Is it better to merge mkfitsrc.sh with mkuboot.sh?
Regards, Joel
On 02/20/2013 06:37 PM, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
To my mind, FIT is pointless. And forcing the kernel build process to create bootloader-specific files doesn't seem like a good idea. Doing so would require pulling in even more outside tools into the kernel build flow.
All you need is to copy the zImage and any relevant .dtb files into /boot, and have U-Boot load the relevant .dtb file by constructing the filename as roughly ${soc}-${board}.dtb, then use the bootz command to boot it. You can have a completely generic boot.scr (or built-in script).
Note: Not all (many?) U-Boot support FIT anyway, so you'd need to flash a new U-Boot to support FIT, so you may as well just flash a new U-Boot that implements the ${soc} and ${board} variables instead. IIRC, there may also be a ${boardname} or similar that's like ${board}, but represents the runtime-detected board for when one U-Boot build actually supports multiple different boards.
On Wed, Feb 20, 2013 at 10:26 PM, Stephen Warren swarren@wwwdotorg.org wrote:
On 02/20/2013 06:37 PM, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
To my mind, FIT is pointless. And forcing the kernel build process to
I don't think so, there are many usecases for FIT and cleanly embedding DTBs is a common usecase, (along with stronger checksums, compression). That is also the basis of my ELC talk tomorrow morning ;-) You could peruse the slides at, http://wwwelinux.org/Fit-boot to get a better understanding of what FIT solves. With the move to multiplatform kernels, it is quite a logical next step to add this support to the kernel build.
create bootloader-specific files doesn't seem like a good idea. Doing so would require pulling in even more outside tools into the kernel build flow.
As for pulling additional tools, dtc is already a part of the kernel so the idea is to use existing tools.
All you need is to copy the zImage and any relevant .dtb files into /boot, and have U-Boot load the relevant .dtb file by constructing the filename as roughly ${soc}-${board}.dtb, then use the bootz command to boot it. You can have a completely generic boot.scr (or built-in script).
FIT doesnt need any such bootloader variables and thus benefits from the simplicity.
Note: Not all (many?) U-Boot support FIT anyway, so you'd need to flash
For these boards, we can special case to doing a non-fit boot to keep them backward-compatibility. I don't think maintaining backward-compatibility requires any additional work, but just building the kernel with a different make target.
Regards, Joel
On 02/21/2013 12:15 AM, Joel A Fernandes wrote:
On Wed, Feb 20, 2013 at 10:26 PM, Stephen Warren swarren@wwwdotorg.org wrote:
On 02/20/2013 06:37 PM, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
To my mind, FIT is pointless. And forcing the kernel build process to
I don't think so, there are many usecases for FIT and cleanly embedding DTBs is a common usecase, (along with stronger checksums, compression). That is also the basis of my ELC talk tomorrow morning ;-) You could peruse the slides at, http://wwwelinux.org/Fit-boot to get a better understanding of what FIT solves. With the move to multiplatform kernels, it is quite a logical next step to add this support to the kernel build.
create bootloader-specific files doesn't seem like a good idea. Doing so would require pulling in even more outside tools into the kernel build flow.
As for pulling additional tools, dtc is already a part of the kernel so the idea is to use existing tools.
Well, dtc won't be part of the kernel build process for ARM at least once the device tree files are move outside the kernel.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/20/2013 11:26 PM, Stephen Warren wrote:
On 02/20/2013 06:37 PM, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
To my mind, FIT is pointless. And forcing the kernel build process to create bootloader-specific files doesn't seem like a good idea. Doing so would require pulling in even more outside tools into the kernel build flow.
No, it requires no more tools than we have today. It's still just mkimage and a device tree file.
All you need is to copy the zImage and any relevant .dtb files into /boot, and have U-Boot load the relevant .dtb file by constructing the filename as roughly ${soc}-${board}.dtb, then use the bootz command to boot it. You can have a completely generic boot.scr (or built-in script).
Note you still have to copy N dtb files into the filesystem. Or one file, the FIT image.
Note: Not all (many?) U-Boot support FIT anyway, so you'd need to flash a new U-Boot to support FIT, so you may as well just flash a new U-Boot that implements the ${soc} and ${board} variables instead. IIRC, there may also be a ${boardname} or similar that's like ${board}, but represents the runtime-detected board for when one U-Boot build actually supports multiple different boards.
And enable bootz as well. Just about as many boards enable that as do FIT. And FIT has been around for years, bootz not so. So in theory folks with old/odd boards that didn't bother to get mainlined in U-Boot could still have FIT support added, easily.
- -- Tom
On 02/21/2013 06:29 AM, Tom Rini wrote:
On 02/20/2013 11:26 PM, Stephen Warren wrote:
On 02/20/2013 06:37 PM, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
To my mind, FIT is pointless. And forcing the kernel build process to create bootloader-specific files doesn't seem like a good idea. Doing so would require pulling in even more outside tools into the kernel build flow.
No, it requires no more tools than we have today. It's still just mkimage and a device tree file.
mkimage isn't required today, if you use the bootz target rather than bootm. Using bootz is a huge improvement since you don't need to use bootloader-specific tools to create the kernel boot images; just slap them on the disk, and you're done. Nothing could be simpler.
All you need is to copy the zImage and any relevant .dtb files into /boot, and have U-Boot load the relevant .dtb file by constructing the filename as roughly ${soc}-${board}.dtb, then use the bootz command to boot it. You can have a completely generic boot.scr (or built-in script).
Note you still have to copy N dtb files into the filesystem. Or one file, the FIT image.
But you have to generate the FIT image that packages all the files together.
The long-term goal for device tree is to move the *.dts files out of the kernel source. It's unclear if there will be some single central repository for them, or whether vendors will host their own repository just for their own boards. The latter would probably scale the best. Having separate distro packages for different vendors' or different SoCs' device tree would also scale the best; forcing usage of FIT where everything gets merged together doesn't interact well with that.
Note: Not all (many?) U-Boot support FIT anyway, so you'd need to flash a new U-Boot to support FIT, so you may as well just flash a new U-Boot that implements the ${soc} and ${board} variables instead. IIRC, there may also be a ${boardname} or similar that's like ${board}, but represents the runtime-detected board for when one U-Boot build actually supports multiple different boards.
And enable bootz as well. Just about as many boards enable that as do FIT. And FIT has been around for years, bootz not so. So in theory folks with old/odd boards that didn't bother to get mainlined in U-Boot could still have FIT support added, easily.
I guess that's true, but enabling bootz in a U-Boot ends up bringing in a lot less conceptual complexity, and I would guess a bit less code too.
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
I'm not looking to add yet another crappy uboot image format to the kernel just for the hell of uboot.
And don't think that the dtbs in the kernel are going to stay there. The longer term plan is for them to end up in an external git tree, entirely separate from the kernel source. So anything that ties them with the kernel build process will ultimately have to be removed again.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/21/2013 05:37 AM, Russell King - ARM Linux wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
I'm not looking to add yet another crappy uboot image format to the kernel just for the hell of uboot.
Note that FIT images are relatively old (docs date back to March 2008). This is more of another effort to try and update what the kernel uses.
And don't think that the dtbs in the kernel are going to stay there. The longer term plan is for them to end up in an external git tree, entirely separate from the kernel source. So anything that ties them with the kernel build process will ultimately have to be removed again.
Well, once the device trees move to some external location, this script could also easily move out with it. And I'd hope there'd be interest out in the rest of the world for a single file boots on multiple platforms image format. If the ability to do this was actually well known, it might even stop people from re-inventing the wheel for fun (re-inventing due to design problems with the previous wheel is acceptable).
We're just about to make a lot of folks lives harder (whatever the userbase is for omap2plus_defconfig). How much time do we want to spend offering up alternatives?
- -- Tom
On Thu, Feb 21, 2013 at 08:20:56AM -0500, Tom Rini wrote:
On 02/21/2013 05:37 AM, Russell King - ARM Linux wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
I'm not looking to add yet another crappy uboot image format to the kernel just for the hell of uboot.
Note that FIT images are relatively old (docs date back to March 2008). This is more of another effort to try and update what the kernel uses.
So it's five years old and people haven't been that interested in it so far. That speaks volumes...
And don't think that the dtbs in the kernel are going to stay there. The longer term plan is for them to end up in an external git tree, entirely separate from the kernel source. So anything that ties them with the kernel build process will ultimately have to be removed again.
Well, once the device trees move to some external location, this script could also easily move out with it. And I'd hope there'd be interest out in the rest of the world for a single file boots on multiple platforms image format.
We've had that for the last 18 years. It's called zImage. The real problem is that uboot has been particularly difficult over this - uboot and _only_ uboot wants to use its own special file formats. No other boot loader on ARM has ever wanted this stuff - everything else has been totally happy with zImage.
uboot even now supports zImage through the bootz command, so it now supports the "native" format used on ARM.
We're just about to make a lot of folks lives harder (whatever the userbase is for omap2plus_defconfig). How much time do we want to spend offering up alternatives?
We're about to make it harder how? By forcing them to use DT blobs? Or by forcing them to have to use the combined zImage + DT format because their boot loaders are soo broken that they can't deal with multiple images?
Yes, things have become a _little_ harder since OMAP has switched to multiplatform, but it really isn't that bad. My build and boot test system still works, and it uses uImage for all the OMAP platforms. You just have to give the right LOADADDR= argument to the kernel to make the uImage generation work again. Sure, the resulting uImage can't be loaded at a different address, but you if you need to change that, you can quite easily move the existing uImage out and re-run make ARCH=arm uImage LOADADDR=<something else> and hey presto, you end up with the uImage generated for a different address.
But: we wouldn't be in this situation if it weren't for these absurd uboot uImage formats in the first place. Or even be needing this FIT stuff if zImage was just used directly.
uboot dug _itself_ into this hole. It's uboot's problem.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/21/2013 08:46 AM, Russell King - ARM Linux wrote:
On Thu, Feb 21, 2013 at 08:20:56AM -0500, Tom Rini wrote:
On 02/21/2013 05:37 AM, Russell King - ARM Linux wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
I'm not looking to add yet another crappy uboot image format to the kernel just for the hell of uboot.
Note that FIT images are relatively old (docs date back to March 2008). This is more of another effort to try and update what the kernel uses.
So it's five years old and people haven't been that interested in it so far. That speaks volumes...
No, it was just rejected a while back over in PowerPC-land by gcl (whom I talked with, but won't put words in his mouth here) and no one thought about bringing it forward again until now.
And don't think that the dtbs in the kernel are going to stay there. The longer term plan is for them to end up in an external git tree, entirely separate from the kernel source. So anything that ties them with the kernel build process will ultimately have to be removed again.
Well, once the device trees move to some external location, this script could also easily move out with it. And I'd hope there'd be interest out in the rest of the world for a single file boots on multiple platforms image format.
We've had that for the last 18 years. It's called zImage. The real problem is that uboot has been particularly difficult over this - uboot and _only_ uboot wants to use its own special file formats. No other boot loader on ARM has ever wanted this stuff - everything else has been totally happy with zImage.
uboot even now supports zImage through the bootz command, so it now supports the "native" format used on ARM.
We're just about to make a lot of folks lives harder (whatever the userbase is for omap2plus_defconfig). How much time do we want to spend offering up alternatives?
We're about to make it harder how? By forcing them to use DT blobs? Or by forcing them to have to use the combined zImage + DT format because their boot loaders are soo broken that they can't deal with multiple images?
None of that. We're making it harder since most folks don't have a board with 'bootz' built-in and the 'uImage' target doesn't build now (unless, yes, you pass LOADADDR to make) so they either format it by hand (/ adjust their local scripting) or do what you're doing.
Yes, things have become a _little_ harder since OMAP has switched to multiplatform, but it really isn't that bad. My build and boot test system still works, and it uses uImage for all the OMAP platforms. You just have to give the right LOADADDR= argument to the kernel to make the uImage generation work again. Sure, the resulting uImage can't be loaded at a different address, but you if you need to change that, you can quite easily move the existing uImage out and re-run make ARCH=arm uImage LOADADDR=<something else> and hey presto, you end up with the uImage generated for a different address.
But: we wouldn't be in this situation if it weren't for these absurd uboot uImage formats in the first place. Or even be needing this FIT stuff if zImage was just used directly.
FIT isn't required. FIT is just trying to offer a nice usability thing to folks. A point of device trees is a single image works in a lot of places. FIT gives you a single file works in a lot of places.
uboot dug _itself_ into this hole. It's uboot's problem.
A whole lot of people dug this particular hole. Joel is trying to offer up a solution that maybe makes things easier for everyone. Or it gets rejected here too and distros will come up with their N different ways to try and provide easier experiences to the end user.
- -- Tom
On Thu, 21 Feb 2013, Tom Rini wrote:
FIT isn't required. FIT is just trying to offer a nice usability thing to folks.
Usability is often counter-balanced by maintenance costs.
A point of device trees is a single image works in a lot of places. FIT gives you a single file works in a lot of places.
The whole _point_ for bringing DT into the ARM land was to decouple the the various platform specific stuff from the kernel, and even allow existing distros to work on yet-to-be-created devices unchanged, just like X86 PC class machines. Now you're suggesting that the kernel and the DT information should be glued together again. That looks like a move backward to me. Yet, having that platform stuff in plain C code as we had before, given it is maintained properly, is still the most efficient way to achieve that usability wise.
So let's stop kidding ourselves and be coherent please: either we move device specifics away from the kernel, or we keep them together. In other words, the DT should ideally come preinstalled with the bootloader on a given board/device for distros to not even have to care about it, or we put that data back inside the kernel and dispense ourselves from all the added DT overhead entirely. But an hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.
uboot dug _itself_ into this hole. It's uboot's problem.
A whole lot of people dug this particular hole. Joel is trying to offer up a solution that maybe makes things easier for everyone. Or it gets rejected here too and distros will come up with their N different ways to try and provide easier experiences to the end user.
Nothing being perfect, it is probably unreasonable to think that every board will start shipping with complete and correct DT description, etc. But so is the state of FIT support right now. That solution to make things easier for everyone should actually make that DT vs kernel separation more effective and provide better mechanisms for gluing the various DTBs to their respective boards, and not to glue them to the kernel to populate a distro filesystem with them.
Nicolas
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 02/21/2013 12:25 PM, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Tom Rini wrote:
[snip]
uboot dug _itself_ into this hole. It's uboot's problem.
A whole lot of people dug this particular hole. Joel is trying to offer up a solution that maybe makes things easier for everyone. Or it gets rejected here too and distros will come up with their N different ways to try and provide easier experiences to the end user.
Nothing being perfect, it is probably unreasonable to think that every board will start shipping with complete and correct DT description, etc. But so is the state of FIT support right now. That solution to make things easier for everyone should actually make that DT vs kernel separation more effective and provide better mechanisms for gluing the various DTBs to their respective boards, and not to glue them to the kernel to populate a distro filesystem with them.
I very much agree here. And in the end, what I really really want to avoid is every distribution (or similar grouping of stuff) coming up with N different ways to solve the problem of "how do I get the user the right device tree to go with $whatever board they happen to be running". If the clever solution everyone comes up with is some other container that's not FIT, that's fine, patches welcome and happily reviewed for whatever the solution is. I just don't want people thinking this is a problem that hasn't been thought of before.
- -- Tom
On Thu, 21 Feb 2013, Tom Rini wrote:
On 02/21/2013 12:25 PM, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Tom Rini wrote:
[snip]
uboot dug _itself_ into this hole. It's uboot's problem.
A whole lot of people dug this particular hole. Joel is trying to offer up a solution that maybe makes things easier for everyone. Or it gets rejected here too and distros will come up with their N different ways to try and provide easier experiences to the end user.
Nothing being perfect, it is probably unreasonable to think that every board will start shipping with complete and correct DT description, etc. But so is the state of FIT support right now. That solution to make things easier for everyone should actually make that DT vs kernel separation more effective and provide better mechanisms for gluing the various DTBs to their respective boards, and not to glue them to the kernel to populate a distro filesystem with them.
I very much agree here. And in the end, what I really really want to avoid is every distribution (or similar grouping of stuff) coming up with N different ways to solve the problem of "how do I get the user the right device tree to go with $whatever board they happen to be running".
DT installation must be outside of the distribution's responsibilities. It should be the OEM's responsibility, just like BIOS updates for PCs which don't come from Fedora/Debian/Ubuntu. Obviously, having the dts files in the kernel tree does confuse people in that regard, but that must not deter people from doing the right thing.
If the clever solution everyone comes up with is some other container that's not FIT, that's fine, patches welcome and happily reviewed for whatever the solution is. I just don't want people thinking this is a problem that hasn't been thought of before.
Ideally, there should be no such containers. You should simply pick any kernel, or install your distro of choice, and run that on any "DT ready" hardware. A distro could list the minimum version of a DTB some particular boards were tested with, just like they sometimes do for some PC BIOSes. That said, maybe some provision for DTB versioning would be a good idea if not done already.
Nicolas
On 02/21/2013 12:21 PM, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Tom Rini wrote:
On 02/21/2013 12:25 PM, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Tom Rini wrote:
[snip]
uboot dug _itself_ into this hole. It's uboot's problem.
A whole lot of people dug this particular hole. Joel is trying to offer up a solution that maybe makes things easier for everyone. Or it gets rejected here too and distros will come up with their N different ways to try and provide easier experiences to the end user.
Nothing being perfect, it is probably unreasonable to think that every board will start shipping with complete and correct DT description, etc. But so is the state of FIT support right now. That solution to make things easier for everyone should actually make that DT vs kernel separation more effective and provide better mechanisms for gluing the various DTBs to their respective boards, and not to glue them to the kernel to populate a distro filesystem with them.
I very much agree here. And in the end, what I really really want to avoid is every distribution (or similar grouping of stuff) coming up with N different ways to solve the problem of "how do I get the user the right device tree to go with $whatever board they happen to be running".
DT installation must be outside of the distribution's responsibilities. It should be the OEM's responsibility, just like BIOS updates for PCs which don't come from Fedora/Debian/Ubuntu. Obviously, having the dts files in the kernel tree does confuse people in that regard, but that must not deter people from doing the right thing.
The guidance that has been given in the past is that the kernel zImage and DTB /must/ be stored "in the same location", whether that means the /boot filesystem, flash partitions, or whatever, so that if required, the kernel and DTB can be updated at the same time, and using the same process, so it's guaranteed to be easy enough to update the DTB if you already know how to update the kernel.
Has that guidance changed?
Also, how can the OEM provide a DTB? The distro is responsible for installing all the filesystem content. There's no defined way of passing a DTB from some pre-bootloader firmware into the bootloader and through to the kernel; the only way to get a DTB to the kernel right now is for the bootloader to load it itself (either as part of a single file, or as a separate file) and pass it to the kernel. So, there's really no way for an OEM to provide a DTB in a BIOS-like fashion.
Why shouldn't the OEM just provide their *.dts files, and people can either compile them and put them into /boot, or distros can package them and the package will install them into /boot. That's extremely simple and while each distro will have to create their own packaging script, that's something they already know how to do, and a package that just dumps a file onto a disk is extremely simple, so people wouldn't have to go inventing distro-specific solutions.
If U-Boot always searched a disk for e.g. /boot/boot.scr or similar and just executed that, and there was a standard boot.scr that worked on all boards by use of e.g. bootz, ${soc}, ${board}, then that could be distro-agnostic too. And life would be simple, without the need for any extra build tools at all.
If the clever solution everyone comes up with is some other container that's not FIT, that's fine, patches welcome and happily reviewed for whatever the solution is. I just don't want people thinking this is a problem that hasn't been thought of before.
Ideally, there should be no such containers. You should simply pick any kernel, or install your distro of choice, and run that on any "DT ready" hardware. A distro could list the minimum version of a DTB some particular boards were tested with, just like they sometimes do for some PC BIOSes. That said, maybe some provision for DTB versioning would be a good idea if not done already.
Dear Stephen,
In message 5126778A.4040705@wwwdotorg.org you wrote:
If U-Boot always searched a disk for e.g. /boot/boot.scr or similar and just executed that, and there was a standard boot.scr that worked on all boards by use of e.g. bootz, ${soc}, ${board}, then that could be distro-agnostic too. And life would be simple, without the need for any extra build tools at all.
If the world was so simple, we could eventually do that. But it ain't so. Just consider the typical "diskless" system that boots over the network, using DHCP + TFTP, where the server will provide a single file only. Or systems that require sub-second boot times, where you don't want to spend time in running boot scripts. etc.
Best regards,
Wolfgang Denk
On 02/21/2013 12:57 PM, Wolfgang Denk wrote:
Dear Stephen,
In message 5126778A.4040705@wwwdotorg.org you wrote:
If U-Boot always searched a disk for e.g. /boot/boot.scr or similar and just executed that, and there was a standard boot.scr that worked on all boards by use of e.g. bootz, ${soc}, ${board}, then that could be distro-agnostic too. And life would be simple, without the need for any extra build tools at all.
If the world was so simple, we could eventually do that. But it ain't so. Just consider the typical "diskless" system that boots over the network, using DHCP + TFTP, where the server will provide a single file only.
I use TFTP routinely to boot my boards, and load separate zImage and DTB files from the server without issue, using the exact same filenames as when I load them from a /boot directory on eMMC or SD.
Or systems that require sub-second boot times, where you don't want to spend time in running boot scripts. etc.
Admittedly this is conjecture, but surely the actual IO time to load a zImage vastly vastly dwarfs the time take to parse an "ext2load" or "fsload" command to do so, so running scripts to do this isn't relevant. And besides, there's always some script, whether it's a boot.scr file or built into the U-Boot environment.
Dear Stephen,
In message 51267E0A.3060204@wwwdotorg.org you wrote:
so. Just consider the typical "diskless" system that boots over the network, using DHCP + TFTP, where the server will provide a single file only.
I use TFTP routinely to boot my boards, and load separate zImage and DTB files from the server without issue, using the exact same filenames as when I load them from a /boot directory on eMMC or SD.
You may do this in a development environment; I do the same routinely here. But there are users out there who commission large installations like that; they simply do not want to have multiple files for each system (and not even for each configuration).
And besides, there's always some script, whether it's a boot.scr file or built into the U-Boot environment.
Not really - especially not when booting using Falcon mode.
Best regards,
Wolfgang Denk
On Thu, 21 Feb 2013, Stephen Warren wrote:
On 02/21/2013 12:21 PM, Nicolas Pitre wrote:
DT installation must be outside of the distribution's responsibilities. It should be the OEM's responsibility, just like BIOS updates for PCs which don't come from Fedora/Debian/Ubuntu. Obviously, having the dts files in the kernel tree does confuse people in that regard, but that must not deter people from doing the right thing.
The guidance that has been given in the past is that the kernel zImage and DTB /must/ be stored "in the same location", whether that means the /boot filesystem, flash partitions, or whatever, so that if required, the kernel and DTB can be updated at the same time, and using the same process, so it's guaranteed to be easy enough to update the DTB if you already know how to update the kernel.
Where were such guidance given?
I don't think this is wise anyway. The DT content for a given piece of hardware should stabilize quickly, while the kernel can be updated for years. So there is no need for making the DTB update into the same process as the kernel.
And as I stated earlier, the distros will care about kernel updates, but they shouldn't have to care about DTB updates.
The only reason why we do have to update DTBs frequently is because the whole thing is still being developed and established. But frequent DTB updates are not the end goal, and therefore we shouldn't design installation and update mechanisms around the current state of affairs which is not meant to persist.
Has that guidance changed?
Consider this as my plea to have it changed.
Also, how can the OEM provide a DTB? The distro is responsible for installing all the filesystem content.
And that's probably not a good idea either.
There's no defined way of passing a DTB from some pre-bootloader firmware into the bootloader and through to the kernel;
That doesn't have to stay that way, does it? Maybe those efforts put on things like FIT or the like could be applied to this problem instead?
Yet that depends how you define a pre-bootloader and a bootloader. Some people call UEFI the bootloader and grub or u-boot running on top as the "second stage" bootloader. I've heard people talking about making u-boot into a _user_ of the DTB in addition to the kernel, which would make some sense. In that case, the second stage bootloader could be somewhat generic as well. But if that layer is generic, then there must be some kind of standard adaptation layer or API provided by the first stage bootloader to the second stage bootloader. Passing on some hardware configuration payload would be part of that API.
Sidenote: there is noise about using ACPI instead of DT on ARM servers for example, so that's another reason why distributions should not engage into this responsibility too much.
And obviously the DTB has to be tied to the machine specific boot layer. And the method to retrieve it would, obviously, be machine specific at that point. This doesn't have to come frfom the filesystem used by the final operating system.
the only way to get a DTB to the kernel right now is for the bootloader to load it itself (either as part of a single file, or as a separate file) and pass it to the kernel. So, there's really no way for an OEM to provide a DTB in a BIOS-like fashion.
That will have to change.
Why shouldn't the OEM just provide their *.dts files, and people can either compile them and put them into /boot, or distros can package them and the package will install them into /boot. That's extremely simple and while each distro will have to create their own packaging script, that's something they already know how to do, and a package that just dumps a file onto a disk is extremely simple, so people wouldn't have to go inventing distro-specific solutions.
Because that shouldn't be the distro's business. Because the distros shouldn't have to be aware of all possible DTBs around and gather them on their install media and figure out during installation which one to use, or configure the bootloader so it knows which one to pass back to the kernel, when the OEMs may as well just ship the right one with their hardware without the distro to care at all. Because a distro should be installable on some future hardware that didn't exist when that distro was put together. Because, ideally, you should be able to pop out the disk (or SD card) from device X and put it into device Y and just boot without changing anything.
The device tree data is describing hardware, it should be tied to the hardware, not to the distribution. Hardware doesn't change, kernels and distros do.
If U-Boot always searched a disk for e.g. /boot/boot.scr or similar and just executed that, and there was a standard boot.scr that worked on all boards by use of e.g. bootz, ${soc}, ${board}, then that could be distro-agnostic too. And life would be simple, without the need for any extra build tools at all.
No, life isn't that simple. First, not everyone wants to use U-Boot. And even if U-Boot was universally used/loved, there is no way you could have a boot.scr that would work for every board. In some cases there might not even a usable device or filesystem for U-Boot to find the /boot directory. Alternate bootloaders might not have a btrfs driver for digging into the installed distro filesystem. Etc.
Whereas, if the bootloader simply provides a DTB using any method deemed appropriate by the OEM, then the kernel and therefore the distros are freed from this extra and avoidable maintenance cost. The _only_ contract between the bootloader and the kernel/distro should be this passing of a DTB (and initrd) in a standardized way.
Extending this contract to include a given /boot directory on a specific filesystem type, a preformatted boot.scr with predetermined variables, and other similar high-level requirements is going to make a system with zero flexibility and susceptible to boot failure as soon as one little detail gets messed up. Such system would simplify some people's life of course, but is going to signify hell for others.
Then, some distros will support a certain list of hardware devices while other distros won't because of different release schedule, etc. That issue is no more if the DTB comes with the hardware.
Nicolas
On 02/21/2013 02:18 PM, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Stephen Warren wrote:
On 02/21/2013 12:21 PM, Nicolas Pitre wrote:
DT installation must be outside of the distribution's responsibilities. It should be the OEM's responsibility, just like BIOS updates for PCs which don't come from Fedora/Debian/Ubuntu. Obviously, having the dts files in the kernel tree does confuse people in that regard, but that must not deter people from doing the right thing.
The guidance that has been given in the past is that the kernel zImage and DTB /must/ be stored "in the same location", whether that means the /boot filesystem, flash partitions, or whatever, so that if required, the kernel and DTB can be updated at the same time, and using the same process, so it's guaranteed to be easy enough to update the DTB if you already know how to update the kernel.
Where were such guidance given?
IIRC, it's been discussed a number of times on the Linux ARM kernel mailing list and at the various ARM workshops at kernel summit and/or Linaro Connect. It has been a while, so maybe the advice was simply supposed to age out, but I certainly haven't seen any explicit indication that the advice was temporary or rescinded.
The only reason why we do have to update DTBs frequently is because the whole thing is still being developed and established. But frequent DTB updates are not the end goal, and therefore we shouldn't design installation and update mechanisms around the current state of affairs which is not meant to persist.
It will always be under development.
People will make mistakes and want to fix them.
Someone will want to use a previously unsupported feature of some HW and then write the DT bindings for that feature for the first time. E.g. Tegra's one-wire controller isn't that commonly used, so we have no binding for it yet despite it being maybe a couple years after starting DT work for Tegra. The AC'97 was only recently supported.
Now I agree that this probably will settle down eventually. However, HW will have been widely distributed well before the DT bindings are feature-complete and bug-free. Any solution needs to take that into account, rather than only attempting to solve the situation after the hardware is obsolete and hence the bindings are stable.
There's no defined way of passing a DTB from some pre-bootloader firmware into the bootloader and through to the kernel;
That doesn't have to stay that way, does it? Maybe those efforts put on things like FIT or the like could be applied to this problem instead?
Perhaps. But where can the DTB be stored?
a) In the system's boot flash alongside U-Boot, or at least some first-/early-stage firmware that can determine where in RAM it was loaded to, and pass it to some later-stage software in a standard fashion.
b) In a filesystem, alongside the kernel.
c) In some other flash device, or partition.
There often isn't any such flash device/partition for (c), so that option is out as a /general/ rule.
(a) would require a firmware upgrade process in order to upgrade the DTB. Using this option is very explicitly against the guidance I was talking about a few paragraphs up.
So that just leaves (b). And in this case, you may as well use U-Boot's (or any other bootloader) ability to load foo.dtb just like it loads zImage.
Yet that depends how you define a pre-bootloader and a bootloader. Some people call UEFI the bootloader and grub or u-boot running on top as the "second stage" bootloader. I've heard people talking about making u-boot into a _user_ of the DTB in addition to the kernel, which would make some sense.
For better or worse, that is true for upstream U-Boot on Tegra. The DTB is built into U-Boot in this case, and is a different DTB than what's passed to the kernel, because U-Boot has historically not used identical DT bindings to the real reviewed bindings as defined by the kernel's Documentation/devicetree/bindings. It's getting better, but there is still some way to go. The same is probably true for Exynos5, but I haven't investigated the situation there in detail.
...
Sidenote: there is noise about using ACPI instead of DT on ARM servers for example, so that's another reason why distributions should not engage into this responsibility too much.
Hmmm. That sounds a lot like making most of my last 2 years' work redundant:-( It'd be a big pity to have that all wasted.
...
If U-Boot always searched a disk for e.g. /boot/boot.scr or similar and just executed that, and there was a standard boot.scr that worked on all boards by use of e.g. bootz, ${soc}, ${board}, then that could be distro-agnostic too. And life would be simple, without the need for any extra build tools at all.
No, life isn't that simple. First, not everyone wants to use U-Boot.
Sure.
And even if U-Boot was universally used/loved, there is no way you could have a boot.scr that would work for every board.
I disagree there. The following should work for anything, with U-boot's boot script set to automatically search for this boot.scr on available boot media, load, and execute it:
# similar comment for the actual partition number setenv rootpart 1 part uuid ${devtype} ${devnum}:${rootpart} uuid
# Needs to be auto-generated by a distro boot.scr creation script, # just like grub.cfg is on x86: # bootargs_extra would be HW-specific parameters that U-Boot's default # environment provides, if any are required (hopefully not) setenv bootargs ... root=PARTUUID=${uuid} ${bootargs_extra}
ext2load ${devtype} ${devnum}:${rootpart} ${kernel_addr_r} /boot/zImage ext2load ${devtype} ${devnum}:${rootpart} ${fdt_addr_r} /boot/${soc}-${board}.dtb bootz ${kernel_addr_r} - ${fdt_addr_r}
And yes, "part uuid" and "bootz" are new, but recall that FIT support isn't available everywhere either, so you can't use that argument against it.
And yes, some people creating dedicated embedded systems might not want a boot script on disk, but for a general-purpose setup running arbitrary Linux distros, this seems like the most flexible thing to do, just like Grub's grub.cfg.
In some cases there might not even a usable device or filesystem for U-Boot to find the /boot directory. Alternate bootloaders might not have a btrfs driver for digging into the installed distro filesystem. Etc.
/boot often has constraints dictated by the bootloader, e.g. must be ext2 not some other fancy filesystem on x86.
If there's no filesystem for U-Boot/... to load a zImage/... from, then you're already outside the realm of a general purpose machine that supports arbitrary Linux distros, and into the realm of custom embedded systems. Those will always require some custom non-generic work to get them too boot the way you want. But the system creator/integrator gets to do that, not the Linux distro.
Whereas, if the bootloader simply provides a DTB using any method deemed appropriate by the OEM, then the kernel and therefore the distros are freed from this extra and avoidable maintenance cost. The _only_ contract between the bootloader and the kernel/distro should be this passing of a DTB (and initrd) in a standardized way.
That contract isn't influenced by how/where the bootloader obtains the DTB, so that statement isn't relevant to the use of FIT.
On Thu, Feb 21, 2013 at 05:10:36PM -0700, Stephen Warren wrote:
On 02/21/2013 02:18 PM, Nicolas Pitre wrote:
Where were such guidance given?
IIRC, it's been discussed a number of times on the Linux ARM kernel mailing list and at the various ARM workshops at kernel summit and/or Linaro Connect. It has been a while, so maybe the advice was simply supposed to age out, but I certainly haven't seen any explicit indication that the advice was temporary or rescinded.
And this is why Linaro Connects are a bad idea. They give the illusion of decisions being taken and finalize, taking the decision process out of the open source community.
It can be viewed as corporate takeover of open source. (I realise that comment is going to get peoples backs up - but unfortunately you can't call an apple a pear.)
What is the really sad thing is that at no point do the Linaro Connects really come back to the community (such as this mailing list) and say "this is what was discussed, and this is what we think, can we get agreement". Instead what happens is that stuff gets discussed behind closed doors, work starts, patches come out, and then it gets discussed openly.
And... as for the meeting notes system... I looked at the ARM booting stuff on Linaro's website - it contains no real useful information, it looks like it's fallen by the way side. So what's the point of having a page or two on it?
Someone will want to use a previously unsupported feature of some HW and then write the DT bindings for that feature for the first time. E.g. Tegra's one-wire controller isn't that commonly used, so we have no binding for it yet despite it being maybe a couple years after starting DT work for Tegra. The AC'97 was only recently supported.
Now I agree that this probably will settle down eventually. However, HW will have been widely distributed well before the DT bindings are feature-complete and bug-free. Any solution needs to take that into account, rather than only attempting to solve the situation after the hardware is obsolete and hence the bindings are stable.
Tell me then - how is it possible for my laptop to boot correctly with its ACPI data which describes its hardware, and that ACPI data hardly ever has to be updated. Many PC systems have been doing this for years, with various degrees of success - but generally once you have a working system it stays working and never needs to have its ACPI data updated.
On 02/21/2013 05:39 PM, Russell King - ARM Linux wrote:
On Thu, Feb 21, 2013 at 05:10:36PM -0700, Stephen Warren wrote:
On 02/21/2013 02:18 PM, Nicolas Pitre wrote:
...
Someone will want to use a previously unsupported feature of some HW and then write the DT bindings for that feature for the first time. E.g. Tegra's one-wire controller isn't that commonly used, so we have no binding for it yet despite it being maybe a couple years after starting DT work for Tegra. The AC'97 was only recently supported.
Now I agree that this probably will settle down eventually. However, HW will have been widely distributed well before the DT bindings are feature-complete and bug-free. Any solution needs to take that into account, rather than only attempting to solve the situation after the hardware is obsolete and hence the bindings are stable.
Tell me then - how is it possible for my laptop to boot correctly with its ACPI data which describes its hardware, and that ACPI data hardly ever has to be updated. Many PC systems have been doing this for years, with various degrees of success - but generally once you have a working system it stays working and never needs to have its ACPI data updated.
I'm not too familiar with ACPI or even low-level x86/PC architecture, so forgive me if I'm talking crap, but I suspect there are quite a few reasons for the differences here:
* PC systems in general are rather simpler and more standardized. Even where "differentiation" can exist, it's encapsulated behind simple self-contained interfaces.
* As an example, most random "expansion" devices on a PC are PCIe or USB based, even built-in stuff such as WiFi cards etc. On ARM, there are no standards, so anything goes. Hence, PC expansion cards are much more self-contained and interact with other devices less, whereas on ARM you have a whole spaghetti slew of chips, GPIOs, regulators, interrupts, ...
* I think ARM designs are exposed to a wider audience earlier on in their maturity cycle. On x86, there are a number of standards for ACPI etc., and the first people to interact with boards and implement these standards are BIOS and board vendors, and then the boards get out to the general public, and they at least basically all confirm to those standards and work. However, on ARM, we're developing those standards well after the fact; after the HW has shipped and been in use, retro-fitting the standards etc. So, we're being exposed to a much earlier point in the whole cycle than we can even see on x86. I bet x86 CPU, chipset, BIOS, and board vendors see the kind of change in ACPI tables that we're seeing in DT right now.
* I imagine there are fewer different sets of people (CPU vendors, BIOS vendors) having to work on defining ACPI standards than there are ARM SoC vendors and Linux developer who're working on DT standards.
* x86 is very mature, so there's a lot more incremental development and compatibility. ARM and DT-on-ARM are very new, so it's still the wild west.
On Thu, Feb 21, 2013 at 12:25:21PM -0500, Nicolas Pitre wrote:
So let's stop kidding ourselves and be coherent please: either we move device specifics away from the kernel, or we keep them together. In other words, the DT should ideally come preinstalled with the bootloader on a given board/device for distros to not even have to care about it, or we put that data back inside the kernel and dispense ourselves from all the added DT overhead entirely. But an hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.
Just to thread jack a bit here..
We've been using DT on production embedded stuff sice about 2.6.20ish on PPC and now ARM. We treat the dtb as a kernel version specific file, much like an initrd and ensure that the kernel only ever boots with its proper dtb. This is based on experience that the dtbs change depending on the state of the drivers in the kernel, what gets mainlined and when, etc.
Embedding this stuff into the bootloader is *not* desirable for my embedded scenarios. We don't use FIT (or uboot) but we do the same thing: a single image is constructed with the proper dtb, kernel and initrd, and that is what the bootloader boots.
Why? This is an embedded appliance product. We need to be able to deliver firmware upgrades that *work*. We can't brick the board because the bootloader and kernel get out of sync. The boot loader has to be *simple*, it has to boot every past, present and future kernel or we start taking risks that a firmware flash will end up bricking it.
People making dev boards and distros for them certainly have different requirements, but we've decided that the single image approach is the best for appliance style products.
Jason
On Thu, Feb 21, 2013 at 11:27:24AM -0700, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 12:25:21PM -0500, Nicolas Pitre wrote:
So let's stop kidding ourselves and be coherent please: either we move device specifics away from the kernel, or we keep them together. In other words, the DT should ideally come preinstalled with the bootloader on a given board/device for distros to not even have to care about it, or we put that data back inside the kernel and dispense ourselves from all the added DT overhead entirely. But an hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.
Just to thread jack a bit here..
We've been using DT on production embedded stuff sice about 2.6.20ish on PPC and now ARM. We treat the dtb as a kernel version specific file, much like an initrd and ensure that the kernel only ever boots with its proper dtb. This is based on experience that the dtbs change depending on the state of the drivers in the kernel, what gets mainlined and when, etc.
Just be aware that on ARM at some point, the DT files will be moved out of the kernel source and held in an independent git tree; there won't be any intention of "kernel version dependence" - if there is then kernel maintanence has failed - and the use of DT has failed too.
The stated point of DT is that it describes the hardware. If there's a dependence on the kernel version, then it's doing more than describing the hardware - it would be describing the kernel implementation as well.
Embedding this stuff into the bootloader is *not* desirable for my embedded scenarios. We don't use FIT (or uboot) but we do the same thing: a single image is constructed with the proper dtb, kernel and initrd, and that is what the bootloader boots.
Note that the combined zImage plus DTB today on ARM is just a stop-gap for boot loaders which provide only an ATAG based booting solution. It's not there as a long term kernel facility. It will probably be going away when the DT files are moved out of the kernel source tree.
Why? This is an embedded appliance product. We need to be able to deliver firmware upgrades that *work*. We can't brick the board because the bootloader and kernel get out of sync. The boot loader has to be *simple*, it has to boot every past, present and future kernel or we start taking risks that a firmware flash will end up bricking it.
If the DT file describes the hardware properly, then you shouldn't need to update it. Or if the DT file contains a bug (eg, because it doesn't describe how things are wired) then it's wrong whatever.
But whatever. There's absolutely no problem storing the DT file separately from the boot loader, and also separately from the kernel itself (this is something that has been stated will be the normal way that DT will be dealt with when DT on ARM was first mooted.)
If that's not the case, then we're really just playing one huge game with DT, causing lots of pointless churn for no benefit to people like you, and we might as well stop kidding ourselves that DT gives us any advantages what so ever. We might as well go back to putting this data right back into the kernel in C file form.
On Thu, Feb 21, 2013 at 07:08:20PM +0000, Russell King - ARM Linux wrote:
We've been using DT on production embedded stuff sice about 2.6.20ish on PPC and now ARM. We treat the dtb as a kernel version specific file, much like an initrd and ensure that the kernel only ever boots with its proper dtb. This is based on experience that the dtbs change depending on the state of the drivers in the kernel, what gets mainlined and when, etc.
Just be aware that on ARM at some point, the DT files will be moved out of the kernel source and held in an independent git tree; there won't be any intention of "kernel version dependence" - if there is then kernel maintanence has failed - and the use of DT has failed too.
Yes, I've read that in this thread. I have to say that is a scary thought, if the seperate DT's get all fragmented like uboot is then it is just going to be a huge PITA.
The stated point of DT is that it describes the hardware. If there's a dependence on the kernel version, then it's doing more than describing the hardware - it would be describing the kernel implementation as well.
Again yes, but.. My experience has been a bit different, *pragmatically* the DTs are configuring Linux to work with that hardware, and the Linux expectations can leak into that.
The DT changes seem to tail off as the HW and drivers mature, our PPC stuff doesn't change much at all these days, but the our ARM DT is churning like crazy..
I think this pattern is very likely to repeat every time a new SOC is introduced. Until everything is mainlined and agreed on the DTs for that SOC will churn.
Some of this seems to be caused by the complexity of the OF standard, the flexibility it allows and the fact it is being pushed into areas the standard doesn't cover. With more than one way to do everything stuff gets done wrong, or people change their minds, or whatever.
Note that the combined zImage plus DTB today on ARM is just a stop-gap for boot loaders which provide only an ATAG based booting solution. It's not there as a long term kernel facility. It will probably be going away when the DT files are moved out of the kernel source tree.
Thats fine for me, we don't use that mechanism.
Why? This is an embedded appliance product. We need to be able to deliver firmware upgrades that *work*. We can't brick the board because the bootloader and kernel get out of sync. The boot loader has to be *simple*, it has to boot every past, present and future kernel or we start taking risks that a firmware flash will end up bricking it.
If the DT file describes the hardware properly, then you shouldn't need to update it. Or if the DT file contains a bug (eg, because it doesn't describe how things are wired) then it's wrong whatever.
.. or Linux changes how it interprets the DT.
.. or Linux requires additional HW blocks to be described in the DT.
If that's not the case, then we're really just playing one huge game with DT, causing lots of pointless churn for no benefit to people like you, and we might as well stop kidding ourselves that DT gives us any advantages what so ever. We might as well go back to putting this data right back into the kernel in C file form.
Well, when we switched PPC to DT it was a huge improvement in these areas: - Describing and binding I2C devices - .. MDIO Phys - Passing ethernet MAC addresses into drivers - Routing and binding GPIOs to linux drivers (like I2C, MDIO, resets etc) - Passing HW specific options into buried drivers, eg options for I2C hwmon devices, LED configuration for phys, etc
All of which had OF mechanisms, but no good C mechanism (at the time at least)
I also noticed a decrease in churn, the PPC DT descriptions have been more stable that the prior C versions. The C versions often tended to need small revisions as the kernel changed.
Further, our boards are almost always very different than the eval/dev platforms provided by the vendor. The splitting up and modularizing of the kernel code that has come along with DT has made it much simpler to support our specific boards.
So, I strongly disagree that DT is bringing no benefit to my kind of application.
Jason
On Thu, 21 Feb 2013, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 12:25:21PM -0500, Nicolas Pitre wrote:
So let's stop kidding ourselves and be coherent please: either we move device specifics away from the kernel, or we keep them together. In other words, the DT should ideally come preinstalled with the bootloader on a given board/device for distros to not even have to care about it, or we put that data back inside the kernel and dispense ourselves from all the added DT overhead entirely. But an hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.
Just to thread jack a bit here..
We've been using DT on production embedded stuff sice about 2.6.20ish on PPC and now ARM. We treat the dtb as a kernel version specific file, much like an initrd and ensure that the kernel only ever boots with its proper dtb. This is based on experience that the dtbs change depending on the state of the drivers in the kernel, what gets mainlined and when, etc.
For embedded appliance product you may do as you wish. Nobody will interfere in the way you develop and support your own products (as long as you honor the applicable licenses of course).
But here we're discussing ARM Linux distributions having to deal with different hardware devices. It simply doesn't make sense to bundle every hardware specific data with the kernel in that context.
Embedding this stuff into the bootloader is *not* desirable for my embedded scenarios. We don't use FIT (or uboot) but we do the same thing: a single image is constructed with the proper dtb, kernel and initrd, and that is what the bootloader boots.
No one is advocating to embed the DT stuff in the bootloader. The DTB may be buggy and/or incomplete and being able to update it safely i.e. independently from the bootloader is necessary.
Why? This is an embedded appliance product. We need to be able to deliver firmware upgrades that *work*. We can't brick the board because the bootloader and kernel get out of sync. The boot loader has to be *simple*, it has to boot every past, present and future kernel or we start taking risks that a firmware flash will end up bricking it.
People making dev boards and distros for them certainly have different requirements, but we've decided that the single image approach is the best for appliance style products.
Absolutely. And in your case, DT is not bringing any benefit over the previous situation where everything was compiled into the kernel. I suspect you're not using the multi-platform support either which is one of the major endeavor on ARM right now.
Nicolas
On Thu, Feb 21, 2013 at 02:57:46PM -0500, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 12:25:21PM -0500, Nicolas Pitre wrote:
So let's stop kidding ourselves and be coherent please: either we move device specifics away from the kernel, or we keep them together. In other words, the DT should ideally come preinstalled with the bootloader on a given board/device for distros to not even have to care about it, or we put that data back inside the kernel and dispense ourselves from all the added DT overhead entirely. But an hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.
Just to thread jack a bit here..
We've been using DT on production embedded stuff sice about 2.6.20ish on PPC and now ARM. We treat the dtb as a kernel version specific file, much like an initrd and ensure that the kernel only ever boots with its proper dtb. This is based on experience that the dtbs change depending on the state of the drivers in the kernel, what gets mainlined and when, etc.
For embedded appliance product you may do as you wish. Nobody will interfere in the way you develop and support your own products (as long as you honor the applicable licenses of course).
I was specifically responding to your statement that 'a hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.'
We have been making good use of such an arrangement, and it is defintely not 'the wrong message' for certain applications. In fact, as I said, it is probably the *right* message for embedded users.
Even if I was a distro user, the idea that my dt and kernel would be decoupled is very scary. Realize that today, my Kirkwood systems require a different DT for at least 3.7 and 3.8 kernels, and quite possibly different again for 3.9!!
This will eventually settle on kirkwood, but I bet the same pattern will repeat on the next new SOC.
I would have thought keeping the device tree data and kernel together is preferred for most cases as it is more inline with Documentation/stable_api_nonsense.txt. Making the DT a strong stable API boundary sounds really hard to me, and if the churn on ARM so far is indication, it may not be realistic..
But here we're discussing ARM Linux distributions having to deal with different hardware devices. It simply doesn't make sense to bundle every hardware specific data with the kernel in that context.
Distros already ship huge kernels with modules for every hardware out there. Shipping all the DTs as well doesn't seem like a problem.
I am thinking something like /lib/device-tree/`uname -r`/...
Where (taking a PC analog) the bootloader is told to grab: /boot/vmlinuz-3.9 /boot/initrd.img-3.9 /lib/device-tree/3.9/ti/omap/foo-bar-board
The kernel build can be nice and uniform, while the distro can provide scripts/tools to bundle the kernel zimage, kernel modules, initramfs stuff and dtb into something bootable - be it FIT, uimage, bootz script, grub script or whatever.
Embedding this stuff into the bootloader is *not* desirable for my embedded scenarios. We don't use FIT (or uboot) but we do the same thing: a single image is constructed with the proper dtb, kernel and initrd, and that is what the bootloader boots.
No one is advocating to embed the DT stuff in the bootloader. The DTB may be buggy and/or incomplete and being able to update it safely i.e. independently from the bootloader is necessary.
Sorry, what did you mean by: 'the DT should ideally come preinstalled with the bootloader on a given board/device'
?
My point was simply that any scenario where the bootloader grabs a DTB that is not strongly associated with the kernel it is going to boot is not desirable.
People making dev boards and distros for them certainly have different requirements, but we've decided that the single image approach is the best for appliance style products.
Absolutely. And in your case, DT is not bringing any benefit over the previous situation where everything was compiled into the kernel. I
Strongly disagree, see my prior email to Russell. DT is a very big improvement over the old way of C coding the same data.
Jason
On Thu, 21 Feb 2013, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 02:57:46PM -0500, Nicolas Pitre wrote:
For embedded appliance product you may do as you wish. Nobody will interfere in the way you develop and support your own products (as long as you honor the applicable licenses of course).
I was specifically responding to your statement that 'a hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.'
We have been making good use of such an arrangement, and it is defintely not 'the wrong message' for certain applications. In fact, as I said, it is probably the *right* message for embedded users.
No it is not. FIT is about bundling a multi-platform kernel with a bunch of DTBs together in a single file. I don't think you need that for your embedded system. The "wrong message" here is to distribute multiple DTBs around, whether it is with FIT or on a distro install media.
Even if I was a distro user, the idea that my dt and kernel would be decoupled is very scary.
That was still the design goal for DT.
Realize that today, my Kirkwood systems require a different DT for at least 3.7 and 3.8 kernels, and quite possibly different again for 3.9!!
Understandable, given that Kirkwood is being ported to DT right now. But if DT had been used from the start when we introduced Kirkwood support to Linux back in 2008 you most likely wouldn't have to change the DTB on your board at all today.
This will eventually settle on kirkwood, but I bet the same pattern will repeat on the next new SOC.
Possible, although new SOCs do start with DT from the start which is much easier than trying to retrofit it to existing code without breaking things. And given that patterns emerge, there is no need to redesign new bindings for every new SOC.
I would have thought keeping the device tree data and kernel together is preferred for most cases as it is more inline with Documentation/stable_api_nonsense.txt. Making the DT a strong stable API boundary sounds really hard to me, and if the churn on ARM so far is indication, it may not be realistic..
The DT is meant to describe hardware. As far as I know, the hardware I own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
Distros already ship huge kernels with modules for every hardware out there. Shipping all the DTs as well doesn't seem like a problem.
But it is! Even shipping multiple kernels _is_ a problem for them. Hence this multi-platform kernel effort. Otherwise why would we bother?
According to your logic, distros could package and distribute BIOS updates for all the X86 systems out there. After all, if they did, they would guarantee even better support on the hardware they target and not have to carry those ACPI quirks in the kernel, no?
Ask them if they find this idea rejoicing. You might be surprised.
I am thinking something like /lib/device-tree/`uname -r`/...
Where (taking a PC analog) the bootloader is told to grab: /boot/vmlinuz-3.9 /boot/initrd.img-3.9 /lib/device-tree/3.9/ti/omap/foo-bar-board
And what is the advantage over not having to carry all those files at all on your filesystem?
The kernel build can be nice and uniform, while the distro can provide scripts/tools to bundle the kernel zimage, kernel modules, initramfs stuff and dtb into something bootable - be it FIT, uimage, bootz script, grub script or whatever.
Or they may simply not bother if the bootloader that comes with the hardware already does the right thing which is even better.
Embedding this stuff into the bootloader is *not* desirable for my embedded scenarios. We don't use FIT (or uboot) but we do the same thing: a single image is constructed with the proper dtb, kernel and initrd, and that is what the bootloader boots.
No one is advocating to embed the DT stuff in the bootloader. The DTB may be buggy and/or incomplete and being able to update it safely i.e. independently from the bootloader is necessary.
Sorry, what did you mean by: 'the DT should ideally come preinstalled with the bootloader on a given board/device'
?
When you acquire some hardware, it should come with a DTB and bootloader pre-installed, ready to boot any distribution (as long as its kernel supports the SoC of course). Your hardware vendor should offer DTB updates on its website. The DTB should not be compiled into the bootloader so DTB updates can be done independently from risky bootloader updates.
Nicolas
On Thu, Feb 21, 2013 at 05:05:54PM -0500, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 02:57:46PM -0500, Nicolas Pitre wrote:
For embedded appliance product you may do as you wish. Nobody will interfere in the way you develop and support your own products (as long as you honor the applicable licenses of course).
I was specifically responding to your statement that 'a hybrid mixed solution like FIT is IMHO the worst of both worlds and sending a wrong message.'
We have been making good use of such an arrangement, and it is defintely not 'the wrong message' for certain applications. In fact, as I said, it is probably the *right* message for embedded users.
No it is not. FIT is about bundling a multi-platform kernel with a bunch of DTBs together in a single file. I don't think you need that for your embedded system. The "wrong message" here is to distribute multiple DTBs around, whether it is with FIT or on a distro install media.
Actually we do this on PPC, the boot kernel image runs on three similar hardware platforms, the image has three DTBs built into it and the right one is selected at runtime. The kernel boot image does this (call it a second stage boot loader), not the primary boot loader.
I strongly disagree with the idea that keeping the DTB seperated from the kernel is appropriate for all users, or even most users. To me that only seems appropriate for certain kinds of hardware, eg general purpose computing devices that are designed to primarily run a Linux distro.
An embedded SOC eval board, a development platform, an embedded appliance - these are cases where the kernel and DTB should generally be more tightly coupled.
This is more or less how PPC has evolved, big commerical PPC systems like Apple's and IBM's stuff all provide a DTB to the kernel - and this is actually a bit different then the DT's people are writing for SOCs, it is firmware generated and includes a full description of all the probed hardware - including pluggable PCI cards and other stuff. The hardware is also left configured so there is less for Linux to do and less that needs to be described in DT.
While embedded focused PPC stuff seems to tend to keep the kernel and DT together.
This will eventually settle on kirkwood, but I bet the same pattern will repeat on the next new SOC.
Possible, although new SOCs do start with DT from the start which is much easier than trying to retrofit it to existing code without breaking things. And given that patterns emerge, there is no need to redesign new bindings for every new SOC.
Disagree. We are already seeing patching now for 2nd generation DT bindings to fix flaws in bindings that were introduced earlier. I hope the rate will slow down, but the need will probably never go away completely. :(
This is already standing on top of the work that was done to establish DT patterns for embedded PPC..
The DT is meant to describe hardware. As far as I know, the hardware I own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
.. and the patching I mention above are largely driven by either a change in understanding of how OF should describe the hardware, or a change in understanding of how the driver should treat the hardware.
The recent patching for the tegra PCI-E bridge is instructive in this regard, Theirry learned how to drive the chip in a way that creates a single PCI domain - this necessitates a change in how the DT models that hardware block.
There are lots of ways to model the same hardware in DT.
Distros already ship huge kernels with modules for every hardware out there. Shipping all the DTs as well doesn't seem like a problem.
But it is! Even shipping multiple kernels _is_ a problem for them. Hence this multi-platform kernel effort. Otherwise why would we bother?
Multiple *kernel packages* is a big problem, one *kernel package* is generally not.
It is already the case on x86 that a kernel package can't boot out of the box. The distro builds a box-specific initramfs on boot that minimally includes enough modules to access the root fs storage. Grabbing a box specific DT as well is a tiny additional step.
Bear in mind, that like for storage, when the kernel is installed the system is *already running*. This means it knows what storage modules are needed, and similarly it knows the content of the DTB it is using. It can do three things with this: - See if /lib/device-tree/.. contains a compatible DTB, if so use the version from /lib - Save the DTB to /boot/my-board-dtb and use it - Realize that it is OEM provided and comes from the firmware, do nothing
So things can very much be fully automated.
According to your logic, distros could package and distribute BIOS updates for all the X86 systems out there. After all, if they did, they would guarantee even better support on the hardware they target and not have to carry those ACPI quirks in the kernel, no?
The distros are going to include uboot packages and people are going to try and support a wide range of boards through uboot - so yah, on ARM distros are packaging full BIOS updates for ARM. This doesn't seem to be a problem.
If the DTs are moved out of the kernel, then the distros will build and package them too.
Heck, on x86 some distros do make use of the runtime ACPI patching stuff to fix particularly broken firmware.
Ask them if they find this idea rejoicing. You might be surprised.
This stuff is never rejoicing. But give the distro two choices,
- Include the /lib/device-tree/ .. stuff from the kernel build and things will work robustly *on systems that require it* - Don't, and a kernel update or firmware update might randomly result in boot failure
Which do you think they will they pick?
Relying on OEMs to provide working firmware has been a *nightmare* on x86. There is no reason to think ARM OEMs would do any better. Minimizing the amount of OEM specific junk that needs to be used is a good thing.
Heck, just try to get an OEM supported mainline kernel for some of the eval boards they ship. Good luck....
Sorry, what did you mean by: 'the DT should ideally come preinstalled with the bootloader on a given board/device'
When you acquire some hardware, it should come with a DTB and bootloader pre-installed, ready to boot any distribution (as long as its kernel supports the SoC of course). Your hardware vendor should offer DTB updates on its website. The DTB should not be compiled into the bootloader so DTB updates can be done independently from risky bootloader updates.
Like I said, that is fine for some hardware. However, if the kernel is expected to boot from bare metal or very close to it (ie embedded focused) then I think it is the wrong approach.
Considering that embedded is one the of the major users of the ARM kernel right now, I don't think it is clever to push that group into a bad solution.
Again, I think the stable_api_nonsense.txt ideas apply here. Minimizing the list of stuff that falls in the 'stable, because of firmware' category seems like a smart thing to do.
Jason
On 02/21/2013 04:11 PM, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 05:05:54PM -0500, Nicolas Pitre wrote:
...
The DT is meant to describe hardware. As far as I know, the hardware I own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
.. and the patching I mention above are largely driven by either a change in understanding of how OF should describe the hardware, or a change in understanding of how the driver should treat the hardware.
The recent patching for the tegra PCI-E bridge is instructive in this regard, Theirry learned how to drive the chip in a way that creates a single PCI domain - this necessitates a change in how the DT models that hardware block.
While I certainly don't disagree with the sentiment of your email, Tegra PCIe is a bad example. There is no finalized DT binding yet for the Tegra PCIe HW. Thierry is in the process of defining it for the first time.
On 02/21/2013 05:11:06 PM, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 05:05:54PM -0500, Nicolas Pitre wrote:
No it is not. FIT is about bundling a multi-platform kernel with a bunch of DTBs together in a single file. I don't think you need
that
for your embedded system. The "wrong message" here is to distribute multiple DTBs around, whether it is with FIT or on a distro install media.
Actually we do this on PPC, the boot kernel image runs on three similar hardware platforms, the image has three DTBs built into it and the right one is selected at runtime. The kernel boot image does this (call it a second stage boot loader), not the primary boot loader.
I strongly disagree with the idea that keeping the DTB seperated from the kernel is appropriate for all users, or even most users. To me that only seems appropriate for certain kinds of hardware, eg general purpose computing devices that are designed to primarily run a Linux distro.
An embedded SOC eval board, a development platform, an embedded appliance - these are cases where the kernel and DTB should generally be more tightly coupled.
This is more or less how PPC has evolved, big commerical PPC systems like Apple's and IBM's stuff all provide a DTB to the kernel - and this is actually a bit different then the DT's people are writing for SOCs, it is firmware generated and includes a full description of all the probed hardware - including pluggable PCI cards and other stuff. The hardware is also left configured so there is less for Linux to do and less that needs to be described in DT.
While embedded focused PPC stuff seems to tend to keep the kernel and DT together.
At least on the Freescale side of "embedded focused PPC stuff", we have not kept the kernel and DT together. It's actually U-Boot that the dts files in the kernel tree are tied to, since they contain assumptions about how U-Boot lays out the memory map (there are some inherent limits to "the device tree just describes the hardware", barring some radical changes in the form device trees take), which things U-Boot will fill in/modify, and what U-Boot looks for to find out where to make the modification. Usually U-Boot is the only relevant loader for a particular board, but not always -- hence "adder875-redboot.dts" and "adder875-uboot.dts". Even when U-Boot is the only relevant loader, there are sometimes changes from one version or configuration of U-Boot to another that cause problems (e.g. the device trees that come in "32b" and "36b" variants).
-Scott
On Thu, Feb 21, 2013 at 04:11:06PM -0700, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 05:05:54PM -0500, Nicolas Pitre wrote:
No it is not. FIT is about bundling a multi-platform kernel with a bunch of DTBs together in a single file. I don't think you need that for your embedded system. The "wrong message" here is to distribute multiple DTBs around, whether it is with FIT or on a distro install media.
Actually we do this on PPC, the boot kernel image runs on three similar hardware platforms, the image has three DTBs built into it and the right one is selected at runtime. The kernel boot image does this (call it a second stage boot loader), not the primary boot loader.
If that's something that PPC does, great. It's not something that we have any support for on ARM, nor do we have any intention at present to add support for it.
The stop-gap for ATAG-passing boot loaders, where you can append a DTB object to the zImage, and have the zImage update the DTB with information from the ATAG stuff will only work with a _single_ DTB.
I strongly disagree with the idea that keeping the DTB seperated from the kernel is appropriate for all users, or even most users. To me that only seems appropriate for certain kinds of hardware, eg general purpose computing devices that are designed to primarily run a Linux distro.
Actually, this is not really even our decision. This direction was rather set by Linus T. himself when he found out what has been going on in OMAP, with all the massive amounts of data which are present in the kernel. He has said that he doesn't want that data carried in the kernel source. Hell, this is not the first time he's objected; he's threatened to delete all our defconfig files because they're getting too many.
So, like it or not, we're going to face the same problem with DTBs that we face with the sprawing code we have in the kernel, and which we've had with the defconfigs. Linus _will_ at some point get pissed off with them and threaten to delete them.
And it won't matter what you think one bit when Linus makes that decision. Nor what we think. Because he'll either complain and give an ultimatum, or he'll just do it.
Disagree. We are already seeing patching now for 2nd generation DT bindings to fix flaws in bindings that were introduced earlier. I hope the rate will slow down, but the need will probably never go away completely. :(
This is where pressure back to people to stop them behaving like this is important - and it _isn't_ being helped by having the DTBs as part of the kernel - it makes it _too_ easy for this kind of stuff to happen without proper controls.
Multiple *kernel packages* is a big problem, one *kernel package* is generally not.
It is already the case on x86 that a kernel package can't boot out of the box. The distro builds a box-specific initramfs on boot that minimally includes enough modules to access the root fs storage. Grabbing a box specific DT as well is a tiny additional step.
You're confused there. You're comparing the wrong things.
On x86, the modules needed for the rootfs are generally held in an initramfs, because that provides an easy way to collect together the parts of the system that are needed to find the rootfs.
However, that's not what we're talking about when we're talking about DTB. An initramfs doesn't describe the hardware. So you're comparing apples and oranges and expecting us to take you seriously for doing so.
What you should be comparing in this instance is DTB with ACPI. ACPI describes the hardware on which you're booting your x86 kernel. It says what devices are present in the system (which may change while the kernel is running - think laptops which gain ports when you dock them.)
You don't see x86 distros including large chunks of ACPI data on their DVDs... That's all provided by the motherboard OEM.
Bear in mind, that like for storage, when the kernel is installed the system is *already running*. This means it knows what storage modules are needed, and similarly it knows the content of the DTB it is using. It can do three things with this:
- See if /lib/device-tree/.. contains a compatible DTB, if so use the version from /lib
- Save the DTB to /boot/my-board-dtb and use it
- Realize that it is OEM provided and comes from the firmware, do nothing
So things can very much be fully automated.
You've a chicken and egg problem there. If the kernel is already running on a DT-based system, then it has already been provided with a DTB. That DTB is available from the kernel itself, and can be saved. But what's the point if _that_ kernel was already able to get it from somewhere - probably provided via the board firmware in the first place.
See?
At the point where you have your first kernel running for the install, you must already be using the right DTB file which must have come from somewhere. The egg(DTB) must come before the chicken(kernel).
According to your logic, distros could package and distribute BIOS updates for all the X86 systems out there. After all, if they did, they would guarantee even better support on the hardware they target and not have to carry those ACPI quirks in the kernel, no?
The distros are going to include uboot packages and people are going to try and support a wide range of boards through uboot - so yah, on ARM distros are packaging full BIOS updates for ARM. This doesn't seem to be a problem.
I've not noticed ubuntu providing a kernel, let alone a uboot for my OMAP boards nor for my Cubox...
If the DTs are moved out of the kernel, then the distros will build and package them too.
Heck, on x86 some distros do make use of the runtime ACPI patching stuff to fix particularly broken firmware.
They may patch issues with the ACPI data, but they do _not_ include the entire ACPI data structure for things like laptops and the like. That's the point - they don't want to include full ACPI data dumps because then they're into the realm of having to know precisely all the details of the platform that they're running on - and they probably don't have enough information to know that.
For instance, how would it know if I docked my laptop and suddenly a serial and printer port becomes available if it was using a fixed set of data?
Ask them if they find this idea rejoicing. You might be surprised.
This stuff is never rejoicing. But give the distro two choices,
- Include the /lib/device-tree/ .. stuff from the kernel build and things will work robustly *on systems that require it*
- Don't, and a kernel update or firmware update might randomly result in boot failure
Which do you think they will they pick?
Relying on OEMs to provide working firmware has been a *nightmare* on x86. There is no reason to think ARM OEMs would do any better. Minimizing the amount of OEM specific junk that needs to be used is a good thing.
All these arguments you're bringing up are arguments that have been made against DT since before ARM went that route. Grant Likely etc all made assurances that this would _not_ be a problem, and that anyone making incompatible DT changes would be taken out and shot (not literally, but you get the idea.)
If what you're saying is true, then we've been mislead about DT, and we're currently going through a pointless exercise which is only going to result in increased complexity and more things to go wrong with booting a kernel. And we should stop this DT farce now.
Heck, just try to get an OEM supported mainline kernel for some of the eval boards they ship. Good luck....
What is being aimed for on things like OMAP is the ability to describe the SoC and everything external to the SoC by DT. What that means is that the SoC configuration gets described by DT. The external devices like Ethernet controllers gets described by DT. The keyboard matrix gets described by DT. And so forth. Which should ultimately mean that a new OMAP board gets support by just supplying the appropriate DT file.
Sure, it's not there yet (it's still being worked on, just like much of the other DT conversion process) - remember, DT on ARM is _very_ new, and is in its infancy. We _expect_ things to change at the moment all over the place because we're still developing various bits of basic infrastructure necessary to describe parts of our SoCs - such as most recently pin muxing.
Pin muxing hasn't been in DT before - it's a totally new concept there which has had to be developed from scratch.
Dear Nicolas,
In message alpine.LFD.2.03.1302211624561.6419@syhkavp.arg you wrote:
No it is not. FIT is about bundling a multi-platform kernel with a bunch of DTBs together in a single file. I don't think you need that
Actually this is neither the only, nor even the primary purpose of FIT images; these have a much wider scope of usage scenarios.
The DT is meant to describe hardware. As far as I know, the hardware I own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
There is other hardware available (for example FPGA based) where this does not apply.
Best regards,
Wolfgang Denk
On Fri, Feb 22, 2013 at 12:18:48AM +0100, Wolfgang Denk wrote:
The DT is meant to describe hardware. As far as I know, the hardware I own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
There is other hardware available (for example FPGA based) where this does not apply.
Agreed.. We do that here as well, the DT is also used to describe the functionality inside FPGA(s). We do things like declare a GPIO controller inside the FPGA, then stack the bitbang MDIO/I2C on top of that, then declare a bunch of devices on those busses. DT makes this extremely straightforward.
However, it is critical that the DT, kernel and FPGA are matched together - we always arrange things so that the DTB, kernel and FPGA config are bundled together and update atomically during firmware upgrade.
Xilinx's Zynq is a great example of this kind of stuff, FWIW. IIRC Xilinx has a DT generator from their IP tools, so you can literally go into their design software, configure the hardware IP blocks, and get back a FPGA config and a DT to go with it.
Jason
On 02/21/2013 05:28 PM, Jason Gunthorpe wrote:
On Fri, Feb 22, 2013 at 12:18:48AM +0100, Wolfgang Denk wrote:
The DT is meant to describe hardware. As far as I know, the hardware I own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
There is other hardware available (for example FPGA based) where this does not apply.
Agreed.. We do that here as well, the DT is also used to describe the functionality inside FPGA(s). We do things like declare a GPIO controller inside the FPGA, then stack the bitbang MDIO/I2C on top of that, then declare a bunch of devices on those busses. DT makes this extremely straightforward.
However, it is critical that the DT, kernel and FPGA are matched together - we always arrange things so that the DTB, kernel and FPGA config are bundled together and update atomically during firmware upgrade.
Xilinx's Zynq is a great example of this kind of stuff, FWIW. IIRC Xilinx has a DT generator from their IP tools, so you can literally go into their design software, configure the hardware IP blocks, and get back a FPGA config and a DT to go with it.
The desired FPGA use case is DT updates after booting the kernel. This has nothing to do with FIT images. And if the FPGA tools generate the DTB, then it is certainly not tied to the kernel.
Rob
On Thu, Feb 21, 2013 at 06:19:05PM -0600, Rob Herring wrote:
The desired FPGA use case is DT updates after booting the kernel. This has nothing to do with FIT images. And if the FPGA tools generate the DTB, then it is certainly not tied to the kernel.
Completely unrelated, but do you have any pointer for how to do this? Hot plugging a 'dtb fragment' into the kernel would be really handy..
I'm thinking something like adding a tree below a PCI controller describing a PCI device and sub nodes, similar to what Thierry was doing for his Avionics. How would interrupt maps and phandles be managed across the main dtb and the 'hot plugged' dtb?
Regards, Jason
On 02/21/2013 08:22 PM, Jason Gunthorpe wrote:
On Thu, Feb 21, 2013 at 06:19:05PM -0600, Rob Herring wrote:
The desired FPGA use case is DT updates after booting the kernel. This has nothing to do with FIT images. And if the FPGA tools generate the DTB, then it is certainly not tied to the kernel.
Completely unrelated, but do you have any pointer for how to do this? Hot plugging a 'dtb fragment' into the kernel would be really handy..
Look at "Introducing Device Tree Overlays" on devicetree-discuss and lkml.
I'm thinking something like adding a tree below a PCI controller describing a PCI device and sub nodes, similar to what Thierry was doing for his Avionics. How would interrupt maps and phandles be managed across the main dtb and the 'hot plugged' dtb?
Generally you don't describe the devices on PCI unless there are non-discoverable aspects like a sideband gpio.
Rob
On Thu, Feb 21, 2013 at 9:22 PM, Jason Gunthorpe jgunthorpe@obsidianresearch.com wrote:
On Thu, Feb 21, 2013 at 06:19:05PM -0600, Rob Herring wrote:
The desired FPGA use case is DT updates after booting the kernel. This has nothing to do with FIT images. And if the FPGA tools generate the DTB, then it is certainly not tied to the kernel.
Completely unrelated, but do you have any pointer for how to do this? Hot plugging a 'dtb fragment' into the kernel would be really handy..
This doesn't answer the full question on how FPGA tools generate DTB, but it is a huge problem for BeagleBone add-on hardware that we have some mechanism to dynamically load DT fragments. Pantelis posted some work in that direction[1] and has continued development of his patches and we've been using those extensively with BeagleBone kernel development[2].
It would be great if the FPGA folks would get on-board in supporting the dynamically loadable DT overlay fragments by reviewing and supporting upstream acceptance of the code.
[1] http://lwn.net/Articles/531569/ [2] http://github.com/beagleboard/kernel/tree/3.8
I'm thinking something like adding a tree below a PCI controller describing a PCI device and sub nodes, similar to what Thierry was doing for his Avionics. How would interrupt maps and phandles be managed across the main dtb and the 'hot plugged' dtb?
Regards, Jason _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
On Fri, Feb 22, 2013 at 02:56:33AM -0500, Jason Kridner wrote:
The desired FPGA use case is DT updates after booting the kernel. This has nothing to do with FIT images. And if the FPGA tools generate the DTB, then it is certainly not tied to the kernel.
Completely unrelated, but do you have any pointer for how to do this? Hot plugging a 'dtb fragment' into the kernel would be really handy..
This doesn't answer the full question on how FPGA tools generate DTB, but it is a huge problem for BeagleBone add-on hardware that we have
In all my cases I hand code the DT's for the FPGA IP, they get quite big, the one I'm looking at now has over 17 DT nodes inside the FPGA already. This is why we are re-using the DT framework to describe the chip, not building a C coded PCI driver to do the same work.
some mechanism to dynamically load DT fragments. Pantelis posted some work in that direction[1] and has continued development of his patches and we've been using those extensively with BeagleBone kernel development[2].
Yes, I saw this. They look really interesting, I think the notion of hot plugging a DT fragment into the main DT is useful for many applications..
Jason
Dear Jason,
In message 20130221232821.GA2823@obsidianresearch.com you wrote:
own seems to be rather static and stable, and unlike software there is no way I can change it (soldering irons don't count).
There is other hardware available (for example FPGA based) where this does not apply.
Agreed.. We do that here as well, the DT is also used to describe the functionality inside FPGA(s). We do things like declare a GPIO controller inside the FPGA, then stack the bitbang MDIO/I2C on top of that, then declare a bunch of devices on those busses. DT makes this extremely straightforward.
However, it is critical that the DT, kernel and FPGA are matched together - we always arrange things so that the DTB, kernel and FPGA config are bundled together and update atomically during firmware upgrade.
Agreed.
Xilinx's Zynq is a great example of this kind of stuff, FWIW. IIRC
Indeed - Xilinx's Zynq, Altera's SoC FPGA, and others.
Such highly flexible hardware configurations require a new level of software support that by far exceeds the classic static setups of more "PC-like" systems where the only change you would expect is adding or removing some PCI cards or the like.
Best regards,
Wolfgang Denk
On 02/21/2013 03:05 PM, Nicolas Pitre wrote:
On Thu, 21 Feb 2013, Jason Gunthorpe wrote:
...
Distros already ship huge kernels with modules for every hardware out there. Shipping all the DTs as well doesn't seem like a problem.
But it is! Even shipping multiple kernels _is_ a problem for them. Hence this multi-platform kernel effort. Otherwise why would we bother?
According to your logic, distros could package and distribute BIOS updates for all the X86 systems out there. After all, if they did, they would guarantee even better support on the hardware they target and not have to carry those ACPI quirks in the kernel, no?
Ask them if they find this idea rejoicing. You might be surprised.
Well they ship x86 CPU firmware updates according to the boot log on one of my systems at least...
On Thu, Feb 21, 2013 at 04:45:37PM -0700, Stephen Warren wrote:
Well they ship x86 CPU firmware updates according to the boot log on one of my systems at least...
Correction: CPU microcode updates. That's updating the microcode in the CPU which runs the x86 instruction set. It's done to fix problems discovered with execution of the CPU instruction set.
That's entirely separate from ACPI or DT blobs provided by the OEM.
"Jason" == Jason Gunthorpe jgunthorpe@obsidianresearch.com writes:
Hi,
Jason> We've been using DT on production embedded stuff sice about 2.6.20ish Jason> on PPC and now ARM. We treat the dtb as a kernel version specific Jason> file, much like an initrd and ensure that the kernel only ever boots Jason> with its proper dtb. This is based on experience that the dtbs change Jason> depending on the state of the drivers in the kernel, what gets Jason> mainlined and when, etc.
Jason> Embedding this stuff into the bootloader is *not* desirable for my Jason> embedded scenarios. We don't use FIT (or uboot) but we do the same Jason> thing: a single image is constructed with the proper dtb, kernel and Jason> initrd, and that is what the bootloader boots.
Jason> Why? This is an embedded appliance product. We need to be able to Jason> deliver firmware upgrades that *work*. We can't brick the board Jason> because the bootloader and kernel get out of sync. The boot loader has Jason> to be *simple*, it has to boot every past, present and future kernel Jason> or we start taking risks that a firmware flash will end up bricking Jason> it.
I'm having the exact same experience (and handle it in the same way).
I know this was not the intention of DT, but unfortunately this is how things have turned out so far.
Dear Russell,
In message 20130221134656.GC17852@n2100.arm.linux.org.uk you wrote:
Note that FIT images are relatively old (docs date back to March 2008). This is more of another effort to try and update what the kernel uses.
So it's five years old and people haven't been that interested in it so far. That speaks volumes...
Well, your attitude to (not) accept anything like that is well known; there have been few previews attempts, which have all been rejected in basicly the same way.
But now the situation has changed - multiplatform support adds a number of use cases where this feature comes in handy, so it pops up again.
Best regards,
Wolfgang Denk
On Thu, Feb 21, 2013 at 4:37 AM, Russell King - ARM Linux linux@arm.linux.org.uk wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Hello, I've been spinning some work-in-progress patches for FIT build support in the kernel. With the move to multiplatform support on OMAP, I feel it is a good time to add FIT support, also looking at the proliferating number of dtbs, as it is a nice way
I'm not looking to add yet another crappy uboot image format to the kernel just for the hell of uboot.
And don't think that the dtbs in the kernel are going to stay there. The longer term plan is for them to end up in an external git tree, entirely separate from the kernel source. So anything that ties them with the kernel build process will ultimately have to be removed again.
Sorry for hopping in late as I have been travelling.
Guess much has been said, but for courtesy sake till dts/u-boot build support is dropped, would you consider accepting a simple extension to mkuboot.sh with minimal Kbuild changes? I can promise there will be quite a few users.
Lots of folks want embedded device trees and there are already out of tree hacks floating as you know that do this. FIT approach to this is more cleaner I feel.
Let me know what you can do.
Thanks, Joel
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Any comments on this approach? Is it better to merge mkfitsrc.sh with mkuboot.sh?
I know this was discussed quite extensively yesterday, but here is my take on it:
Given the recent complications from multiplatform, we really saw a strong reason to _not_ do the final boot wrapping in the kernel build system. Produce the zImage and the DTB files, and have a surrounding script that bundles the two in a format that your particular device needs.
Most distros have scripts to handle the "make install" step of a kernel build. That's where this belongs, not in the actual build step.
-Olof
On Fri 2013-02-22 08:00:44, Olof Johansson wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Any comments on this approach? Is it better to merge mkfitsrc.sh with mkuboot.sh?
I know this was discussed quite extensively yesterday, but here is my take on it:
Given the recent complications from multiplatform, we really saw a strong reason to _not_ do the final boot wrapping in the kernel build system. Produce the zImage and the DTB files, and have a surrounding script that bundles the two in a format that your particular device needs.
Most distros have scripts to handle the "make install" step of a kernel build. That's where this belongs, not in the actual build step.
Not sure I agree here:
1) zImage was designed to boot from floppy, without any kind of bootloader on i386
2) There are no distributions on many embedded systems. And no good place where to store the scripts. Pavel
On Mon, Mar 18, 2013 at 05:36:53PM +0100, Pavel Machek wrote:
On Fri 2013-02-22 08:00:44, Olof Johansson wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Any comments on this approach? Is it better to merge mkfitsrc.sh with mkuboot.sh?
I know this was discussed quite extensively yesterday, but here is my take on it:
Given the recent complications from multiplatform, we really saw a strong reason to _not_ do the final boot wrapping in the kernel build system. Produce the zImage and the DTB files, and have a surrounding script that bundles the two in a format that your particular device needs.
Most distros have scripts to handle the "make install" step of a kernel build. That's where this belongs, not in the actual build step.
Not sure I agree here:
- zImage was designed to boot from floppy, without any kind of
bootloader on i386
zImage on x86 != zImage on ARM.
They only share the fact that they're both compressed and self-extracting.
ARM has never supported any kind of floppy based boot - mainly because floppies have never been "executable" like they are on x86; all that previous boot loaders ever did was load the kernel into RAM using some method and call address 0, with a data structure (formerly struct params, later atags) at a fixed address, later passing that address to the kernel.
The best floppies got was being able to run a script or load an executable off the floppy under the ROM'd OS, and our zImage itself doesn't fit the format required for that.
Our zImage has always been just a pure straight binary compressed kernel format. Nothing more.
On Mon 2013-03-18 16:44:26, Russell King - ARM Linux wrote:
On Mon, Mar 18, 2013 at 05:36:53PM +0100, Pavel Machek wrote:
On Fri 2013-02-22 08:00:44, Olof Johansson wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Any comments on this approach? Is it better to merge mkfitsrc.sh with mkuboot.sh?
I know this was discussed quite extensively yesterday, but here is my take on it:
Given the recent complications from multiplatform, we really saw a strong reason to _not_ do the final boot wrapping in the kernel build system. Produce the zImage and the DTB files, and have a surrounding script that bundles the two in a format that your particular device needs.
Most distros have scripts to handle the "make install" step of a kernel build. That's where this belongs, not in the actual build step.
Not sure I agree here:
- zImage was designed to boot from floppy, without any kind of
bootloader on i386
zImage on x86 != zImage on ARM.
They only share the fact that they're both compressed and self-extracting.
ARM has never supported any kind of floppy based boot - mainly because floppies have never been "executable" like they are on x86; all that previous boot loaders ever did was load the kernel into RAM using some
Yes, I know. That's why I said "on i386".
What I wanted to say is that kernel build traditionaly produced something useful, something bootloader can actually boot. Currently, make uImage produces u-boot image. Please keep that capability. Pavel
On Mon, Mar 18, 2013 at 06:49:32PM +0100, Pavel Machek wrote:
What I wanted to say is that kernel build traditionaly produced something useful, something bootloader can actually boot. Currently, make uImage produces u-boot image. Please keep that capability.
Unfortunately, there is a fundamental problem with uImage. It encodes the load address, and that is utterly incompatible with the goal of having a kernel image which boots on multiple platforms.
uImage isn't going anywhere for the time being, but you will find that:
a) recent kernels will just wrap the zImage without a DTB, so appended DTBs can't be used with it b) it won't work with MULTIPLATFORM-enabled kernels, unless you provide make with a LOADADDR= argument to tell it where the uImage is to be loaded - which is platform specific.
(b) fundamentally can't be solved in kbuild. If you want to use uImage with multiple platforms from the same zImage, then you need to generate multiple uImage files with different load addresses. That's purely - and solely - down to the dysfunctional uImage format.
On Mon 2013-03-18 17:57:46, Russell King - ARM Linux wrote:
On Mon, Mar 18, 2013 at 06:49:32PM +0100, Pavel Machek wrote:
What I wanted to say is that kernel build traditionaly produced something useful, something bootloader can actually boot. Currently, make uImage produces u-boot image. Please keep that capability.
Unfortunately, there is a fundamental problem with uImage. It encodes the load address, and that is utterly incompatible with the goal of having a kernel image which boots on multiple platforms.
uImage isn't going anywhere for the time being, but you will find that:
a) recent kernels will just wrap the zImage without a DTB, so appended DTBs can't be used with it b) it won't work with MULTIPLATFORM-enabled kernels, unless you provide make with a LOADADDR= argument to tell it where the uImage is to be loaded - which is platform specific.
(b) fundamentally can't be solved in kbuild. If you want to use uImage with multiple platforms from the same zImage, then you need to generate multiple uImage files with different load addresses. That's purely - and solely - down to the dysfunctional uImage format.
I know. uImage is not perfect. But solution should be "adopt a better format" (ARM FIT?), not "drop support for making useful images altogether" as Olof proposed before. Pavel
On 03/18/2013 12:04 PM, Pavel Machek wrote:
On Mon 2013-03-18 17:57:46, Russell King - ARM Linux wrote:
On Mon, Mar 18, 2013 at 06:49:32PM +0100, Pavel Machek wrote:
What I wanted to say is that kernel build traditionaly produced something useful, something bootloader can actually boot. Currently, make uImage produces u-boot image. Please keep that capability.
Unfortunately, there is a fundamental problem with uImage. It encodes the load address, and that is utterly incompatible with the goal of having a kernel image which boots on multiple platforms.
uImage isn't going anywhere for the time being, but you will find that:
a) recent kernels will just wrap the zImage without a DTB, so appended DTBs can't be used with it b) it won't work with MULTIPLATFORM-enabled kernels, unless you provide make with a LOADADDR= argument to tell it where the uImage is to be loaded - which is platform specific.
(b) fundamentally can't be solved in kbuild. If you want to use uImage with multiple platforms from the same zImage, then you need to generate multiple uImage files with different load addresses. That's purely - and solely - down to the dysfunctional uImage format.
I know. uImage is not perfect. But solution should be "adopt a better format" (ARM FIT?), not "drop support for making useful images altogether" as Olof proposed before.
Raw zImage /is/ the useful format that should be adopted.
Dear Stephen,
In message 51475997.2060008@wwwdotorg.org you wrote:
Raw zImage /is/ the useful format that should be adopted.
This "one size fits all" approch does fit everywhere. There are a number of users (including _big_ commercial ones, with _large_ numebrs of systems in the field) that have simple requirements like:
- how can I find out if the image I just loaded to RAM in OK (OK maning some checksum is correct, or some crypto-key could be verified, or ...) ?
- how can I find out which version of image is installed on this device in the flash? Can I print something like an ID string, or a build timestamp, etc?
etc. etc.
zImage may work well in many cases, but there are also many cases where in image format that allows for additional meta-data is mandatory.
Best regards,
Wolfgang Denk
Dear Russell,
In message 20130318175746.GJ30923@n2100.arm.linux.org.uk you wrote:
Unfortunately, there is a fundamental problem with uImage. It encodes the load address, and that is utterly incompatible with the goal of having a kernel image which boots on multiple platforms.
I'm not sure if you are aware that U-Boot supports uImages that can be run from _any_ load address (using the IH_TYPE_KERNEL_NOLOAD image type). If anybody was willing to keep uImage support in the kernel this would probably be a convenient way to go.
But in any case, the uImage format is deprecated. Support for FIT images was added more than five years ago, and new developments should rather focus on supporting these.
Best regards,
Wolfgang Denk
On Mon, Mar 18, 2013 at 05:36:53PM +0100, Pavel Machek wrote:
On Fri 2013-02-22 08:00:44, Olof Johansson wrote:
On Wed, Feb 20, 2013 at 07:37:10PM -0600, Joel A Fernandes wrote:
Any comments on this approach? Is it better to merge mkfitsrc.sh with mkuboot.sh?
I know this was discussed quite extensively yesterday, but here is my take on it:
Given the recent complications from multiplatform, we really saw a strong reason to _not_ do the final boot wrapping in the kernel build system. Produce the zImage and the DTB files, and have a surrounding script that bundles the two in a format that your particular device needs.
Most distros have scripts to handle the "make install" step of a kernel build. That's where this belongs, not in the actual build step.
Not sure I agree here:
Lets try and stop this again here. I think perhaps the KVM tool example is instructive here. For the various reasons that close association with the kernel can be helpful for things (the exposure and ease of being found), it would be nice if the tooling to expand single kernel image into single "bootable" image was with the kernel. But it's not a requirement. And it's not even necessarily the best for the tooling either. So, lets drop the idea of getting this into the kernel and if people really do wish to extend FIT such that we can easily spit out a FIT image that works on omap* and tegra* or what have you, and add a FIT parser to GRUB, great, get to work. No need to be tied to the kernel for this.
participants (13)
-
Jason Gunthorpe -
Jason Kridner -
Joel A Fernandes -
Nicolas Pitre -
Olof Johansson -
Pavel Machek -
Peter Korsgaard -
Rob Herring -
Russell King - ARM Linux -
Scott Wood -
Stephen Warren -
Tom Rini -
Wolfgang Denk