RFC: Devices for files and partitions
Hi,
At present U-Boot avoids the concept of 'opening' a file. Being in a bootloader environment, it is normally better to take the action immediately and avoid any caching, for example, since there is no background task to clean up afterwards.
Having said that, the concept of a file is quite useful, for example to write the output of a command to a file, or to open a file and read it a line at a time.
Another case has come to light in that EFI wants to access files using a file handle. This currently uses parallel data structures and does not map very well in U-Boot.
Finally, partitions has a similar issue, where defining them as a device can have benefits, e.g. to specify the device to use directly, rather than with the 'type dev:part' approach, perhaps providing them in the devicetree, etc.
For the above reasons, I propose that we implement, as an option, support for files and partitions within driver model.
Regards, Simon
On Tue, Jul 06, 2021 at 05:05:19PM -0600, Simon Glass wrote:
Hi,
At present U-Boot avoids the concept of 'opening' a file. Being in a bootloader environment, it is normally better to take the action immediately and avoid any caching, for example, since there is no background task to clean up afterwards.
Having said that, the concept of a file is quite useful, for example to write the output of a command to a file, or to open a file and read it a line at a time.
Another case has come to light in that EFI wants to access files using a file handle. This currently uses parallel data structures and does not map very well in U-Boot.
Finally, partitions has a similar issue, where defining them as a device can have benefits, e.g. to specify the device to use directly, rather than with the 'type dev:part' approach, perhaps providing them in the devicetree, etc.
For the above reasons, I propose that we implement, as an option, support for files and partitions within driver model.
+1
# Nobody has commented yet :)
Regarding a "file (or file descriptor)", we have already implemented the same concept in efi_loader. So technically, it won't be a hard-work. Regarding "partitions as udevice," I have posted an experimental patch [1]. So it must also be feasible.
One of my concerns is what benefit end users may enjoy.
-Takahiro Akashi
[1] https://lists.denx.de/pipermail/u-boot/2019-February/357937.html https://lists.denx.de/pipermail/u-boot/2019-February/357934.html
Regards, Simon
On 7/9/21 4:44 AM, AKASHI Takahiro wrote:
On Tue, Jul 06, 2021 at 05:05:19PM -0600, Simon Glass wrote:
Hi,
At present U-Boot avoids the concept of 'opening' a file. Being in a bootloader environment, it is normally better to take the action immediately and avoid any caching, for example, since there is no background task to clean up afterwards.
Having said that, the concept of a file is quite useful, for example to write the output of a command to a file, or to open a file and read it a line at a time.
Another case has come to light in that EFI wants to access files using a file handle. This currently uses parallel data structures and does not map very well in U-Boot.
Not having a file descriptor slows down file operation because for each individual access to a file we
* read the partition table * look up the file in the directory structure
If you only use the 'load' command, this does not matter because there is only a single read. But in the UEFI sub-system we use multiple API calls:
* open the volume * open the file * read the file size (for buffer allocation) * read the file
The file descriptors in the UEFI sub-system only hold device, partition ID, file path.
Finally, partitions has a similar issue, where defining them as a device can have benefits, e.g. to specify the device to use directly, rather than with the 'type dev:part' approach, perhaps providing them in the devicetree, etc.
For the above reasons, I propose that we implement, as an option, support for files and partitions within driver model.
With partitions as devices we will have to read the partition table only once when the block device is probed. - For the UEFI sub-system we need all block devices to be probed before calling the UEFI binary (e.g. GRUB).
+1
# Nobody has commented yet :)
Regarding a "file (or file descriptor)", we have already implemented the same concept in efi_loader. So technically, it won't be a hard-work. Regarding "partitions as udevice," I have posted an experimental patch [1]. So it must also be feasible.
Our UEFI file descriptor is not helpful because it does not buffer the relevant data to speed up file access. E.g. for the FAT driver a file descriptor could buffer some information from the directory entry (file attributes, dates) and some from the FAT table (assigned extends).
One of my concerns is what benefit end users may enjoy.
Improved performance.
Best regards
Heinrich
-Takahiro Akashi
[1] https://lists.denx.de/pipermail/u-boot/2019-February/357937.html https://lists.denx.de/pipermail/u-boot/2019-February/357934.html
Regards, Simon
Hi,
On Thu, 8 Jul 2021 at 23:33, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 7/9/21 4:44 AM, AKASHI Takahiro wrote:
On Tue, Jul 06, 2021 at 05:05:19PM -0600, Simon Glass wrote:
Hi,
At present U-Boot avoids the concept of 'opening' a file. Being in a bootloader environment, it is normally better to take the action immediately and avoid any caching, for example, since there is no background task to clean up afterwards.
Having said that, the concept of a file is quite useful, for example to write the output of a command to a file, or to open a file and read it a line at a time.
Another case has come to light in that EFI wants to access files using a file handle. This currently uses parallel data structures and does not map very well in U-Boot.
Not having a file descriptor slows down file operation because for each individual access to a file we
- read the partition table
- look up the file in the directory structure
If you only use the 'load' command, this does not matter because there is only a single read. But in the UEFI sub-system we use multiple API calls:
- open the volume
- open the file
- read the file size (for buffer allocation)
- read the file
The file descriptors in the UEFI sub-system only hold device, partition ID, file path.
Arguably that is a potential benefit, dependent on us implementing (write-through?) caching of this info.
Finally, partitions has a similar issue, where defining them as a device can have benefits, e.g. to specify the device to use directly, rather than with the 'type dev:part' approach, perhaps providing them in the devicetree, etc.
For the above reasons, I propose that we implement, as an option, support for files and partitions within driver model.
With partitions as devices we will have to read the partition table only once when the block device is probed. - For the UEFI sub-system we need all block devices to be probed before calling the UEFI binary (e.g. GRUB).
It would be better if UEFI could support lazy probing, but that's off-topic here.
+1
# Nobody has commented yet :)
Thank you for commenting.
Regarding a "file (or file descriptor)", we have already implemented the same concept in efi_loader. So technically, it won't be a hard-work. Regarding "partitions as udevice," I have posted an experimental patch [1]. So it must also be feasible.
Our UEFI file descriptor is not helpful because it does not buffer the relevant data to speed up file access. E.g. for the FAT driver a file descriptor could buffer some information from the directory entry (file attributes, dates) and some from the FAT table (assigned extends).
One of my concerns is what benefit end users may enjoy.
Improved performance.
With a bit of effort, yes, with UEFI at least, by the sound of it.
I'm not sure how much difference it would make for non-EFI use though.
To me this is an internal thing, a bit like driver model, where the benefit is easier development and scripting, which might benefit end users indirectly through faster time-to-market and fewer bugs?
Takahiro, since you have already started this work, do you want to continue it? I think the original patch needed a test.
Best regards
Heinrich
-Takahiro Akashi
[1] https://lists.denx.de/pipermail/u-boot/2019-February/357937.html https://lists.denx.de/pipermail/u-boot/2019-February/357934.html
Regards, Simon
Regards, Simon
participants (3)
-
AKASHI Takahiro -
Heinrich Schuchardt -
Simon Glass