[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU on several devices of the same type. Dev_part - number of partitions on the device. FS - information about file system on the device (fat, ext2/ext4, ubi). start - start offset from the beginning of the Device (byte or LBA number addressed) size - maximal number of blocks to be stored on the Device (expressed with Bytes of LBA number) (protection from overwriting the whole device)
Example:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size | ---------------------------------------------------------------------- u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" | root.img | img | mmc | 0 | 5 | "-" | "-" | "-" | root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
command | cmd | "-" | "-" | "-" | "-" | "-" | "-" |
[*] - support for sending command via DFU (see below) [**] - "-" - indicates that no value is provided for this field. Despite of that it is mandatory to facilitate parsing.
Please share any possible use cases - for now and for the future. My goal is to devise dfu_alt_info structure which would work for us for a long time.
After setting up new format, it would be possible to replace different dfu command variations (i.e. dfu mmc 0, dfu nand) with only one command "dfu".
------------------------------------------------------------------------ DFU extension - Command execution. ------------------------------------------------------------------------
As a follow up I plan to add support for sending file filled with command[s]. This would provide interface for automated tests and force board to reset itself after emulated reset command from dfu-util program.
dfu_alt_info = "command cmd - - - - - -;"
and then on host: dfu-util -aN -D file_with_commands
The end step would be to extend dfu-util to support -C switch, which would allow to send u-boot command quoted as text:
dfu-util -aN -C "reset"
On Tue, Jul 16, 2013 at 5:35 PM, Lukasz Majewski wrote:
The end step would be to extend dfu-util to support -C switch, which would allow to send u-boot command quoted as text:
dfu-util -aN -C "reset"
Instead of adding a new, rather u-boot specific option to dfu-util, would implementation of the commonly used "-" filename for stdin work for you?
echo "reset" | dfu-util -aN -
Cheers, Tormod
On Tue, 16 Jul 2013 23:27:44 +0200 Tormod Volden lists.tormod@gmail.com wrote:
Hi Tormod,
On Tue, Jul 16, 2013 at 5:35 PM, Lukasz Majewski wrote:
The end step would be to extend dfu-util to support -C switch, which would allow to send u-boot command quoted as text:
dfu-util -aN -C "reset"
Instead of adding a new, rather u-boot specific option to dfu-util, would implementation of the commonly used "-" filename for stdin work for you?
Definitely YES :-).
echo "reset" | dfu-util -aN -
I think, that above is more versatile solution.
Cheers, Tormod
Regards, Lukasz Majewski
Hello Lukasz,
Am 16.07.2013 17:35, schrieb Lukasz Majewski:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Full Ack!
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU on several devices of the same type. Dev_part - number of partitions on the device.
Should this be "number of the partition on the device"
You mean here the mtd partition for storing, right?
FS - information about file system on the device (fat, ext2/ext4, ubi). start - start offset from the beginning of the Device (byte or LBA number addressed) size - maximal number of blocks to be stored on the Device (expressed with Bytes of LBA number) (protection from overwriting the whole device)
Example:
Maybe dummy questions ...
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" |
Is this enough information? Where to store the uImage file on the ext4 partition?
root.img | img | mmc | 0 | 5 | "-" | "-" | "-" |
img means here: getting an image and storing it on the mtd partition "Dev_part" if start and size are marked with "-", beginning on start of the partition?
Wouldn't it be better to use here mtd partition names instead numbers for "Dev_part"?
What if "start" and "size" are filled with values for the "Type" "img"? Or is this forbidden for the "Type" "img"?
root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
s/uImage/rootfs.img ? s/file/img ?
For the FS "ubi" we need to specify, how to burn this into nand. I think we have no "ubi format" command. With "ubi write ..." we can only write ubi volumes ... and we want here to burn an ubi image, which was created with ubinize and contains one or more ubi volumes.
Maybe usecase for updating ubi volumes: ubivolumename | img | nand | 0 | 3 | ubivol | "-" | "-" |
If you want to update an ubivolume ...
results in this steps: - get image per dfu @ img_addr We need to get here the complete image, as ubi write cannot write incremental ... - ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset ^ From where get we this parameter ... value in "start"? - ubi write img_addr ubivolumename ${filesize}
command | cmd | "-" | "-" | "-" | "-" | "-" | "-" |
[*] - support for sending command via DFU (see below) [**] - "-" - indicates that no value is provided for this field. Despite of that it is mandatory to facilitate parsing.
Please share any possible use cases - for now and for the future. My goal is to devise dfu_alt_info structure which would work for us for a long time.
After setting up new format, it would be possible to replace different dfu command variations (i.e. dfu mmc 0, dfu nand) with only one command "dfu".
DFU extension - Command execution.
As a follow up I plan to add support for sending file filled with command[s]. This would provide interface for automated tests and force board to reset itself after emulated reset command from dfu-util program.
dfu_alt_info = "command cmd - - - - - -;"
and then on host: dfu-util -aN -D file_with_commands
The end step would be to extend dfu-util to support -C switch, which would allow to send u-boot command quoted as text:
dfu-util -aN -C "reset"
bye, Heiko
On Wed, 17 Jul 2013 12:26:35 +0200 Heiko Schocher hs@denx.de wrote,
Hi Heiko,
Hello Lukasz,
Am 16.07.2013 17:35, schrieb Lukasz Majewski:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Full Ack!
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU on several devices of the same type. Dev_part - number of partitions on the device.
Should this be "number of the partition on the device"
No. I made a mistake. It should be "partition to which we want to write"
You mean here the mtd partition for storing, right?
Yes, number of the partition to store data. The exact address will be extracted from mtdparts or partitions env variable.
FS - information about file system on the device (fat, ext2/ext4, ubi). start - start offset from the beginning of the Device (byte or LBA number addressed) size - maximal number of blocks to be stored on the Device (expressed with Bytes of LBA number) (protection from overwriting the whole device)
Example:
Maybe dummy questions ...
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" |
Is this enough information? Where to store the uImage file on the ext4 partition?
To store uImage file on ext4/fat/ext2 partition it is enough to only give:
uImage mmc 0 2,
which maps to the following command:
ext4write mmc 0:2 0x44000000 /uImage [size]
The file size is taken from number of sent bytes via DFU/USB.
With writing to file system, one need to first store the whole transmitted data to a buffer and then store the (big) buffer on the SD/eMMC device.
Since, we aren't supporting "seek" kind of write to current ext4 implementation at u-boot, the "big" buffer must be used.
root.img | img | mmc | 0 | 5 | "-" | "-" | "-" |
img means here: getting an image and storing it on the mtd partition "Dev_part" if start and size are marked with "-", beginning on start of the partition?
No, here img is for example a rootfs image. When storing it on the eMMC, it would be better to specify destination partition.
Wouldn't it be better to use here mtd partition names instead numbers for "Dev_part"?
I'd prefer to create a new entrance here (Part_name):
NAME | Type | Device | Dev_num | Dev_part | Part_name | FS | start | size |
I would like to avoid situation with ambiguous fields. One field shall only serve for one (clear) purpose. When not needed/used - field shall be marked as "-".
What if "start" and "size" are filled with values for the "Type" "img"? Or is this forbidden for the "Type" "img"?
I think, that each "Type" shall have predefined set of allowed attributes:
1. img: "Device" + "Dev num" + "Dev parts" 2. raw: "Device" + "Dev num" + "start" + "size" 3. file: "Device" + "Dev num" + "Dev parts" + "FS"
and so on.
root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
s/uImage/rootfs.img ? s/file/img ?
NAME: uImage and rootfs.img maps to files sent to board.
The "NAME" is used thereafter to provide exact name for file system write (ext4write/ fatwrite).
With DFU the distinction between dfu entities (files, raw images, etc) is performed via alt settings (-aX switch at dfu-util).
For the FS "ubi" we need to specify, how to burn this into nand. I think we have no "ubi format" command.
Is already ubi format command available at u-boot?
_As a side note:_
I'm now developing proof-of-concept idea of executing set of commands sent to u-boot by dfu-util [***].
Rationale for this is the lack of ability to reset u-boot after sending data to u-boot via DFU. dfu-util has -R option, but this seems to reset usb interface, not the target device.
I will set an env:
setenv dfu "dfu mmc 0;${dfu_commands}"
Then at dfu itself, I will read commands to be performed. Then I will set $dfu_commands env and exit from dfu command.
With "ubi write ..." we can only write ubi volumes ... and we want here to burn an ubi image, which was created with ubinize and contains one or more ubi volumes
Maybe usecase for updating ubi volumes: ubivolumename | img | nand | 0 | 3 | ubivol | "-" | "-" |
If you want to update an ubivolume ...
results in this steps:
- get image per dfu @ img_addr We need to get here the complete image, as ubi write cannot write incremental ...
So each volume shall have different dfu entity. Then those volumes are stored at ram with different addresses [*]?
- ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset ^ From where get we
this parameter ... value in "start"?
So here you get information about the ("Dev_part=3") partition start offset [**]?
- ubi write img_addr ubivolumename ${filesize}
Here you combine volumes sent at [*] and store it at correct offset [**]?
I'm not an expert about ubi, so please be patient with my questions :-).
For me it looks like a mixture of ordinary u-boot commands with dfu transfer[s] needed to perform correct ubi image write. Maybe [***] will help you?
command | cmd | "-" | "-" | "-" | "-" | "-" | "-" |
[*] - support for sending command via DFU (see below) [**] - "-" - indicates that no value is provided for this field. Despite of that it is mandatory to facilitate parsing.
Please share any possible use cases - for now and for the future. My goal is to devise dfu_alt_info structure which would work for us for a long time.
After setting up new format, it would be possible to replace different dfu command variations (i.e. dfu mmc 0, dfu nand) with only one command "dfu".
DFU extension - Command execution.
As a follow up I plan to add support for sending file filled with command[s]. This would provide interface for automated tests and force board to reset itself after emulated reset command from dfu-util program.
dfu_alt_info = "command cmd - - - - - -;"
and then on host: dfu-util -aN -D file_with_commands
The end step would be to extend dfu-util to support -C switch, which would allow to send u-boot command quoted as text:
dfu-util -aN -C "reset"
bye, Heiko
On Wed, Jul 17, 2013 at 4:34 PM, Lukasz Majewski wrote:
_As a side note:_
I'm now developing proof-of-concept idea of executing set of commands sent to u-boot by dfu-util [***].
Rationale for this is the lack of ability to reset u-boot after sending data to u-boot via DFU. dfu-util has -R option, but this seems to reset usb interface, not the target device.
Actually, dfu-util -R sends a DFU_DETACH request before performing the USB reset. The reason for this was long unclear to me, until I looked at the uboot dfu code (inherited from openmoko):
http://git.denx.de/?p=u-boot.git;a=blob;f=drivers/usb/gadget/f_dfu.c;h=d7ae0...
"Proprietary extension: 'detach' from idle mode and get back to runtime mode in case of USB Reset. As much as I dislike this, we just can't use every USB bus reset to switch back to runtime mode, since at least the Linux USB stack likes to send a number of resets in a row :( "
The hack/abuse of sending a DFU_DETACH when in DFU mode is not part of the DFU standard, but dfu-util supports it since openmoko was for long the major target for dfu-util.
If you don't need to switch back from DFU mode to runtime mode, maybe you can use this to reset u-boot?
Regards, Tormod
Hello Lukasz,
Am 17.07.2013 16:34, schrieb Lukasz Majewski:
On Wed, 17 Jul 2013 12:26:35 +0200 Heiko Schocher hs@denx.de wrote,
Hi Heiko,
Hello Lukasz,
Am 16.07.2013 17:35, schrieb Lukasz Majewski:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Full Ack!
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU on several devices of the same type. Dev_part - number of partitions on the device.
Should this be "number of the partition on the device"
No. I made a mistake. It should be "partition to which we want to write"
You mean here the mtd partition for storing, right?
Yes, number of the partition to store data. The exact address will be extracted from mtdparts or partitions env variable.
Ok.
FS - information about file system on the device (fat, ext2/ext4, ubi). start - start offset from the beginning of the Device (byte or LBA number addressed) size - maximal number of blocks to be stored on the Device (expressed with Bytes of LBA number) (protection from overwriting the whole device)
Example:
Maybe dummy questions ...
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" |
Is this enough information? Where to store the uImage file on the ext4 partition?
To store uImage file on ext4/fat/ext2 partition it is enough to only give:
uImage mmc 0 2,
which maps to the following command:
ext4write mmc 0:2 0x44000000 /uImage [size]
Hmm... and what if I have my uImage in /boot/ ?
The file size is taken from number of sent bytes via DFU/USB.
Yes.
With writing to file system, one need to first store the whole transmitted data to a buffer and then store the (big) buffer on the SD/eMMC device.
Ok, same for an ubi image.
Since, we aren't supporting "seek" kind of write to current ext4 implementation at u-boot, the "big" buffer must be used.
Ok, there are places to optimize ;-)
root.img | img | mmc | 0 | 5 | "-" | "-" | "-" |
img means here: getting an image and storing it on the mtd partition "Dev_part" if start and size are marked with "-", beginning on start of the partition?
No, here img is for example a rootfs image. When storing it on the eMMC, it would be better to specify destination partition.
Wouldn't it be better to use here mtd partition names instead numbers for "Dev_part"?
I'd prefer to create a new entrance here (Part_name):
NAME | Type | Device | Dev_num | Dev_part | Part_name | FS | start | size |
I would like to avoid situation with ambiguous fields. One field shall only serve for one (clear) purpose. When not needed/used - field shall be marked as "-".
Ok. but this gives long constructs ...
What if "start" and "size" are filled with values for the "Type" "img"? Or is this forbidden for the "Type" "img"?
I think, that each "Type" shall have predefined set of allowed attributes:
- img: "Device" + "Dev num" + "Dev parts"
- raw: "Device" + "Dev num" + "start" + "size"
- file: "Device" + "Dev num" + "Dev parts" + "FS"
and so on.
If we introduce a "Part_name" this should be also possible:
4. img: "Device" + "Dev num" + "Part_name" 5. file: "Device" + "Dev num" + "Part_name" + "FS"
or?
root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
s/uImage/rootfs.img ? s/file/img ?
NAME: uImage and rootfs.img maps to files sent to board.
The "NAME" is used thereafter to provide exact name for file system write (ext4write/ fatwrite).
Ah! If we have to write in a subdirectory, "NAME" is the fullpath + filename?
With DFU the distinction between dfu entities (files, raw images, etc) is performed via alt settings (-aX switch at dfu-util).
For the FS "ubi" we need to specify, how to burn this into nand. I think we have no "ubi format" command.
Is already ubi format command available at u-boot?
No.
_As a side note:_
I'm now developing proof-of-concept idea of executing set of commands sent to u-boot by dfu-util [***].
Ok.
Rationale for this is the lack of ability to reset u-boot after sending data to u-boot via DFU. dfu-util has -R option, but this seems to reset usb interface, not the target device.
That answers my next question which comed in my mind ...
I will set an env:
setenv dfu "dfu mmc 0;${dfu_commands}"
Then at dfu itself, I will read commands to be performed. Then I will set $dfu_commands env and exit from dfu command.
Hmm... is this not oversized for the "reset board" use case?
With "ubi write ..." we can only write ubi volumes ... and we want here to burn an ubi image, which was created with ubinize and contains one or more ubi volumes
Maybe usecase for updating ubi volumes: ubivolumename | img | nand | 0 | 3 | ubivol | "-" | "-" |
If you want to update an ubivolume ...
results in this steps:
- get image per dfu @ img_addr We need to get here the complete image, as ubi write cannot write incremental ...
So each volume shall have different dfu entity. Then those volumes are stored at ram with different addresses [*]?
Yes for every volume have different dfu entity. You update only one volume with one dfu-util command, or? So no different addresses needed ...
- ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset ^ From where get we
this parameter ... value in "start"?
So here you get information about the ("Dev_part=3") partition start offset [**]?
Yes ... what is with the "vid_header_offset" parameter?
- ubi write img_addr ubivolumename ${filesize}
Here you combine volumes sent at [*] and store it at correct offset [**]?
Yes! But just one volume at once ...
I'm not an expert about ubi, so please be patient with my questions :-).
No problem! I also just learning ;-)
The above steps are just to get the idea, what must be done for updating an ubi image, as this were the steps if you type in commands.
For me it looks like a mixture of ordinary u-boot commands with dfu transfer[s] needed to perform correct ubi image write. Maybe [***] will help you?
If we have the dfu_buf address availiable ... maybe it is worth a try ... thinking of sending per dfu a "run upd_ubi" and the "upd_ubi" command do the complete update ... but, if we update more than one image ... this will not work.
bye, Heiko
On Thu, 18 Jul 2013 07:36:40 +0200 Heiko Schocher hs@denx.de wrote,
Hi Heiko,
Hello Lukasz,
Am 17.07.2013 16:34, schrieb Lukasz Majewski:
On Wed, 17 Jul 2013 12:26:35 +0200 Heiko Schocher hs@denx.de wrote,
Hi Heiko,
Hello Lukasz,
Am 16.07.2013 17:35, schrieb Lukasz Majewski:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Full Ack!
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU on several devices of the same type. Dev_part - number of partitions on the device.
Should this be "number of the partition on the device"
No. I made a mistake. It should be "partition to which we want to write"
You mean here the mtd partition for storing, right?
Yes, number of the partition to store data. The exact address will be extracted from mtdparts or partitions env variable.
Ok.
FS - information about file system on the device (fat, ext2/ext4, ubi). start - start offset from the beginning of the Device (byte or LBA number addressed) size - maximal number of blocks to be stored on the Device (expressed with Bytes of LBA number) (protection from overwriting the whole device)
Example:
Maybe dummy questions ...
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" |
Is this enough information? Where to store the uImage file on the ext4 partition?
To store uImage file on ext4/fat/ext2 partition it is enough to only give:
uImage mmc 0 2,
which maps to the following command:
ext4write mmc 0:2 0x44000000 /uImage [size]
Hmm... and what if I have my uImage in /boot/ ?
Then you need to define NAME in a following way: /boot/uImage.
I'm also aware, that this is NOT an elegant solution.
The file size is taken from number of sent bytes via DFU/USB.
Yes.
With writing to file system, one need to first store the whole transmitted data to a buffer and then store the (big) buffer on the SD/eMMC device.
Ok, same for an ubi image.
Since, we aren't supporting "seek" kind of write to current ext4 implementation at u-boot, the "big" buffer must be used.
Ok, there are places to optimize ;-)
Frankly, handling those large buffers is cumbersome. I've observed that our current sdhci implementation has some issues with handling such large transfers.
root.img | img | mmc | 0 | 5 | "-" | "-" | "-" |
img means here: getting an image and storing it on the mtd partition "Dev_part" if start and size are marked with "-", beginning on start of the partition?
No, here img is for example a rootfs image. When storing it on the eMMC, it would be better to specify destination partition.
Wouldn't it be better to use here mtd partition names instead numbers for "Dev_part"?
I'd prefer to create a new entrance here (Part_name):
NAME | Type | Device | Dev_num | Dev_part | Part_name | FS | start | size |
I would like to avoid situation with ambiguous fields. One field shall only serve for one (clear) purpose. When not needed/used - field shall be marked as "-".
Ok. but this gives long constructs ...
I know, but we also want to clean up it once for a long time. I'm very open for other ideas :-).
What if "start" and "size" are filled with values for the "Type" "img"? Or is this forbidden for the "Type" "img"?
I think, that each "Type" shall have predefined set of allowed attributes:
- img: "Device" + "Dev num" + "Dev parts"
- raw: "Device" + "Dev num" + "start" + "size"
- file: "Device" + "Dev num" + "Dev parts" + "FS"
and so on.
If we introduce a "Part_name" this should be also possible:
- img: "Device" + "Dev num" + "Part_name"
- file: "Device" + "Dev num" + "Part_name" + "FS"
or?
Yes. This also looks valid.
root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
s/uImage/rootfs.img ? s/file/img ?
NAME: uImage and rootfs.img maps to files sent to board.
The "NAME" is used thereafter to provide exact name for file system write (ext4write/ fatwrite).
Ah! If we have to write in a subdirectory, "NAME" is the fullpath + filename?
As stated above, NAME needs to be extended by fullpath.
With DFU the distinction between dfu entities (files, raw images, etc) is performed via alt settings (-aX switch at dfu-util).
For the FS "ubi" we need to specify, how to burn this into nand. I think we have no "ubi format" command.
Is already ubi format command available at u-boot?
No.
_As a side note:_
I'm now developing proof-of-concept idea of executing set of commands sent to u-boot by dfu-util [***].
Ok.
Rationale for this is the lack of ability to reset u-boot after sending data to u-boot via DFU. dfu-util has -R option, but this seems to reset usb interface, not the target device.
That answers my next question which comed in my mind ...
However, as Tormod pointed out we could perform reset at DFU_DEATACH internal DFU state.
I will set an env:
setenv dfu "dfu mmc 0;${dfu_commands}"
Then at dfu itself, I will read commands to be performed. Then I will set $dfu_commands env and exit from dfu command.
Hmm... is this not oversized for the "reset board" use case?
This is not only for reset :-). For me it seems possible to send a bunch of commands via DFU to be executed on by one.
$dfu_commanfs=mmc read 0x44000000 0:2;crc32 0x44000000 0x80;..... other...commands;reset
But having only reset implemented when dfu-util -R is executed is also a good idea.
With "ubi write ..." we can only write ubi volumes ... and we want here to burn an ubi image, which was created with ubinize and contains one or more ubi volumes
Maybe usecase for updating ubi volumes: ubivolumename | img | nand | 0 | 3 | ubivol | "-" | "-" |
If you want to update an ubivolume ...
results in this steps:
- get image per dfu @ img_addr We need to get here the complete image, as ubi write cannot write incremental ...
So each volume shall have different dfu entity. Then those volumes are stored at ram with different addresses [*]?
Yes for every volume have different dfu entity. You update only one volume with one dfu-util command, or? So no different addresses needed ...
- ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset ^ From where get we
this parameter ... value in "start"?
So here you get information about the ("Dev_part=3") partition start offset [**]?
Yes ... what is with the "vid_header_offset" parameter?
- ubi write img_addr ubivolumename ${filesize}
Here you combine volumes sent at [*] and store it at correct offset [**]?
Yes! But just one volume at once ...
I'm not an expert about ubi, so please be patient with my questions :-).
No problem! I also just learning ;-)
The above steps are just to get the idea, what must be done for updating an ubi image, as this were the steps if you type in commands.
For me it looks like a mixture of ordinary u-boot commands with dfu transfer[s] needed to perform correct ubi image write. Maybe [***] will help you?
If we have the dfu_buf address availiable ... maybe it is worth a try ... thinking of sending per dfu a "run upd_ubi" and the "upd_ubi" command do the complete update ... but, if we update more than one image ... this will not work.
Ok.
bye, Heiko
Dear Heiko Schocher,
Hello Lukasz,
Am 16.07.2013 17:35, schrieb Lukasz Majewski:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Full Ack!
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU
on several devices of the same type.Dev_part - number of partitions on the device.
Should this be "number of the partition on the device"
You mean here the mtd partition for storing, right?
FS - information about file system on the device (fat,
ext2/ext4, ubi).start - start offset from the beginning of the Device (byte or LBA
number addressed)size - maximal number of blocks to be stored on the Device
(expressed with Bytes of LBA number) (protection from overwriting the whole device)Example:
Maybe dummy questions ...
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" |
Is this enough information? Where to store the uImage file on the ext4 partition?
root.img | img | mmc | 0 | 5 | "-" | "-" | "-" |
img means here: getting an image and storing it on the mtd partition "Dev_part" if start and size are marked with "-", beginning on start of the partition?
Wouldn't it be better to use here mtd partition names instead numbers for "Dev_part"?
What if "start" and "size" are filled with values for the "Type" "img"? Or is this forbidden for the "Type" "img"?
root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
s/uImage/rootfs.img ? s/file/img ?
For the FS "ubi" we need to specify, how to burn this into nand. I think we have no "ubi format" command.
Try "nand write.trimffs" to write UBI images produced with ubinize .
Best regards, Marek Vasut
Hello Marek,
Am 18.07.2013 06:17, schrieb Marek Vasut:
Dear Heiko Schocher,
Hello Lukasz,
Am 16.07.2013 17:35, schrieb Lukasz Majewski:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
Full Ack!
Proposed new format for single dfu entity:
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
where:
Name - name of the alt setting (as seen by dfu-util -l) Type - description of the image (raw, file, img, command [*]) Device - physical device on which data are stored (mmc, nand, "-") Dev_num - number of the device - it is possible to store data via DFU
on several devices of the same type.Dev_part - number of partitions on the device.
Should this be "number of the partition on the device"
You mean here the mtd partition for storing, right?
FS - information about file system on the device (fat,
ext2/ext4, ubi).start - start offset from the beginning of the Device (byte or LBA
number addressed)size - maximal number of blocks to be stored on the Device
(expressed with Bytes of LBA number) (protection from overwriting the whole device)Example:
Maybe dummy questions ...
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
u-boot | raw | mmc | 0 | "-" [**] | "-" | 0x80 | 0x100 | uImage | file | mmc | 0 | 2 | ext4 | "-" | "-" |
Is this enough information? Where to store the uImage file on the ext4 partition?
root.img | img | mmc | 0 | 5 | "-" | "-" | "-" |
img means here: getting an image and storing it on the mtd partition "Dev_part" if start and size are marked with "-", beginning on start of the partition?
Wouldn't it be better to use here mtd partition names instead numbers for "Dev_part"?
What if "start" and "size" are filled with values for the "Type" "img"? Or is this forbidden for the "Type" "img"?
root.img | raw | mmc | 0 | "-" | "-" | 0x1000| 0x4000|
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
s/uImage/rootfs.img ? s/file/img ?
For the FS "ubi" we need to specify, how to burn this into nand. I think we have no "ubi format" command.
Try "nand write.trimffs" to write UBI images produced with ubinize .
This solves not the erasecounter problem, or?
For UBI we need something like this: http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
But I am not an UBI expert. It is possible I overlook something obvious ...
bye, Heiko
Dear Heiko Schocher,
In message 51E77A1D.90403@denx.de you wrote:
Try "nand write.trimffs" to write UBI images produced with ubinize .
This solves not the erasecounter problem, or?
For UBI we need something like this: http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
But I am not an UBI expert. It is possible I overlook something obvious ...
No, you don't. Devices managed by UBI should never be erased by other, non-UBI-aware tools.
Best regards,
Wolfgang Denk
Hi,
Dear Heiko Schocher,
In message 51E77A1D.90403@denx.de you wrote:
Try "nand write.trimffs" to write UBI images produced with ubinize .
This solves not the erasecounter problem, or?
For UBI we need something like this: http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
But I am not an UBI expert. It is possible I overlook something obvious ...
No, you don't. Devices managed by UBI should never be erased by other, non-UBI-aware tools.
I based my reply on the following commit in U-Boot and the fact that write.trimffs is used to flash UBI images. Maybe I was wrong?
commit c9494866df835bcee68e17339aec1090faa704da Author: Ben Gardiner bengardiner@nanometrics.ca Date: Tue Jun 14 16:35:07 2011 -0400
cmd_nand: add nand write.trimffs command
Add another nand write. variant, trimffs. This command will request of nand_write_skip_bad() that all trailing all-0xff pages will be dropped from eraseblocks when they are written to flash as-per the reccommended behaviour of the UBI FAQ [1].
The function that implements this timming is the drop_ffs() function by Artem Bityutskiy, ported from the mtd-utils tree.
[1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
Best regards, Marek Vasut
Hello Marek,
Am 18.07.2013 17:10, schrieb Marek Vasut:
Hi,
Dear Heiko Schocher,
In message51E77A1D.90403@denx.de you wrote:
Try "nand write.trimffs" to write UBI images produced with ubinize .
This solves not the erasecounter problem, or?
For UBI we need something like this: http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
But I am not an UBI expert. It is possible I overlook something obvious ...
No, you don't. Devices managed by UBI should never be erased by other, non-UBI-aware tools.
I based my reply on the following commit in U-Boot and the fact that write.trimffs is used to flash UBI images. Maybe I was wrong?
commit c9494866df835bcee68e17339aec1090faa704da Author: Ben Gardinerbengardiner@nanometrics.ca Date: Tue Jun 14 16:35:07 2011 -0400
cmd_nand: add nand write.trimffs command Add another nand write. variant, trimffs. This command will request of nand_write_skip_bad() that all trailing all-0xff pages will be dropped from eraseblocks when they are written to flash as-per the reccommended behaviour of the UBI FAQ [1]. The function that implements this timming is the drop_ffs() function by Artem Bityutskiy, ported from the mtd-utils tree. [1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
Yes, that sounds as a step in the right direction, but where are the erasecounters handled, as described in [1] ?
And as this is a "ubi function" and not nand specific, the command should start with "ubi ..." ... as we have a "ubi write ...", but ubi write is only for ubi volumes ... i tend to say, we need a "ubi format ..." similiar to ubiformat in the mtd utils [2] ...
[2] http://git.infradead.org/mtd-utils.git ubiformat found in /ubi-utils/ubiformat.c
bye, Heiko
Dear Heiko Schocher,
Hello Marek,
Am 18.07.2013 17:10, schrieb Marek Vasut:
Hi,
Dear Heiko Schocher,
In message51E77A1D.90403@denx.de you wrote:
Try "nand write.trimffs" to write UBI images produced with ubinize .
This solves not the erasecounter problem, or?
For UBI we need something like this: http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
But I am not an UBI expert. It is possible I overlook something obvious ...
No, you don't. Devices managed by UBI should never be erased by other, non-UBI-aware tools.
I based my reply on the following commit in U-Boot and the fact that write.trimffs is used to flash UBI images. Maybe I was wrong?
commit c9494866df835bcee68e17339aec1090faa704da Author: Ben Gardinerbengardiner@nanometrics.ca Date: Tue Jun 14 16:35:07 2011 -0400
cmd_nand: add nand write.trimffs command Add another nand write. variant, trimffs. This command will request of nand_write_skip_bad() that all trailing all-0xff pages will be dropped from eraseblocks when they are written to flash as-per the reccommended behaviour of the UBI FAQ [1]. The function that implements this timming is the drop_ffs() function by Artem Bityutskiy, ported from the mtd-utils tree. [1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algoYes, that sounds as a step in the right direction, but where are the erasecounters handled, as described in [1] ?
I don't think they're handled anywhere.
And as this is a "ubi function" and not nand specific, the command should start with "ubi ..." ... as we have a "ubi write ...", but ubi write is only for ubi volumes ... i tend to say, we need a "ubi format ..." similiar to ubiformat in the mtd utils [2] ...
[2] http://git.infradead.org/mtd-utils.git ubiformat found in /ubi-utils/ubiformat.c
Full agreement.
Best regards, Marek Vasut
On Tue, Jul 16, 2013 at 05:35:21PM +0200, Lukasz Majewski wrote:
Dear All,
Since DFU usage at u-boot is spreading to different device types (MMC, NAND), file systems, raw partitions, ubi, etc, I think that it is a good moment to unify and structure the form of dfu_alt_info environment variable.
[snip]
NAME | Type | Device | Dev_num | Dev_part | FS | start | size |
[snip]
u-boot | raw | nand | 0 | "-" | "-" | 0x100 | 0x100 | uImage | file | nand | 0 | 3 | ubi | "-" | "-" |
We also need to cover:
uImage | raw | nand | 0 | "kernel" | "-" | "-" | "-" |
Since partitions provide start/size. And I'm not totally sure we're providing enough information for 'destination resides in this part of UBI container'. It could be "/boot/uImage in rootfs in ubi0" or "raw to kernel0 in ubi0".
On Jul 18, 2013, at 12:39 PM, Tom Rini trini@ti.com wrote:
uImage | raw | nand | 0 | "kernel" | "-" | "-" | "-" |
Since partitions provide start/size.
I've got some WIP that pulls the alt info from a GPT partition map on mmc. That alt settings in DFU can be strings or numbers was handy to allow the name to identify the partition.
-Mike
On Thu, 18 Jul 2013 13:30:55 -0400 Michael Cashwell mboards@prograde.net wrote:
On Jul 18, 2013, at 12:39 PM, Tom Rini trini@ti.com wrote:
uImage | raw | nand | 0 | "kernel" | "-" | "-" | "-" |
Since partitions provide start/size.
I've got some WIP that pulls the alt info from a GPT partition map on mmc.
Extraction the part of dfu_alt_info from GPT or $partitions env variable (as defined at e.g. Trats) is also a good idea.
The best solution would be to produce alt settings information as a mix of GPT layout information (extraction of part num, start offset, size... any more?) [*] and some predefined dfu_alt_info env variable. In this way we can reuse already available code (since GPT layout is already in place for most of targets).
But what about mtd parts (Nand/ UBI)? It shall be also doable to extract information [*] from it. Am I correct?
That alt settings in DFU can be strings or numbers was handy to allow the name to identify the partition.
-Mike
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 07/18/2013 04:17 PM, Lukasz Majewski wrote:
On Thu, 18 Jul 2013 13:30:55 -0400 Michael Cashwell mboards@prograde.net wrote:
On Jul 18, 2013, at 12:39 PM, Tom Rini trini@ti.com wrote:
uImage | raw | nand | 0 | "kernel" | "-" | "-" | "-" |
Since partitions provide start/size.
I've got some WIP that pulls the alt info from a GPT partition map on mmc.
Extraction the part of dfu_alt_info from GPT or $partitions env variable (as defined at e.g. Trats) is also a good idea.
The best solution would be to produce alt settings information as a mix of GPT layout information (extraction of part num, start offset, size... any more?) [*] and some predefined dfu_alt_info env variable. In this way we can reuse already available code (since GPT layout is already in place for most of targets).
But what about mtd parts (Nand/ UBI)? It shall be also doable to extract information [*] from it. Am I correct?
Right, we support that today even in DFU.
- -- Tom
On Thu, 18 Jul 2013 13:30:55 -0400 Michael Cashwell mboards@prograde.net wrote,
On Jul 18, 2013, at 12:39 PM, Tom Rini trini@ti.com wrote:
uImage | raw | nand | 0 | "kernel" | "-" | "-" | "-" |
Since partitions provide start/size.
Hi Michael,
I've got some WIP that pulls the alt info from a GPT partition map on mmc. That alt settings in DFU can be strings or numbers was handy to allow the name to identify the partition.
Will you be ready to show the mentioned patch in a near future?
-Mike
Dear All,
As a follow up from u-boot's mini-summit at ELCE 2013, I would like to once again share the new format of dfu_alt_info env variable (for those who couldn't attend).
dfu_alt_info_cleanup:
|---+-------------+-------------+---------+----------------------| | | <dev_name>: | <alt_name>, | <type> | extra params | |---+-------------+-------------+---------+----------------------| | 1 | mmc0: | rootfs, | part; | - | | 2 | mmc0: | uImage, | file, | part, dir; | | 3 | nand0: | u-boot.bin, | raw, | start, size; | | 4 | mmc0: | u-boot.bin, | raw, | start_lba, size_lba; | | 5 | ram0: | uImage, | ram, | start, size; | | 6 | ubi0: | kernel0, | raw; | - | |---+-------------+-------------+---------+----------------------| | 7 | cmd: | cmd, | cmd; | - |
Some useful info:
+ Each <type> is responsible for providing information about additional parameters (in conjunction with <dev_name>)
+ Ad 1. Extra parameters are extracted from GPT/MBR/MTD
+ Part can be given as text or number (text -> number conversion will be performed anyway)
+ Part: Ext2/4, FAT, UBI?
I'm especially interested in feedback from AM335x owners. I would like to know how UBI partitions are handled there.
Dear Lukasz,
In message 20131031182541.3a3020c6@amdc308.digital.local you wrote:
As a follow up from u-boot's mini-summit at ELCE 2013, I would like to once again share the new format of dfu_alt_info env variable (for those who couldn't attend).
dfu_alt_info_cleanup:
|---+-------------+-------------+---------+----------------------| | | <dev_name>: | <alt_name>, | <type> | extra params | |---+-------------+-------------+---------+----------------------| | 1 | mmc0: | rootfs, | part; | - | | 2 | mmc0: | uImage, | file, | part, dir; | | 3 | nand0: | u-boot.bin, | raw, | start, size; | | 4 | mmc0: | u-boot.bin, | raw, | start_lba, size_lba; | | 5 | ram0: | uImage, | ram, | start, size; | | 6 | ubi0: | kernel0, | raw; | - | |---+-------------+-------------+---------+----------------------| | 7 | cmd: | cmd, | cmd; | - |
Why can we not have a common format for all media supporting the "raw" type? I mean, why can we not also use "start, size" for mmc? It should be trivial to convert into a LBA number.
Best regards,
Wolfgang Denk
Hi Wolfgang,
Dear Lukasz,
In message 20131031182541.3a3020c6@amdc308.digital.local you wrote:
As a follow up from u-boot's mini-summit at ELCE 2013, I would like to once again share the new format of dfu_alt_info env variable (for those who couldn't attend).
dfu_alt_info_cleanup:
|---+-------------+-------------+---------+----------------------| | | <dev_name>: | <alt_name>, | <type> | extraparams | |---+-------------+-------------+---------+----------------------| | 1 | mmc0: | rootfs, | part; | - | | 2 | mmc0: | uImage, | file, | part, dir; | | 3 | nand0: | u-boot.bin, | raw, | start, size; | | 4 | mmc0: | u-boot.bin, | raw, | start_lba, size_lba; | | 5 | ram0: | uImage, | ram, | start, size; | | 6 | ubi0: | kernel0, | raw; | - | |---+-------------+-------------+---------+----------------------| | 7 | cmd: | cmd, | cmd; | - |
Why can we not have a common format for all media supporting the "raw" type? I mean, why can we not also use "start, size" for mmc? It should be trivial to convert into a LBA number.
I think, that I could do better with this table.
What I have meant, is that the "raw" extra parameters are the same in both cases. The interpretation of start and size (if they are LBA or "normal" addresses) depends on the <dev_name>.
Best regards,
Wolfgang Denk
Best regards, Lukasz Majewski
Dear Lukasz,
In message 20131031222010.082d453f@jawa you wrote:
Why can we not have a common format for all media supporting the "raw" type? I mean, why can we not also use "start, size" for mmc? It should be trivial to convert into a LBA number.
I think, that I could do better with this table.
What I have meant, is that the "raw" extra parameters are the same in both cases. The interpretation of start and size (if they are LBA or "normal" addresses) depends on the <dev_name>.
I think they should _not_ be interpreted differently. I would expect that "start" is an offset in bytes from the beginning of the device, for all of the cases. Similar, "size" would be the number of bytes, for all cases - bot bytes here and sectors there.
Best regards,
Wolfgang Denk
Hi Wolfgang,
Dear Lukasz,
In message 20131031222010.082d453f@jawa you wrote:
Why can we not have a common format for all media supporting the "raw" type? I mean, why can we not also use "start, size" for mmc? It should be trivial to convert into a LBA number.
I think, that I could do better with this table.
What I have meant, is that the "raw" extra parameters are the same in both cases. The interpretation of start and size (if they are LBA or "normal" addresses) depends on the <dev_name>.
I think they should _not_ be interpreted differently. I would expect that "start" is an offset in bytes from the beginning of the device, for all of the cases. Similar, "size" would be the number of bytes, for all cases - bot bytes here and sectors there.
Ok, lets do it in this way. LBA conversion will be done just before eMMC access.
Best regards,
Wolfgang Denk
Hello Lukasz,
Am 31.10.2013 18:25, schrieb Lukasz Majewski:
Dear All,
As a follow up from u-boot's mini-summit at ELCE 2013, I would like to once again share the new format of dfu_alt_info env variable (for those who couldn't attend).
dfu_alt_info_cleanup:
|---+-------------+-------------+---------+----------------------| | |<dev_name>: |<alt_name>, |<type> | extra params | |---+-------------+-------------+---------+----------------------| | 1 | mmc0: | rootfs, | part; | - | | 2 | mmc0: | uImage, | file, | part, dir; | | 3 | nand0: | u-boot.bin, | raw, | start, size; |
One more usecase for nand (just for documentation):
| 4 | nand0: | u-boot | part, | - |
Here is "alt_name" == "mtd partition name" -> start, size from mtd info
| 4 | mmc0: | u-boot.bin, | raw, | start_lba, size_lba; | | 5 | ram0: | uImage, | ram, | start, size; | | 6 | ubi0: | kernel0, | raw; | - |
There are 2 usecases with ubi (Please correct me, if I am wrong):
a) update a complete ubi device:
| 6 | ubi0: | kernel, | part | - | | 7 | ubi0: | kernel, | raw | start, size |
with them, we can update a complete "ubi device" with n ubi volumes in it (This is currently used on the am335x based siemens boards).
I am not sure if ubi is more a "device" or more a "type". But as ubi can be used also on nor flash, I think in the meantime, it is more a device than a type in the sight of dfu ... but I vary here in my opinion...
This usecase is currently implemented for nand, and same code is used as for "raw" or "part" nand usecases, just in the ubi case a flag is set, which does:
- erase nand sectors with WITH_DROP_FFS - erase spare sectors at the end of the area
b) update single ubi volume.
We need this infos for updating a ubi volume
- ubi device name = mtd partition name - vid header offset - ubi volume name - ubi volume size - ubi volume type
- steps for updating a ubi volume - attach to an ubi device "ubi part device_name vid_header_offset" - if volume_name !exist, create it "ubi create volume_name size type" - write ubi volume "ubi write img_addr volume_name size"
| 8 | ubi0: | data, | volume | vid_off, vol_nam, vol_sz, vol_type | ^ device name You see, the list gets long ... or maybe we can cover this usecase with dfu command?
bye, Heiko
participants (8)
-
Heiko Schocher -
Lukasz Majewski -
Lukasz Majewski -
Marek Vasut -
Michael Cashwell -
Tom Rini -
Tormod Volden -
Wolfgang Denk