Proposal: U-Boot memory management
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
1. malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
2. lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
3. EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
4. reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
1. EFI should attach most of its data structures to driver model. This work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
2. Some drivers do EFI reservations today, whether EFI is used for booting or not (e.g. rockchip video rk_vop_probe()).
3. U-Boot doesn't really map arch-specific memory attributes (e.g. armv8's struct mm_region) to EFI ones.
4. EFI duplicates some code from bootm, some of which relates to memory allocation (e.g. FDT fixup).
5. EFI code is used even if EFI is never used to boot
6. EFI allocations can result in the same memory being used as has already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
Lifetime --------
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot: - malloc() space lives forever - lmb lives while setting out images for booting - EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
1. For video, use the driver model API to locate the video regions, or block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to the EFI_MEMORY_... attributes in struct efi_mem_desc.
3. Add all EFI reservations just before booting the app, as we do with devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
4. Avoid setting up EFI memory at the start of U-Boot. Do it only when booting. This looks to require very little effort.
5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
6. Don't worry too much about whether EFI will be used for booting. The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
–
Regards, Simon
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
* Only a minimum initialization of the EFI sub-system happens in efi_init_early(). * Some EFI code is called when probing block devices because we wanted the EFI and the dm part to be integrated. * The rest of the initialization in efi_init_obj_list() is only invoked if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Best regards
Heinrich
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
Regards, Simon
Heinrich
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
Best regards
Heinrich
Hi Heinrich,
On Mon, 18 Dec 2023 at 13:00, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
I see that more as a reservation than an allocation. The 'load' command can never do a useful memory allocation, since the user may want to load a different file to the same address.
We don't 'free' memory reserved for the kernel/ramdisk...the allocation is only temporary. This is what we need to align on. Those allocations should be collated and reported to EFI before booting, then dropped if the boot fails. This in fact works today, with lmb, so we should not need to change it.
Trying to bring in alloc/free semantics for lmb seems unnecessary and confusing. Let's keep in mind the problem we are trying to solve. With bootstd we can set up an lmb and place everything that is needed (in the read_all() method), then boot.
Regards, Simon
Am 18. Dezember 2023 22:03:41 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Mon, 18 Dec 2023 at 13:00, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
I see that more as a reservation than an allocation. The 'load' command can never do a useful memory allocation, since the user may want to load a different file to the same address.
We lack an unload command to free the memory.
We don't 'free' memory reserved for the kernel/ramdisk...the allocation is only temporary. This is what we need to align on. Those allocations should be collated and reported to EFI before booting, then dropped if the boot fails. This in fact works today, with lmb, so we should not need to change it.
Tempory allocations must use a free. This is trivial to implement.
Trying to bring in alloc/free semantics for lmb seems unnecessary and confusing. Let's keep in mind the problem we are trying to solve. With bootstd we can set up an lmb and place everything that is needed (in the read_all() method), then boot.
The problem we want to solve is that we don't have a proper memory managenent.
The current lmb usage is obviously not memory management and must be replaced by alloc/free semantics.
Best regards
Heinrich
Regards, Simon
Hi Heinrich,
On Mon, 18 Dec 2023 at 14:37, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 22:03:41 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Mon, 18 Dec 2023 at 13:00, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
I see that more as a reservation than an allocation. The 'load' command can never do a useful memory allocation, since the user may want to load a different file to the same address.
We lack an unload command to free the memory.
We don't 'free' memory reserved for the kernel/ramdisk...the allocation is only temporary. This is what we need to align on. Those allocations should be collated and reported to EFI before booting, then dropped if the boot fails. This in fact works today, with lmb, so we should not need to change it.
Tempory allocations must use a free. This is trivial to implement.
I 100% disagree, sorry. We don't want to deal with memory-fragmentation issues, etc. This is not an aloc()/free() situation. It is an 'image layout' problem.
We need to align on this point before we can make any progress.
Trying to bring in alloc/free semantics for lmb seems unnecessary and confusing. Let's keep in mind the problem we are trying to solve. With bootstd we can set up an lmb and place everything that is needed (in the read_all() method), then boot.
The problem we want to solve is that we don't have a proper memory managenent.
The current lmb usage is obviously not memory management and must be replaced by alloc/free semantics.
As above. That statement is completely wrong from my POV, sorry.
Regards, Simon
On Mon, Dec 18, 2023 at 02:48:43PM -0700, Simon Glass wrote:
Hi Heinrich,
On Mon, 18 Dec 2023 at 14:37, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 22:03:41 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Mon, 18 Dec 2023 at 13:00, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote: > Hi, > > This records my thoughts after a discussion with Ilias & Heinrich re > memory allocation in U-Boot. > > 1. malloc() > > malloc() is used for programmatic memory allocation. It allows memory > to be freed. It is not designed for very large allocations (e.g. a > 10MB kernel or 100MB ramdisk). > > 2. lmb > > lmb is used for large blocks of memory, such as those needed for a > kernel or ramdisk. Allocation is only transitory, for the purposes of > loading some images and booting. If the boot fails, then all lmb > allocations go away. > > lmb is set up by getting all available memory and then removing what > is used by U-Boot (code, data, malloc() space, etc.) > > lmb reservations have a few flags so that areas of memory can be > provided with attributes > > There are some corner cases...e.g. loading a file does an lmb > allocation but only for the purpose of avoiding a file being loaded > over U-Boot code/data. The allocation is dropped immediately after the > file is loaded. Within the bootm command, or when using standard boot, > this would be fairly easy to solve. > > Linux has renamed lmb to memblock. We should consider doing the same. > > 3. EFI > > EFI has its own memory-allocation tables. > > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > function it can also do smaller allocations similar to malloc(), > although each one uses at least 4KB at present. > > EFI allocations do not go away when a boot fails. > > With EFI it is possible to add allocations post facto, in which case > they are added to the allocation table just as if the memory was > allocated with EFI to begin with. > > The EFI allocations and the lmb allocations use the same memory, so in > principle could conflict. > > EFI allocations are sometimes used to allocate internal U-Boot data as > well, if needed by the EFI app. For example, while efi_image_parse() > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > in the app context and may need to access the memory after U-Boot has > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > mapping as well. > > EFI memory has attributes, including what the memory is used for (to > some degree of granularity). See enum efi_memory_type and struct > efi_mem_desc. In the latter there are also attribute flags - whether > memory is cacheable, etc. > > EFI also has the x86 idea of 'conventional' memory, meaning (I > believe) that below 4GB that isn't reserved for the hardware/system. > This is meaningless, or at least confusing, on ARM systems. > > 4. reservations > > It is perhaps worth mentioning a fourth method of memory management, > where U-Boot reserves chunks of memory before relocation (in > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > region, etc. > > > Problems > —------- > > There are no urgent problems, but here are some things that could be improved: > > 1. EFI should attach most of its data structures to driver model. This > work has started, with the partition support, but more effort would > help. This would make it easier to see which memory is related to > devices and which is separate. > > 2. Some drivers do EFI reservations today, whether EFI is used for > booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
> > 4. EFI duplicates some code from bootm, some of which relates to > memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
> > 5. EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
> > 6. EFI allocations can result in the same memory being used as has > already been allocated by lmb. Users may load files which overwrite > memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
> > > Lifetime > -------- > > We have three different memory allocators with different purposes. Can > we unify them a little? > > Within U-Boot: > - malloc() space lives forever > - lmb lives while setting out images for booting > - EFI (mostly) lives while booting an EFI app > > In practice, EFI is set up early in U-Boot. Some of this is necessary, > some not. EFI allocations stay around forever. This works OK since > large allocations are normally not done in EFI, so memory isn't really > consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
> > What happens to EFI allocations if the app returns? They are still > present, in case another app is run. This seems fine. > > API > –-- > Can we unify some APIs? > > It should be possible to use lmb for large EFI memory allocations, so > long as they are only needed for booting. We effectively do this > today, since EFI does not manage the arrangement of loaded images in > memory. for the most part. > > It would not make sense to use EFI allocation to replace lmb and > malloc(), of course. > > Could we use a common (lower-level) API for allocation, used by both > lmb and EFI? They do have some similarities. However they have > different lifetime constraints (EFI allocations are never dropped, > unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
> > ** Overall, it seems that the existence of memory allocation in > boot-time services has created confusion. Memory allocation is > muddled, with both U-Boot code and boot-time services calling the same > memory allocator. This just has not been clearly thought out. >
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
> > Proposal > —------- > > Here are some ideas: > > 1. For video, use the driver model API to locate the video regions, or > block off the entire framebuffer memory, for all devices as a whole. > Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > 3. Add all EFI reservations just before booting the app, as we do with > devicetree fixup. With this model, malloc() and lmb are used for all > allocation. Then efi_add_memory_map() is called for each region in > turn just before booting. Memory attributes are dealt with above. The > type (enum efi_memory_type) can be determined simply by the data > structure stored in it, as is done today. For example, SMBIOS tables > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > boot-time services. This solves the problem 6. If memory is needed by > an app, allocate it with malloc() and see 3. There are only two > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > efi_allocate_pool(), but most of these seem easy to fix up. For > example, efi_init_event_log() allocates a buffer, but this can be > allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
> > 6. Don't worry too much about whether EFI will be used for booting. > The cost is likely not that great: use bootstage to measure it as is > done for driver model. Try to minmise the cost of its tables, > particularly for execution time, but otherwise just rely on the > ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
I see that more as a reservation than an allocation. The 'load' command can never do a useful memory allocation, since the user may want to load a different file to the same address.
We lack an unload command to free the memory.
We don't 'free' memory reserved for the kernel/ramdisk...the allocation is only temporary. This is what we need to align on. Those allocations should be collated and reported to EFI before booting, then dropped if the boot fails. This in fact works today, with lmb, so we should not need to change it.
Tempory allocations must use a free. This is trivial to implement.
I 100% disagree, sorry. We don't want to deal with memory-fragmentation issues, etc. This is not an aloc()/free() situation. It is an 'image layout' problem.
We need to align on this point before we can make any progress.
We may not need the strictness of malloc/free in all cases, but we do conceptually need to know where things have been shoved in memory that one might expect to need later. We may not need to allocate say 10MB of space up front for a kernel image, but we do need to note that for later. And we need to consider how good/bad the existing lmb logic is for "you might be able to overwrite U-Boot, stop!".
Am 18. Dezember 2023 22:48:43 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Mon, 18 Dec 2023 at 14:37, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 22:03:41 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Mon, 18 Dec 2023 at 13:00, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote: > Hi, > > This records my thoughts after a discussion with Ilias & Heinrich re > memory allocation in U-Boot. > > 1. malloc() > > malloc() is used for programmatic memory allocation. It allows memory > to be freed. It is not designed for very large allocations (e.g. a > 10MB kernel or 100MB ramdisk). > > 2. lmb > > lmb is used for large blocks of memory, such as those needed for a > kernel or ramdisk. Allocation is only transitory, for the purposes of > loading some images and booting. If the boot fails, then all lmb > allocations go away. > > lmb is set up by getting all available memory and then removing what > is used by U-Boot (code, data, malloc() space, etc.) > > lmb reservations have a few flags so that areas of memory can be > provided with attributes > > There are some corner cases...e.g. loading a file does an lmb > allocation but only for the purpose of avoiding a file being loaded > over U-Boot code/data. The allocation is dropped immediately after the > file is loaded. Within the bootm command, or when using standard boot, > this would be fairly easy to solve. > > Linux has renamed lmb to memblock. We should consider doing the same. > > 3. EFI > > EFI has its own memory-allocation tables. > > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > function it can also do smaller allocations similar to malloc(), > although each one uses at least 4KB at present. > > EFI allocations do not go away when a boot fails. > > With EFI it is possible to add allocations post facto, in which case > they are added to the allocation table just as if the memory was > allocated with EFI to begin with. > > The EFI allocations and the lmb allocations use the same memory, so in > principle could conflict. > > EFI allocations are sometimes used to allocate internal U-Boot data as > well, if needed by the EFI app. For example, while efi_image_parse() > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > in the app context and may need to access the memory after U-Boot has > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > mapping as well. > > EFI memory has attributes, including what the memory is used for (to > some degree of granularity). See enum efi_memory_type and struct > efi_mem_desc. In the latter there are also attribute flags - whether > memory is cacheable, etc. > > EFI also has the x86 idea of 'conventional' memory, meaning (I > believe) that below 4GB that isn't reserved for the hardware/system. > This is meaningless, or at least confusing, on ARM systems. > > 4. reservations > > It is perhaps worth mentioning a fourth method of memory management, > where U-Boot reserves chunks of memory before relocation (in > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > region, etc. > > > Problems > —------- > > There are no urgent problems, but here are some things that could be improved: > > 1. EFI should attach most of its data structures to driver model. This > work has started, with the partition support, but more effort would > help. This would make it easier to see which memory is related to > devices and which is separate. > > 2. Some drivers do EFI reservations today, whether EFI is used for > booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
> > 4. EFI duplicates some code from bootm, some of which relates to > memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
> > 5. EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
> > 6. EFI allocations can result in the same memory being used as has > already been allocated by lmb. Users may load files which overwrite > memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
> > > Lifetime > -------- > > We have three different memory allocators with different purposes. Can > we unify them a little? > > Within U-Boot: > - malloc() space lives forever > - lmb lives while setting out images for booting > - EFI (mostly) lives while booting an EFI app > > In practice, EFI is set up early in U-Boot. Some of this is necessary, > some not. EFI allocations stay around forever. This works OK since > large allocations are normally not done in EFI, so memory isn't really > consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
> > What happens to EFI allocations if the app returns? They are still > present, in case another app is run. This seems fine. > > API > –-- > Can we unify some APIs? > > It should be possible to use lmb for large EFI memory allocations, so > long as they are only needed for booting. We effectively do this > today, since EFI does not manage the arrangement of loaded images in > memory. for the most part. > > It would not make sense to use EFI allocation to replace lmb and > malloc(), of course. > > Could we use a common (lower-level) API for allocation, used by both > lmb and EFI? They do have some similarities. However they have > different lifetime constraints (EFI allocations are never dropped, > unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
> > ** Overall, it seems that the existence of memory allocation in > boot-time services has created confusion. Memory allocation is > muddled, with both U-Boot code and boot-time services calling the same > memory allocator. This just has not been clearly thought out. >
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
> > Proposal > —------- > > Here are some ideas: > > 1. For video, use the driver model API to locate the video regions, or > block off the entire framebuffer memory, for all devices as a whole. > Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > 3. Add all EFI reservations just before booting the app, as we do with > devicetree fixup. With this model, malloc() and lmb are used for all > allocation. Then efi_add_memory_map() is called for each region in > turn just before booting. Memory attributes are dealt with above. The > type (enum efi_memory_type) can be determined simply by the data > structure stored in it, as is done today. For example, SMBIOS tables > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > boot-time services. This solves the problem 6. If memory is needed by > an app, allocate it with malloc() and see 3. There are only two > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > efi_allocate_pool(), but most of these seem easy to fix up. For > example, efi_init_event_log() allocates a buffer, but this can be > allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
> > 6. Don't worry too much about whether EFI will be used for booting. > The cost is likely not that great: use bootstage to measure it as is > done for driver model. Try to minmise the cost of its tables, > particularly for execution time, but otherwise just rely on the > ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
I see that more as a reservation than an allocation. The 'load' command can never do a useful memory allocation, since the user may want to load a different file to the same address.
We lack an unload command to free the memory.
We don't 'free' memory reserved for the kernel/ramdisk...the allocation is only temporary. This is what we need to align on. Those allocations should be collated and reported to EFI before booting, then dropped if the boot fails. This in fact works today, with lmb, so we should not need to change it.
Tempory allocations must use a free. This is trivial to implement.
I 100% disagree, sorry. We don't want to deal with memory-fragmentation issues, etc. This is not an aloc()/free() situation. It is an 'image layout' problem.
We need to align on this point before we can make any progress.
Trying to bring in alloc/free semantics for lmb seems unnecessary and confusing. Let's keep in mind the problem we are trying to solve. With bootstd we can set up an lmb and place everything that is needed (in the read_all() method), then boot.
The problem we want to solve is that we don't have a proper memory managenent.
The current lmb usage is obviously not memory management and must be replaced by alloc/free semantics.
As above. That statement is completely wrong from my POV, sorry.
I have absolutely no clue why throwing away allocations in lmb should be beneficial. The EFI subsystem needs to know which memory contains a loaded file or it might pass out that very same memory to EFI binaries.
Take the following user input:
load host 0:1 $a fdt load host 0:1 $b driver.efi bootefi $b load host 0:1 $c kernel.efi bootefi $c $a
We must ensure that memory at $a is not been used by driver.efi when calling AllocatePages(). This requires a memory allocation by the load command.
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
You immediately comprehend that our use of lmb is broken.
Beat regards
Heinrich
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip]
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com:
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip]
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Best regards
Heinrich
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote:
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com:
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip]
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote:
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com:
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip]
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
Furthermore the EFI sub-system would be aware of such memory allocations and respect them. This would of course also hold true for allocations for SMBIOS or ACPI tables.
Best regards
Heinrich
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote:
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com:
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip]
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote:
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com:
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip]
Or take:
load host 0:1 $c kernel.efi load host 0:1 $d initrd.img
How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
I don't want to replace the malloc library which handles large numbets of allocations.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
We will have to review each file load method individually.
Best regards
Heinrich
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote:
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com:
On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote:
[snip] > Or take: > > load host 0:1 $c kernel.efi > load host 0:1 $d initrd.img > > How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation?
Today, invalid checksum as part of some part of the kernel fails. But how do we do this tomorrow, are you suggesting that "load" perform malloc() in some predefined size? If $c is below $d and $c + kernel.efi is now above $d we can throw an error before trying to load, yes. But what about: load host 0:1 $d initrd.img load host 0:1 $c kernel.efi
In that case (which is only marginally contrived, the more real case is loading device tree in to unexpectedly large ramdisk because someone didn't understand the general advice on why device tree is lower than ramdisk address) I'm fine with an error that amounts to "you just corrupted another allocation" and then "fail, reset the board" or so.
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote:
Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: >On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: > >[snip] >> Or take: >> >> load host 0:1 $c kernel.efi >> load host 0:1 $d initrd.img >> >> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? > >Today, invalid checksum as part of some part of the kernel fails. But >how do we do this tomorrow, are you suggesting that "load" perform >malloc() in some predefined size? If $c is below $d and $c + kernel.efi >is now above $d we can throw an error before trying to load, yes. But >what about: >load host 0:1 $d initrd.img >load host 0:1 $c kernel.efi > >In that case (which is only marginally contrived, the more real case is >loading device tree in to unexpectedly large ramdisk because someone >didn't understand the general advice on why device tree is lower than >ramdisk address) I'm fine with an error that amounts to "you just >corrupted another allocation" and then "fail, reset the board" or so. >
Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient.
The load command should permanently allocate memory in that lmb+ library.
We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
If the user issues multiple load commands, he can overwrite previous files.
During boot command execution I guess the different allocations respect each other.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Best regards
Heinrich
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: > > > Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: > >On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: > > > >[snip] > >> Or take: > >> > >> load host 0:1 $c kernel.efi > >> load host 0:1 $d initrd.img > >> > >> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? > > > >Today, invalid checksum as part of some part of the kernel fails. But > >how do we do this tomorrow, are you suggesting that "load" perform > >malloc() in some predefined size? If $c is below $d and $c + kernel.efi > >is now above $d we can throw an error before trying to load, yes. But > >what about: > >load host 0:1 $d initrd.img > >load host 0:1 $c kernel.efi > > > >In that case (which is only marginally contrived, the more real case is > >loading device tree in to unexpectedly large ramdisk because someone > >didn't understand the general advice on why device tree is lower than > >ramdisk address) I'm fine with an error that amounts to "you just > >corrupted another allocation" and then "fail, reset the board" or so. > > > > Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. > > The load command should permanently allocate memory in that lmb+ library. > > We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused.
Our current way of loading things in to memory does not handle the case I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
During boot command execution I guess the different allocations respect each other.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: >On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: >> >> >> Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: >> >On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: >> > >> >[snip] >> >> Or take: >> >> >> >> load host 0:1 $c kernel.efi >> >> load host 0:1 $d initrd.img >> >> >> >> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? >> > >> >Today, invalid checksum as part of some part of the kernel fails. But >> >how do we do this tomorrow, are you suggesting that "load" perform >> >malloc() in some predefined size? If $c is below $d and $c + kernel.efi >> >is now above $d we can throw an error before trying to load, yes. But >> >what about: >> >load host 0:1 $d initrd.img >> >load host 0:1 $c kernel.efi >> > >> >In that case (which is only marginally contrived, the more real case is >> >loading device tree in to unexpectedly large ramdisk because someone >> >didn't understand the general advice on why device tree is lower than >> >ramdisk address) I'm fine with an error that amounts to "you just >> >corrupted another allocation" and then "fail, reset the board" or so. >> > >> >> Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. >> >> The load command should permanently allocate memory in that lmb+ library. >> >> We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. > >Our current way of loading things in to memory does not handle the case >I described, yes. How would what you're proposing handle it?
If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
During boot command execution I guess the different allocations respect each other.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
Regards, Simon
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote:
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: > > > Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: > >On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: > >> > >> > >> Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: > >> >On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: > >> > > >> >[snip] > >> >> Or take: > >> >> > >> >> load host 0:1 $c kernel.efi > >> >> load host 0:1 $d initrd.img > >> >> > >> >> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? > >> > > >> >Today, invalid checksum as part of some part of the kernel fails. But > >> >how do we do this tomorrow, are you suggesting that "load" perform > >> >malloc() in some predefined size? If $c is below $d and $c + kernel.efi > >> >is now above $d we can throw an error before trying to load, yes. But > >> >what about: > >> >load host 0:1 $d initrd.img > >> >load host 0:1 $c kernel.efi > >> > > >> >In that case (which is only marginally contrived, the more real case is > >> >loading device tree in to unexpectedly large ramdisk because someone > >> >didn't understand the general advice on why device tree is lower than > >> >ramdisk address) I'm fine with an error that amounts to "you just > >> >corrupted another allocation" and then "fail, reset the board" or so. > >> > > >> > >> Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. > >> > >> The load command should permanently allocate memory in that lmb+ library. > >> > >> We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. > > > >Our current way of loading things in to memory does not handle the case > >I described, yes. How would what you're proposing handle it? > > If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail.
So you're saying that the load command has to pre-allocate memory? Or as it goes? If the latter, in what size chunks? This starts to get at what Simon was talking about with respect to memory fragmentation. Which to be clear is a problem we have today, we just let things overlap and hope something later catches an incorrect checksum.
I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
During boot command execution I guess the different allocations respect each other.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
On 12/20/23 20:12, Tom Rini wrote:
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote:
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com: > On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: >> >> >> Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: >>> On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: >>>> >>>> >>>> Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: >>>>> On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: >>>>> >>>>> [snip] >>>>>> Or take: >>>>>> >>>>>> load host 0:1 $c kernel.efi >>>>>> load host 0:1 $d initrd.img >>>>>> >>>>>> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? >>>>> >>>>> Today, invalid checksum as part of some part of the kernel fails. But >>>>> how do we do this tomorrow, are you suggesting that "load" perform >>>>> malloc() in some predefined size? If $c is below $d and $c + kernel.efi >>>>> is now above $d we can throw an error before trying to load, yes. But >>>>> what about: >>>>> load host 0:1 $d initrd.img >>>>> load host 0:1 $c kernel.efi >>>>> >>>>> In that case (which is only marginally contrived, the more real case is >>>>> loading device tree in to unexpectedly large ramdisk because someone >>>>> didn't understand the general advice on why device tree is lower than >>>>> ramdisk address) I'm fine with an error that amounts to "you just >>>>> corrupted another allocation" and then "fail, reset the board" or so. >>>>> >>>> >>>> Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. >>>> >>>> The load command should permanently allocate memory in that lmb+ library. >>>> >>>> We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. >>> >>> Our current way of loading things in to memory does not handle the case >>> I described, yes. How would what you're proposing handle it? >> >> If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail. > > So you're saying that the load command has to pre-allocate memory? Or as > it goes? If the latter, in what size chunks? This starts to get at what > Simon was talking about with respect to memory fragmentation. Which to > be clear is a problem we have today, we just let things overlap and hope > something later catches an incorrect checksum. >
I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
When loading a file from a file system we know the filesize beforehand. So allocation is trivial.
The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
During boot command execution I guess the different allocations respect each other.
TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
The range managed by the EFI sub-system extends over all of RAM including the addresses used by load commands. Hence, "the range that covers U-Boot itself (malloc pool and so forth)" includes all memory.
Maybe we could pre-allocate a memory area for file loading. This just requires to define a device-specific high file memory address above which no file loads will be allowed and below which EFI, bootm, etc will not be allowed to allocate memory for further uses.
Best regards
Heinrich
On Fri, Dec 29, 2023 at 05:42:17PM +0100, Heinrich Schuchardt wrote:
On 12/20/23 20:12, Tom Rini wrote:
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote:
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com:
On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote: > > > Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com: > > On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: > > > > On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > [snip] > > > > > > > Or take: > > > > > > > > > > > > > > load host 0:1 $c kernel.efi > > > > > > > load host 0:1 $d initrd.img > > > > > > > > > > > > > > How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? > > > > > > > > > > > > Today, invalid checksum as part of some part of the kernel fails. But > > > > > > how do we do this tomorrow, are you suggesting that "load" perform > > > > > > malloc() in some predefined size? If $c is below $d and $c + kernel.efi > > > > > > is now above $d we can throw an error before trying to load, yes. But > > > > > > what about: > > > > > > load host 0:1 $d initrd.img > > > > > > load host 0:1 $c kernel.efi > > > > > > > > > > > > In that case (which is only marginally contrived, the more real case is > > > > > > loading device tree in to unexpectedly large ramdisk because someone > > > > > > didn't understand the general advice on why device tree is lower than > > > > > > ramdisk address) I'm fine with an error that amounts to "you just > > > > > > corrupted another allocation" and then "fail, reset the board" or so. > > > > > > > > > > > > > > > > Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. > > > > > > > > > > The load command should permanently allocate memory in that lmb+ library. > > > > > > > > > > We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. > > > > > > > > Our current way of loading things in to memory does not handle the case > > > > I described, yes. How would what you're proposing handle it? > > > > > > If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail. > > > > So you're saying that the load command has to pre-allocate memory? Or as > > it goes? If the latter, in what size chunks? This starts to get at what > > Simon was talking about with respect to memory fragmentation. Which to > > be clear is a problem we have today, we just let things overlap and hope > > something later catches an incorrect checksum. > > > > I don't want to replace the malloc library which handles large numbets of allocations.
I'm confused. The normal malloc library is not involved with current image loading, it's direct to memory (with some attempts at sanity checking by lmb). Are you proposing a different allocator with malloc/free like behavior? If so, please outline how it will determine pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
> Closing the eyes when the user loads multiple files does not solve the fragmentation problem.
Yes. I'm only noting that today we just ignore the problem and sometimes catch it via checksums.
> Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again.
OK, so are you saying that we would no longer be loading _to_ a location in memory and instead just be saying "load this thing" and picking where dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
> When loading a file from a file system we know the filesize beforehand. So allocation is trivial. > > The loady command currently does not use the offered size information but could do so.
We should be using that information to make sure we don't overwrite U-Boot itself, but I don't recall how exactly we handle it today off-hand.
If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
During boot command execution I guess the different allocations respect each other.
> TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file.
Reading from non-filesystem flash also has this problem, but we at least specify the amount to read too. But yes, it gets back to what I was asking about on how you're proposing to handle network load cases.
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
The range managed by the EFI sub-system extends over all of RAM including the addresses used by load commands. Hence, "the range that covers U-Boot itself (malloc pool and so forth)" includes all memory.
Maybe we could pre-allocate a memory area for file loading. This just requires to define a device-specific high file memory address above which no file loads will be allowed and below which EFI, bootm, etc will not be allowed to allocate memory for further uses.
We're not just / only an EFI runtime. When we load something persistent to memory outside of EFI, we need to note that, so that when EFI_LOADER beings work it can see this and do whatever it needs to do. And if/when EFI_LOADER returns control back to U-Boot itself, it needs to update that list with any new allocations that were done and mark them appropriately. The user gets full memory minus a bit, to do with as they need for their use case.
On 12/29/23 17:47, Tom Rini wrote:
On Fri, Dec 29, 2023 at 05:42:17PM +0100, Heinrich Schuchardt wrote:
On 12/20/23 20:12, Tom Rini wrote:
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote:
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote:
Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com: > On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote: >> >> >> Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com: >>> On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: >>>> >>>> >>>> Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: >>>>> On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: >>>>>> >>>>>> >>>>>> Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: >>>>>>> On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: >>>>>>> >>>>>>> [snip] >>>>>>>> Or take: >>>>>>>> >>>>>>>> load host 0:1 $c kernel.efi >>>>>>>> load host 0:1 $d initrd.img >>>>>>>> >>>>>>>> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? >>>>>>> >>>>>>> Today, invalid checksum as part of some part of the kernel fails. But >>>>>>> how do we do this tomorrow, are you suggesting that "load" perform >>>>>>> malloc() in some predefined size? If $c is below $d and $c + kernel.efi >>>>>>> is now above $d we can throw an error before trying to load, yes. But >>>>>>> what about: >>>>>>> load host 0:1 $d initrd.img >>>>>>> load host 0:1 $c kernel.efi >>>>>>> >>>>>>> In that case (which is only marginally contrived, the more real case is >>>>>>> loading device tree in to unexpectedly large ramdisk because someone >>>>>>> didn't understand the general advice on why device tree is lower than >>>>>>> ramdisk address) I'm fine with an error that amounts to "you just >>>>>>> corrupted another allocation" and then "fail, reset the board" or so. >>>>>>> >>>>>> >>>>>> Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. >>>>>> >>>>>> The load command should permanently allocate memory in that lmb+ library. >>>>>> >>>>>> We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. >>>>> >>>>> Our current way of loading things in to memory does not handle the case >>>>> I described, yes. How would what you're proposing handle it? >>>> >>>> If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail. >>> >>> So you're saying that the load command has to pre-allocate memory? Or as >>> it goes? If the latter, in what size chunks? This starts to get at what >>> Simon was talking about with respect to memory fragmentation. Which to >>> be clear is a problem we have today, we just let things overlap and hope >>> something later catches an incorrect checksum. >>> >> >> I don't want to replace the malloc library which handles large numbets of allocations. > > I'm confused. The normal malloc library is not involved with current > image loading, it's direct to memory (with some attempts at sanity > checking by lmb). Are you proposing a different allocator with > malloc/free like behavior? If so, please outline how it will determine > pool size, and how we'll use it to load thing to memory.
All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
>> Closing the eyes when the user loads multiple files does not solve the fragmentation problem. > > Yes. I'm only noting that today we just ignore the problem and sometimes > catch it via checksums. > >> Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again. > > OK, so are you saying that we would no longer be loading _to_ a location > in memory and instead just be saying "load this thing" and picking where > dynamically?
Both preassigned and allocator assigned adresses are compatible with memory management.
Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
>> When loading a file from a file system we know the filesize beforehand. So allocation is trivial. >> >> The loady command currently does not use the offered size information but could do so. > > We should be using that information to make sure we don't overwrite > U-Boot itself, but I don't recall how exactly we handle it today > off-hand.
If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
During boot command execution I guess the different allocations respect each other.
> >> TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file. > > Reading from non-filesystem flash also has this problem, but we at least > specify the amount to read too. But yes, it gets back to what I was > asking about on how you're proposing to handle network load cases. >
It depends on the protocol. Http conveys the size before the data. Tftp does not.
If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
The range managed by the EFI sub-system extends over all of RAM including the addresses used by load commands. Hence, "the range that covers U-Boot itself (malloc pool and so forth)" includes all memory.
Maybe we could pre-allocate a memory area for file loading. This just requires to define a device-specific high file memory address above which no file loads will be allowed and below which EFI, bootm, etc will not be allowed to allocate memory for further uses.
We're not just / only an EFI runtime. When we load something persistent to memory outside of EFI, we need to note that, so that when EFI_LOADER beings work it can see this and do whatever it needs to do. And if/when EFI_LOADER returns control back to U-Boot itself, it needs to update that list with any new allocations that were done and mark them appropriately.
The idea of EFI loader returning control is misleading.
You can load an EFI driver binary which provides a device to U-Boot. Whenever U-Boot accesses that driver allocations might occur.
I thought that we already agreed to have only one memory management and get rid of the LMB/EFI memory management duplication.
Best regards
Heinrich
The user gets full memory minus a bit, to do with as they need for their use case.
On Fri, Dec 29, 2023 at 06:09:44PM +0100, Heinrich Schuchardt wrote:
On 12/29/23 17:47, Tom Rini wrote:
On Fri, Dec 29, 2023 at 05:42:17PM +0100, Heinrich Schuchardt wrote:
On 12/20/23 20:12, Tom Rini wrote:
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote:
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote:
On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote: > > > Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com: > > On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com: > > > > On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > > > > Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > > > On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > [snip] > > > > > > > > > Or take: > > > > > > > > > > > > > > > > > > load host 0:1 $c kernel.efi > > > > > > > > > load host 0:1 $d initrd.img > > > > > > > > > > > > > > > > > > How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? > > > > > > > > > > > > > > > > Today, invalid checksum as part of some part of the kernel fails. But > > > > > > > > how do we do this tomorrow, are you suggesting that "load" perform > > > > > > > > malloc() in some predefined size? If $c is below $d and $c + kernel.efi > > > > > > > > is now above $d we can throw an error before trying to load, yes. But > > > > > > > > what about: > > > > > > > > load host 0:1 $d initrd.img > > > > > > > > load host 0:1 $c kernel.efi > > > > > > > > > > > > > > > > In that case (which is only marginally contrived, the more real case is > > > > > > > > loading device tree in to unexpectedly large ramdisk because someone > > > > > > > > didn't understand the general advice on why device tree is lower than > > > > > > > > ramdisk address) I'm fine with an error that amounts to "you just > > > > > > > > corrupted another allocation" and then "fail, reset the board" or so. > > > > > > > > > > > > > > > > > > > > > > Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. > > > > > > > > > > > > > > The load command should permanently allocate memory in that lmb+ library. > > > > > > > > > > > > > > We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. > > > > > > > > > > > > Our current way of loading things in to memory does not handle the case > > > > > > I described, yes. How would what you're proposing handle it? > > > > > > > > > > If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail. > > > > > > > > So you're saying that the load command has to pre-allocate memory? Or as > > > > it goes? If the latter, in what size chunks? This starts to get at what > > > > Simon was talking about with respect to memory fragmentation. Which to > > > > be clear is a problem we have today, we just let things overlap and hope > > > > something later catches an incorrect checksum. > > > > > > > > > > I don't want to replace the malloc library which handles large numbets of allocations. > > > > I'm confused. The normal malloc library is not involved with current > > image loading, it's direct to memory (with some attempts at sanity > > checking by lmb). Are you proposing a different allocator with > > malloc/free like behavior? If so, please outline how it will determine > > pool size, and how we'll use it to load thing to memory. > > All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack.
That's a rather huge change for how U-Boot works.
> > > Closing the eyes when the user loads multiple files does not solve the fragmentation problem. > > > > Yes. I'm only noting that today we just ignore the problem and sometimes > > catch it via checksums. > > > > > Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again. > > > > OK, so are you saying that we would no longer be loading _to_ a location > > in memory and instead just be saying "load this thing" and picking where > > dynamically? > > Both preassigned and allocator assigned adresses are compatible with memory management. > > Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions.
Yes, 64 bit architecture tend to only have alignment requirements while 32bit architectures have both alignment requirements and some memory window requirement. Whatever we implement here needs to handle both cases.
> > > When loading a file from a file system we know the filesize beforehand. So allocation is trivial. > > > > > > The loady command currently does not use the offered size information but could do so. > > > > We should be using that information to make sure we don't overwrite > > U-Boot itself, but I don't recall how exactly we handle it today > > off-hand. > > If the user issues multiple load commands, he can overwrite previous files.
Then it sounds like we lost one benefit of all of this overhead.
> During boot command execution I guess the different allocations respect each other. > > > > > > TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file. > > > > Reading from non-filesystem flash also has this problem, but we at least > > specify the amount to read too. But yes, it gets back to what I was > > asking about on how you're proposing to handle network load cases. > > > > It depends on the protocol. Http conveys the size before the data. Tftp does not. > > If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage.
Yes, and what I'm trying to find out is if what you're suggesting would do anything about it, since previous statements you made implied to me that we would prevent it.
To me, at this point it sounds like what we need is more like persistent memory blocks and a hook that can be called in to for both "give me all known memory blocks" and "add this memory block to the list", so that EFI can do whatever it needs to do upon starting an application and then upon return to U-Boot. Both malloc/free allocations and "load this blob to memory from whatever" allocations would call the appropriate hook for tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
The range managed by the EFI sub-system extends over all of RAM including the addresses used by load commands. Hence, "the range that covers U-Boot itself (malloc pool and so forth)" includes all memory.
Maybe we could pre-allocate a memory area for file loading. This just requires to define a device-specific high file memory address above which no file loads will be allowed and below which EFI, bootm, etc will not be allowed to allocate memory for further uses.
We're not just / only an EFI runtime. When we load something persistent to memory outside of EFI, we need to note that, so that when EFI_LOADER beings work it can see this and do whatever it needs to do. And if/when EFI_LOADER returns control back to U-Boot itself, it needs to update that list with any new allocations that were done and mark them appropriately.
The idea of EFI loader returning control is misleading.
You can load an EFI driver binary which provides a device to U-Boot. Whenever U-Boot accesses that driver allocations might occur.
We cannot, today, load an EFI driver and then make use of it outside of the context of having bootefi'd something, yes?
Or is this about that separate runtime area from another thread?
But even then, if something is making an allocation it needs to inform the rest of the world.
I thought that we already agreed to have only one memory management and get rid of the LMB/EFI memory management duplication.
We've agreed that today, CONFIG_LMB does not fit the use cases that we have in a modern system, for anyones usage. It has been suggested and I know it's somewhere on Ilias' TODO list, to try and take what we have in the EFI_LOADER & co area and make it usable and useful for all contexts. We also have what in some ways feels to me to be the inverse of that, which is Simon's proposal.
We have not agreed that the EFI model and restrictions are what everything will now be using. What's on Ilias' TODD list might bring us in that direction, and I want to see what that looks like in practice. But it's not agreed to as the end goal here.
And to be clear, I also would like to see what Simon's proposed, in practice.
On 12/29/23 18:21, Tom Rini wrote:
On Fri, Dec 29, 2023 at 06:09:44PM +0100, Heinrich Schuchardt wrote:
On 12/29/23 17:47, Tom Rini wrote:
On Fri, Dec 29, 2023 at 05:42:17PM +0100, Heinrich Schuchardt wrote:
On 12/20/23 20:12, Tom Rini wrote:
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote:
Hi,
On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote: > > On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote: >> >> >> Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com: >>> On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote: >>>> >>>> >>>> Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com: >>>>> On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: >>>>>> >>>>>> >>>>>> Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: >>>>>>> On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: >>>>>>>> >>>>>>>> >>>>>>>> Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: >>>>>>>>> On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: >>>>>>>>> >>>>>>>>> [snip] >>>>>>>>>> Or take: >>>>>>>>>> >>>>>>>>>> load host 0:1 $c kernel.efi >>>>>>>>>> load host 0:1 $d initrd.img >>>>>>>>>> >>>>>>>>>> How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? >>>>>>>>> >>>>>>>>> Today, invalid checksum as part of some part of the kernel fails. But >>>>>>>>> how do we do this tomorrow, are you suggesting that "load" perform >>>>>>>>> malloc() in some predefined size? If $c is below $d and $c + kernel.efi >>>>>>>>> is now above $d we can throw an error before trying to load, yes. But >>>>>>>>> what about: >>>>>>>>> load host 0:1 $d initrd.img >>>>>>>>> load host 0:1 $c kernel.efi >>>>>>>>> >>>>>>>>> In that case (which is only marginally contrived, the more real case is >>>>>>>>> loading device tree in to unexpectedly large ramdisk because someone >>>>>>>>> didn't understand the general advice on why device tree is lower than >>>>>>>>> ramdisk address) I'm fine with an error that amounts to "you just >>>>>>>>> corrupted another allocation" and then "fail, reset the board" or so. >>>>>>>>> >>>>>>>> >>>>>>>> Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. >>>>>>>> >>>>>>>> The load command should permanently allocate memory in that lmb+ library. >>>>>>>> >>>>>>>> We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. >>>>>>> >>>>>>> Our current way of loading things in to memory does not handle the case >>>>>>> I described, yes. How would what you're proposing handle it? >>>>>> >>>>>> If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail. >>>>> >>>>> So you're saying that the load command has to pre-allocate memory? Or as >>>>> it goes? If the latter, in what size chunks? This starts to get at what >>>>> Simon was talking about with respect to memory fragmentation. Which to >>>>> be clear is a problem we have today, we just let things overlap and hope >>>>> something later catches an incorrect checksum. >>>>> >>>> >>>> I don't want to replace the malloc library which handles large numbets of allocations. >>> >>> I'm confused. The normal malloc library is not involved with current >>> image loading, it's direct to memory (with some attempts at sanity >>> checking by lmb). Are you proposing a different allocator with >>> malloc/free like behavior? If so, please outline how it will determine >>> pool size, and how we'll use it to load thing to memory. >> >> All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack. > > That's a rather huge change for how U-Boot works. > >>>> Closing the eyes when the user loads multiple files does not solve the fragmentation problem. >>> >>> Yes. I'm only noting that today we just ignore the problem and sometimes >>> catch it via checksums. >>> >>>> Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again. >>> >>> OK, so are you saying that we would no longer be loading _to_ a location >>> in memory and instead just be saying "load this thing" and picking where >>> dynamically? >> >> Both preassigned and allocator assigned adresses are compatible with memory management. >> >> Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions. > > Yes, 64 bit architecture tend to only have alignment requirements while > 32bit architectures have both alignment requirements and some memory > window requirement. Whatever we implement here needs to handle both > cases. > >>>> When loading a file from a file system we know the filesize beforehand. So allocation is trivial. >>>> >>>> The loady command currently does not use the offered size information but could do so. >>> >>> We should be using that information to make sure we don't overwrite >>> U-Boot itself, but I don't recall how exactly we handle it today >>> off-hand. >> >> If the user issues multiple load commands, he can overwrite previous files. > > Then it sounds like we lost one benefit of all of this overhead. > >> During boot command execution I guess the different allocations respect each other. >> >>> >>>> TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file. >>> >>> Reading from non-filesystem flash also has this problem, but we at least >>> specify the amount to read too. But yes, it gets back to what I was >>> asking about on how you're proposing to handle network load cases. >>> >> >> It depends on the protocol. Http conveys the size before the data. Tftp does not. >> >> If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage. > > Yes, and what I'm trying to find out is if what you're suggesting would > do anything about it, since previous statements you made implied to me > that we would prevent it. > > To me, at this point it sounds like what we need is more like persistent > memory blocks and a hook that can be called in to for both "give me all > known memory blocks" and "add this memory block to the list", so that > EFI can do whatever it needs to do upon starting an application and then > upon return to U-Boot. Both malloc/free allocations and "load this blob > to memory from whatever" allocations would call the appropriate hook for > tracking.
In my mind the solution to this entire problem is fairly minor changes to how memory is allocated and only for EFI.
I tried to map out what that would look like and we have IMO got lost in the weeds a bit.
I am not trying to solve the problem of the 'load' command doing an allocation and throwing it away. To be that is WAI, at least until we come up with another type of command. This is one of the reasons for standard boot, allowing a more cohesive approach to booting.
I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
The range managed by the EFI sub-system extends over all of RAM including the addresses used by load commands. Hence, "the range that covers U-Boot itself (malloc pool and so forth)" includes all memory.
Maybe we could pre-allocate a memory area for file loading. This just requires to define a device-specific high file memory address above which no file loads will be allowed and below which EFI, bootm, etc will not be allowed to allocate memory for further uses.
We're not just / only an EFI runtime. When we load something persistent to memory outside of EFI, we need to note that, so that when EFI_LOADER beings work it can see this and do whatever it needs to do. And if/when EFI_LOADER returns control back to U-Boot itself, it needs to update that list with any new allocations that were done and mark them appropriately.
The idea of EFI loader returning control is misleading.
You can load an EFI driver binary which provides a device to U-Boot. Whenever U-Boot accesses that driver allocations might occur.
We cannot, today, load an EFI driver and then make use of it outside of the context of having bootefi'd something, yes?
We already can use iPXE to access iSCSI which provides a U-Boot block device. But it is an app. A similar binary could be compiled as EFI driver return to U-Boot and provide a block device to be directly addressed by the user via the CLI.
Or is this about that separate runtime area from another thread?
But even then, if something is making an allocation it needs to inform the rest of the world.
We should have one memory management. When EFI's AllocatPages() or AllocatePool() is called it should use the same backend as any other allocation (except for the small region managed by malloc()).
I thought that we already agreed to have only one memory management and get rid of the LMB/EFI memory management duplication.
We've agreed that today, CONFIG_LMB does not fit the use cases that we have in a modern system, for anyones usage. It has been suggested and I know it's somewhere on Ilias' TODO list, to try and take what we have in the EFI_LOADER & co area and make it usable and useful for all contexts. We also have what in some ways feels to me to be the inverse of that, which is Simon's proposal.
LMB and efi/loader/efi_memory.c have a big overlap in functionality.
But we use LMB in a unfavorable way discarding all allocations.
Best regards
Heinrich
We have not agreed that the EFI model and restrictions are what everything will now be using. What's on Ilias' TODD list might bring us in that direction, and I want to see what that looks like in practice. But it's not agreed to as the end goal here.
And to be clear, I also would like to see what Simon's proposed, in practice.
On Fri, Dec 29, 2023 at 06:30:43PM +0100, Heinrich Schuchardt wrote:
On 12/29/23 18:21, Tom Rini wrote:
On Fri, Dec 29, 2023 at 06:09:44PM +0100, Heinrich Schuchardt wrote:
On 12/29/23 17:47, Tom Rini wrote:
On Fri, Dec 29, 2023 at 05:42:17PM +0100, Heinrich Schuchardt wrote:
On 12/20/23 20:12, Tom Rini wrote:
On Tue, Dec 19, 2023 at 09:15:21PM -0700, Simon Glass wrote: > Hi, > > On Tue, 19 Dec 2023 at 05:46, Tom Rini trini@konsulko.com wrote: > > > > On Tue, Dec 19, 2023 at 03:15:38AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > Am 19. Dezember 2023 02:26:00 MEZ schrieb Tom Rini trini@konsulko.com: > > > > On Tue, Dec 19, 2023 at 01:01:51AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > Am 19. Dezember 2023 00:31:30 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > On Tue, Dec 19, 2023 at 12:29:19AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > > > > Am 19. Dezember 2023 00:16:40 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > > > On Tue, Dec 19, 2023 at 12:08:31AM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Am 18. Dezember 2023 23:41:08 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > > > > > On Mon, Dec 18, 2023 at 11:34:16PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > > > [snip] > > > > > > > > > > > Or take: > > > > > > > > > > > > > > > > > > > > > > load host 0:1 $c kernel.efi > > > > > > > > > > > load host 0:1 $d initrd.img > > > > > > > > > > > > > > > > > > > > > > How could we ensure that initrd.img is not overwriting a part of kernel.efi without memory allocation? > > > > > > > > > > > > > > > > > > > > Today, invalid checksum as part of some part of the kernel fails. But > > > > > > > > > > how do we do this tomorrow, are you suggesting that "load" perform > > > > > > > > > > malloc() in some predefined size? If $c is below $d and $c + kernel.efi > > > > > > > > > > is now above $d we can throw an error before trying to load, yes. But > > > > > > > > > > what about: > > > > > > > > > > load host 0:1 $d initrd.img > > > > > > > > > > load host 0:1 $c kernel.efi > > > > > > > > > > > > > > > > > > > > In that case (which is only marginally contrived, the more real case is > > > > > > > > > > loading device tree in to unexpectedly large ramdisk because someone > > > > > > > > > > didn't understand the general advice on why device tree is lower than > > > > > > > > > > ramdisk address) I'm fine with an error that amounts to "you just > > > > > > > > > > corrupted another allocation" and then "fail, reset the board" or so. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Our current malloc library cannot manage the complete memory. We need a library like lmb which should also cover the memory management that we currently have in lib/efi/efi_memory.c. This must include a memory type attribute for usage in the GetMemoryMap() service. A management on page level seems sufficient. > > > > > > > > > > > > > > > > > > The load command should permanently allocate memory in that lmb+ library. > > > > > > > > > > > > > > > > > > We need an unload command to free the memory if we want to reuse the memory or we might let the load comand free the memory if exactly the same start address is reused. > > > > > > > > > > > > > > > > Our current way of loading things in to memory does not handle the case > > > > > > > > I described, yes. How would what you're proposing handle it? > > > > > > > > > > > > > > If the load command has to allocate memory for the image and that allocation is kept, any attempt to allocate overlapping memory would fail. > > > > > > > > > > > > So you're saying that the load command has to pre-allocate memory? Or as > > > > > > it goes? If the latter, in what size chunks? This starts to get at what > > > > > > Simon was talking about with respect to memory fragmentation. Which to > > > > > > be clear is a problem we have today, we just let things overlap and hope > > > > > > something later catches an incorrect checksum. > > > > > > > > > > > > > > > > I don't want to replace the malloc library which handles large numbets of allocations. > > > > > > > > I'm confused. The normal malloc library is not involved with current > > > > image loading, it's direct to memory (with some attempts at sanity > > > > checking by lmb). Are you proposing a different allocator with > > > > malloc/free like behavior? If so, please outline how it will determine > > > > pool size, and how we'll use it to load thing to memory. > > > > > > All memory below the stack needs to be managed. Malloc uses a small memory area (a few MiB) above the stack. > > > > That's a rather huge change for how U-Boot works. > > > > > > > Closing the eyes when the user loads multiple files does not solve the fragmentation problem. > > > > > > > > Yes. I'm only noting that today we just ignore the problem and sometimes > > > > catch it via checksums. > > > > > > > > > Fragmentation only happens if we have many concurrent allocations. In EFI we are allocating top down. The number of concurrent allocations is low. Typically a few dozen at most. After terminating an application these should be freed again. > > > > > > > > OK, so are you saying that we would no longer be loading _to_ a location > > > > in memory and instead just be saying "load this thing" and picking where > > > > dynamically? > > > > > > Both preassigned and allocator assigned adresses are compatible with memory management. > > > > > > Architectures and binaries have different requirements. On riscv64 you can load Linux kernel, initrd, fdt anywhere. We don't need predefined addresses there. Other architectures have restrictions. > > > > Yes, 64 bit architecture tend to only have alignment requirements while > > 32bit architectures have both alignment requirements and some memory > > window requirement. Whatever we implement here needs to handle both > > cases. > > > > > > > When loading a file from a file system we know the filesize beforehand. So allocation is trivial. > > > > > > > > > > The loady command currently does not use the offered size information but could do so. > > > > > > > > We should be using that information to make sure we don't overwrite > > > > U-Boot itself, but I don't recall how exactly we handle it today > > > > off-hand. > > > > > > If the user issues multiple load commands, he can overwrite previous files. > > > > Then it sounds like we lost one benefit of all of this overhead. > > > > > During boot command execution I guess the different allocations respect each other. > > > > > > > > > > > > TFTP is problematic because it does not transfer the filesize. We would probably try to allocate a large chunk of memory and then downsize the allocation after reading the whole file. > > > > > > > > Reading from non-filesystem flash also has this problem, but we at least > > > > specify the amount to read too. But yes, it gets back to what I was > > > > asking about on how you're proposing to handle network load cases. > > > > > > > > > > It depends on the protocol. Http conveys the size before the data. Tftp does not. > > > > > > If you don't know the size, you must preallocate a big chunk, check that the download does not exceed it, and downsize the allocation afterwards. This is not a new problem but exists already with current lmb usage. > > > > Yes, and what I'm trying to find out is if what you're suggesting would > > do anything about it, since previous statements you made implied to me > > that we would prevent it. > > > > To me, at this point it sounds like what we need is more like persistent > > memory blocks and a hook that can be called in to for both "give me all > > known memory blocks" and "add this memory block to the list", so that > > EFI can do whatever it needs to do upon starting an application and then > > upon return to U-Boot. Both malloc/free allocations and "load this blob > > to memory from whatever" allocations would call the appropriate hook for > > tracking. > > In my mind the solution to this entire problem is fairly minor changes > to how memory is allocated and only for EFI. > > I tried to map out what that would look like and we have IMO got lost > in the weeds a bit. > > I am not trying to solve the problem of the 'load' command doing an > allocation and throwing it away. To be that is WAI, at least until we > come up with another type of command. This is one of the reasons for > standard boot, allowing a more cohesive approach to booting. > > I will think about this some more...
OK, but please keep in mind that lmb not being at all persistent is a problem for everyone, not just EFI. That really needs to be addressed, maybe with some flags for dis-allowing overwrites to the area. For example, the apple-m1 code to use lmb to find locations for the kernel/etc can be written to more than once (allocate the address, then write to it to start with, even) but the range that covers U-Boot itself (malloc pool and so forth) need to be stopped.
The range managed by the EFI sub-system extends over all of RAM including the addresses used by load commands. Hence, "the range that covers U-Boot itself (malloc pool and so forth)" includes all memory.
Maybe we could pre-allocate a memory area for file loading. This just requires to define a device-specific high file memory address above which no file loads will be allowed and below which EFI, bootm, etc will not be allowed to allocate memory for further uses.
We're not just / only an EFI runtime. When we load something persistent to memory outside of EFI, we need to note that, so that when EFI_LOADER beings work it can see this and do whatever it needs to do. And if/when EFI_LOADER returns control back to U-Boot itself, it needs to update that list with any new allocations that were done and mark them appropriately.
The idea of EFI loader returning control is misleading.
You can load an EFI driver binary which provides a device to U-Boot. Whenever U-Boot accesses that driver allocations might occur.
We cannot, today, load an EFI driver and then make use of it outside of the context of having bootefi'd something, yes?
We already can use iPXE to access iSCSI which provides a U-Boot block device. But it is an app. A similar binary could be compiled as EFI driver return to U-Boot and provide a block device to be directly addressed by the user via the CLI.
To be clear:
Today we can, or cannot, use iPXE to access an iSCSI device and then "ext4load" off of it?
Today we can, or cannot, use iPXE to access an iSCSI device and then "bootefi" another EFI application such as grub which can then load off if it?
Today we can, or cannot, use iPXE to access an iSCSI device and have it place a Linux Kernel Image at $loadaddr which we can then "md" or any other standard U-Boot command with?
My assumption is that we can do the last two, today, and the first one is something that yes, the specification allows, but not something we have today and requires some discussion on how much we want to allow modules-by-another-name to work. I very much don't want to start that discussion right now either.
Or is this about that separate runtime area from another thread?
But even then, if something is making an allocation it needs to inform the rest of the world.
We should have one memory management. When EFI's AllocatPages() or AllocatePool() is called it should use the same backend as any other allocation (except for the small region managed by malloc()).
I thought that we already agreed to have only one memory management and get rid of the LMB/EFI memory management duplication.
We've agreed that today, CONFIG_LMB does not fit the use cases that we have in a modern system, for anyones usage. It has been suggested and I know it's somewhere on Ilias' TODO list, to try and take what we have in the EFI_LOADER & co area and make it usable and useful for all contexts. We also have what in some ways feels to me to be the inverse of that, which is Simon's proposal.
LMB and efi/loader/efi_memory.c have a big overlap in functionality.
But we use LMB in a unfavorable way discarding all allocations.
Yes, we all agree that what we do with CONFIG_LMB today is not good nor what we want long term. In some ways that feels like the single point of agreement.
We have not agreed that the EFI model and restrictions are what everything will now be using. What's on Ilias' TODD list might bring us in that direction, and I want to see what that looks like in practice. But it's not agreed to as the end goal here.
And to be clear, I also would like to see what Simon's proposed, in practice.
Because this part here. We've probably all talked this out a lot. Without an implementation to examine we're probably going to continue to not agree and possibly unintentionally misunderstand each other.
On Mon, Dec 18, 2023 at 02:03:41PM -0700, Simon Glass wrote:
Hi Heinrich,
On Mon, 18 Dec 2023 at 13:00, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Am 18. Dezember 2023 19:12:11 MEZ schrieb Simon Glass sjg@chromium.org:
Hi Heinrich,
On Sat, 16 Dec 2023 at 12:04, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On 12/16/23 19:01, Simon Glass wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
Hello Simon,
thank you for summarizing our discussion.
Some U-Boot drivers including rockchip video inform the EFI sub-system that memory is reserved.
Furthermore drivers like arch/arm/mach-bcm283x/reset.c exist that are still used after ExitBootServices. mmio addresses have to be updated when Linux creates its virtual memory map. Currently this is done via efi_add_runtime_mmio(). A more UEFI style method would be to register an event handler for ExitBootServices() and use ConvertPointer() in the event handler.
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
U-Boot fails to set up RWX properties. E.g. the region where a FIT image is loaded should not be executable.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
Fixup code is not duplicated but invoked via image_setup_libfdt().
- EFI code is used even if EFI is never used to boot
- Only a minimum initialization of the EFI sub-system happens in
efi_init_early().
- Some EFI code is called when probing block devices because we wanted
the EFI and the dm part to be integrated.
- The rest of the initialization in efi_init_obj_list() is only invoked
if an EFI command is invoked.
- EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
The most worrisome issue is that EFI may allocate memory where U-Boot has loaded files like initrd as the EFI sub-system is never informed which memory is used for files.
Loading files should not be possible without creating a memory reservation that becomes visible to the EFI sub-system.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
U-Boot can load EFI drivers which stay resident in memory after the efi_main() method has returned to U-Boot. The next EFI application then may use the driver. Therefore is essential that the EFI subsystem has access to a valid memory model at all times.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
The way lmb is used is a deficiency of U-Boot. E.g. you can load an initrd that overwrites the previously loaded kernel and then try to boot.
What we need is a common memory management library where allocations are never dropped and which is used by all file loads.
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
We have to implement what the UEFI specification requires. Some boot-time services must allocate memory via AllocatePool() or AllocatePages() because that memory is handed out to the caller of an API function and it is the callers obligation to free the memory via FreePool() or FreePages().
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
When video memory is located higher than the stack the EFI sub-system will not make use of the memory.
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
This would require a permanent storage of the reservations. Keep it easy, unify the memory management and make it persistent.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
There is no such thing as EFI memory. We only have one physical memory that we have to keep track of.
It is not possible to set up the EFI memory map if you don't keep track of all memory allocations including all file loads over the whole lifetime of U-Boot.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
If we have a unified memory allocation layer, efi_allocate_pages() and efi_allocate_pool() will be implemented by calls into that layer and issue 6) will vanish.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
The time intensive part of having EFI enabled is scanning file systems for boot files and capsules.
Thank you for your thoughts on this.
It does not look like my write-up has helped at all with getting aligned on this. Do you have any other ideas?
Perhaps we could at least figure out the 'allocation lifetime' approach? It seems clear to me that we have allocations with short lifetimes (e.g. kernel & ramdisk allocations will remain in effect only when booting). Do you agree with that?
We agree that we want to have a unified memory management.
Memory management implies that memory allocations remain in effect until the memory is freed.
This must be true for all allocations whether it is for a kernel, a ramdisk, for loading a file, for an allocation by an EFI binary, or for anything else.
If boot commands like bootm allocate memory and free it after a failure, that time will be short.
If a file is loaded and never unloaded that memory allocation must stay until the OS takes over memory management. This is what is missing in U-Boot.
I see that more as a reservation than an allocation. The 'load' command can never do a useful memory allocation, since the user may want to load a different file to the same address.
We don't 'free' memory reserved for the kernel/ramdisk...the allocation is only temporary. This is what we need to align on. Those allocations should be collated and reported to EFI before booting, then dropped if the boot fails. This in fact works today, with lmb, so we should not need to change it.
I think this emphasises bugs of lmb as being features when they really shouldn't be. There's classes of CVE that we attempt to paper over today with lmb, but should really fix instead. The only thing lmb is doing in bootm is trying to make sure that since we (almost certainly) want to move the fdt, we don't move it on top of something else we're trying to boot (or, oh, ourself I guess too).
Hi Simon
I'll respond to the rest more thoroughly but I since I caught this early,
[...]
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
The TCG event log is only valid in the EFI world and is described by the EFI spec extensions [0]. I prefer it to remain as is
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
–
Regards, Simon
[0] https://trustedcomputinggroup.org/wp-content/uploads/EFI-Protocol-Specificat... Thanks /Ilias
Hi Ilias,
On Fri, Dec 22, 2023 at 12:02 PM Ilias Apalodimas ilias.apalodimas@linaro.org wrote:
Hi Simon
I'll respond to the rest more thoroughly but I since I caught this early,
[...]
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
The TCG event log is only valid in the EFI world and is described by the EFI spec extensions [0]. I prefer it to remain as is
How does that relate to [1] ?
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
–
Regards, Simon
[0] https://trustedcomputinggroup.org/wp-content/uploads/EFI-Protocol-Specificat... Thanks /Ilias
[1] https://trustedfirmware-a.readthedocs.io/en/latest/components/measured_boot/...
Hi Simon,
On Fri, 29 Dec 2023 at 07:36, Simon Glass sjg@chromium.org wrote:
Hi Ilias,
On Fri, Dec 22, 2023 at 12:02 PM Ilias Apalodimas ilias.apalodimas@linaro.org wrote:
Hi Simon
I'll respond to the rest more thoroughly but I since I caught this early,
[...]
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
The TCG event log is only valid in the EFI world and is described by the EFI spec extensions [0]. I prefer it to remain as is
How does that relate to [1] ?
We use that today. The tl;dr is that TF-A measures the early stages, but doesn't have drivers to extend TPM PCRs. Instead, it creates an EventLog which hands over to U-Boot. The TPM subsystem searches for that and if it finds one && PCR0 == 0 replays the EventLog in hardware and copies it over to EFI memory.
That being said, I'd much prefer using a bloblist for that EventLog (which also contains information on replaying), over a DT entry that's only described in TF-A docs. But that's a future improvement.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
–
Regards, Simon
[0] https://trustedcomputinggroup.org/wp-content/uploads/EFI-Protocol-Specificat... Thanks /Ilias
[1] https://trustedfirmware-a.readthedocs.io/en/latest/components/measured_boot/...
Cheers /Ilias
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
7. We need to support doing an allocation when a file is loaded (to ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
7. Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
Regards, Simon
Am 29. Dezember 2023 06:36:09 MEZ schrieb Simon Glass sjg@chromium.org:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
It is unclear what your suggestion means for the address field.
If you still specify an address, what would be the benefit of such a flag?
If -m replaces the address, how do you refer to different loaded files of the same type, e.g. two EFI binaries?
Best regards
Heinrich
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
Regards, Simon
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com:
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
Best regards
Heinrich
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote:
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com:
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
Date: Fri, 29 Dec 2023 11:17:44 -0500 From: Tom Rini trini@konsulko.com
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote:
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com:
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote:
Date: Fri, 29 Dec 2023 11:17:44 -0500 From: Tom Rini trini@konsulko.com
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote:
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com:
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote:
Hi,
This records my thoughts after a discussion with Ilias & Heinrich re memory allocation in U-Boot.
- malloc()
malloc() is used for programmatic memory allocation. It allows memory to be freed. It is not designed for very large allocations (e.g. a 10MB kernel or 100MB ramdisk).
- lmb
lmb is used for large blocks of memory, such as those needed for a kernel or ramdisk. Allocation is only transitory, for the purposes of loading some images and booting. If the boot fails, then all lmb allocations go away.
lmb is set up by getting all available memory and then removing what is used by U-Boot (code, data, malloc() space, etc.)
lmb reservations have a few flags so that areas of memory can be provided with attributes
There are some corner cases...e.g. loading a file does an lmb allocation but only for the purpose of avoiding a file being loaded over U-Boot code/data. The allocation is dropped immediately after the file is loaded. Within the bootm command, or when using standard boot, this would be fairly easy to solve.
Linux has renamed lmb to memblock. We should consider doing the same.
- EFI
EFI has its own memory-allocation tables.
Like lmb, EFI is able to deal with large allocations. But via a 'pool' function it can also do smaller allocations similar to malloc(), although each one uses at least 4KB at present.
EFI allocations do not go away when a boot fails.
With EFI it is possible to add allocations post facto, in which case they are added to the allocation table just as if the memory was allocated with EFI to begin with.
The EFI allocations and the lmb allocations use the same memory, so in principle could conflict.
EFI allocations are sometimes used to allocate internal U-Boot data as well, if needed by the EFI app. For example, while efi_image_parse() uses malloc(), efi_var_mem.c uses EFI allocations since the code runs in the app context and may need to access the memory after U-Boot has exited. Also efi_smbios.c uses allocate_pages() and then adds a new mapping as well.
EFI memory has attributes, including what the memory is used for (to some degree of granularity). See enum efi_memory_type and struct efi_mem_desc. In the latter there are also attribute flags - whether memory is cacheable, etc.
EFI also has the x86 idea of 'conventional' memory, meaning (I believe) that below 4GB that isn't reserved for the hardware/system. This is meaningless, or at least confusing, on ARM systems.
- reservations
It is perhaps worth mentioning a fourth method of memory management, where U-Boot reserves chunks of memory before relocation (in board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() region, etc.
Problems —-------
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
Lifetime
We have three different memory allocators with different purposes. Can we unify them a little?
Within U-Boot:
- malloc() space lives forever
- lmb lives while setting out images for booting
- EFI (mostly) lives while booting an EFI app
In practice, EFI is set up early in U-Boot. Some of this is necessary, some not. EFI allocations stay around forever. This works OK since large allocations are normally not done in EFI, so memory isn't really consumed to any great degree by the boot process.
What happens to EFI allocations if the app returns? They are still present, in case another app is run. This seems fine.
API –-- Can we unify some APIs?
It should be possible to use lmb for large EFI memory allocations, so long as they are only needed for booting. We effectively do this today, since EFI does not manage the arrangement of loaded images in memory. for the most part.
It would not make sense to use EFI allocation to replace lmb and malloc(), of course.
Could we use a common (lower-level) API for allocation, used by both lmb and EFI? They do have some similarities. However they have different lifetime constraints (EFI allocations are never dropped, unlikely lmb).
** Overall, it seems that the existence of memory allocation in boot-time services has created confusion. Memory allocation is muddled, with both U-Boot code and boot-time services calling the same memory allocator. This just has not been clearly thought out.
Proposal —-------
Here are some ideas:
- For video, use the driver model API to locate the video regions, or
block off the entire framebuffer memory, for all devices as a whole. Use efi_add_memory_map()
- Add memory attributes to UCLASS_RAM and use them in EFI, mapping to
the EFI_MEMORY_... attributes in struct efi_mem_desc.
- Add all EFI reservations just before booting the app, as we do with
devicetree fixup. With this model, malloc() and lmb are used for all allocation. Then efi_add_memory_map() is called for each region in turn just before booting. Memory attributes are dealt with above. The type (enum efi_memory_type) can be determined simply by the data structure stored in it, as is done today. For example, SMBIOS tables can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code understands the meaning of each.
- Avoid setting up EFI memory at the start of U-Boot. Do it only when
booting. This looks to require very little effort.
- Avoid calling efi_allocate_pages() and efi_allocate_pool() outside
boot-time services. This solves the problem 6. If memory is needed by an app, allocate it with malloc() and see 3. There are only two efi_allocate_pages() (smbios and efi_runtime). There are more calls of efi_allocate_pool(), but most of these seem easy to fix up. For example, efi_init_event_log() allocates a buffer, but this can be allocated in normal malloc() space or in a bloblist.
- Don't worry too much about whether EFI will be used for booting.
The cost is likely not that great: use bootstage to measure it as is done for driver model. Try to minmise the cost of its tables, particularly for execution time, but otherwise just rely on the ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote:
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote:
Date: Fri, 29 Dec 2023 11:17:44 -0500 From: Tom Rini trini@konsulko.com
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote:
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com:
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote:
Hi,
On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > Hi, > > This records my thoughts after a discussion with Ilias & Heinrich re > memory allocation in U-Boot. > > 1. malloc() > > malloc() is used for programmatic memory allocation. It allows memory > to be freed. It is not designed for very large allocations (e.g. a > 10MB kernel or 100MB ramdisk). > > 2. lmb > > lmb is used for large blocks of memory, such as those needed for a > kernel or ramdisk. Allocation is only transitory, for the purposes of > loading some images and booting. If the boot fails, then all lmb > allocations go away. > > lmb is set up by getting all available memory and then removing what > is used by U-Boot (code, data, malloc() space, etc.) > > lmb reservations have a few flags so that areas of memory can be > provided with attributes > > There are some corner cases...e.g. loading a file does an lmb > allocation but only for the purpose of avoiding a file being loaded > over U-Boot code/data. The allocation is dropped immediately after the > file is loaded. Within the bootm command, or when using standard boot, > this would be fairly easy to solve. > > Linux has renamed lmb to memblock. We should consider doing the same. > > 3. EFI > > EFI has its own memory-allocation tables. > > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > function it can also do smaller allocations similar to malloc(), > although each one uses at least 4KB at present. > > EFI allocations do not go away when a boot fails. > > With EFI it is possible to add allocations post facto, in which case > they are added to the allocation table just as if the memory was > allocated with EFI to begin with. > > The EFI allocations and the lmb allocations use the same memory, so in > principle could conflict. > > EFI allocations are sometimes used to allocate internal U-Boot data as > well, if needed by the EFI app. For example, while efi_image_parse() > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > in the app context and may need to access the memory after U-Boot has > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > mapping as well. > > EFI memory has attributes, including what the memory is used for (to > some degree of granularity). See enum efi_memory_type and struct > efi_mem_desc. In the latter there are also attribute flags - whether > memory is cacheable, etc. > > EFI also has the x86 idea of 'conventional' memory, meaning (I > believe) that below 4GB that isn't reserved for the hardware/system. > This is meaningless, or at least confusing, on ARM systems. > > 4. reservations > > It is perhaps worth mentioning a fourth method of memory management, > where U-Boot reserves chunks of memory before relocation (in > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > region, etc. > > > Problems > —------- > > There are no urgent problems, but here are some things that could be improved: > > 1. EFI should attach most of its data structures to driver model. This > work has started, with the partition support, but more effort would > help. This would make it easier to see which memory is related to > devices and which is separate. > > 2. Some drivers do EFI reservations today, whether EFI is used for > booting or not (e.g. rockchip video rk_vop_probe()). > > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > armv8's struct mm_region) to EFI ones. > > 4. EFI duplicates some code from bootm, some of which relates to > memory allocation (e.g. FDT fixup). > > 5. EFI code is used even if EFI is never used to boot > > 6. EFI allocations can result in the same memory being used as has > already been allocated by lmb. Users may load files which overwrite > memory allocated by EFI.
- We need to support doing an allocation when a file is loaded (to
ensure files do not overlap), without making it too difficult to load multiple files to the same place, if desired.
> > > Lifetime > -------- > > We have three different memory allocators with different purposes. Can > we unify them a little? > > Within U-Boot: > - malloc() space lives forever > - lmb lives while setting out images for booting > - EFI (mostly) lives while booting an EFI app > > In practice, EFI is set up early in U-Boot. Some of this is necessary, > some not. EFI allocations stay around forever. This works OK since > large allocations are normally not done in EFI, so memory isn't really > consumed to any great degree by the boot process. > > What happens to EFI allocations if the app returns? They are still > present, in case another app is run. This seems fine. > > API > –-- > Can we unify some APIs? > > It should be possible to use lmb for large EFI memory allocations, so > long as they are only needed for booting. We effectively do this > today, since EFI does not manage the arrangement of loaded images in > memory. for the most part. > > It would not make sense to use EFI allocation to replace lmb and > malloc(), of course. > > Could we use a common (lower-level) API for allocation, used by both > lmb and EFI? They do have some similarities. However they have > different lifetime constraints (EFI allocations are never dropped, > unlikely lmb). > > ** Overall, it seems that the existence of memory allocation in > boot-time services has created confusion. Memory allocation is > muddled, with both U-Boot code and boot-time services calling the same > memory allocator. This just has not been clearly thought out. > > > Proposal > —------- > > Here are some ideas: > > 1. For video, use the driver model API to locate the video regions, or > block off the entire framebuffer memory, for all devices as a whole. > Use efi_add_memory_map() > > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > 3. Add all EFI reservations just before booting the app, as we do with > devicetree fixup. With this model, malloc() and lmb are used for all > allocation. Then efi_add_memory_map() is called for each region in > turn just before booting. Memory attributes are dealt with above. The > type (enum efi_memory_type) can be determined simply by the data > structure stored in it, as is done today. For example, SMBIOS tables > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > understands the meaning of each. > > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > booting. This looks to require very little effort. > > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > boot-time services. This solves the problem 6. If memory is needed by > an app, allocate it with malloc() and see 3. There are only two > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > efi_allocate_pool(), but most of these seem easy to fix up. For > example, efi_init_event_log() allocates a buffer, but this can be > allocated in normal malloc() space or in a bloblist. > > 6. Don't worry too much about whether EFI will be used for booting. > The cost is likely not that great: use bootstage to measure it as is > done for driver model. Try to minmise the cost of its tables, > particularly for execution time, but otherwise just rely on the > ability to disable EFI_LOADER.
- Add a flag to the 'load' command:
-m <type> - make an lmb allocation for the file <type> is the image type to use (kernel, ramdisk, flat_dt)
any existing allocation for that type will be automatically freed first. If <type> is "none" then no freeing is possible: any loaded images just stack up in lmb.
Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
I'm not sure if we still want the proposed lmb command? Perhaps it is not needed.
Regards, Simon
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote:
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote:
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote:
Date: Fri, 29 Dec 2023 11:17:44 -0500 From: Tom Rini trini@konsulko.com
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote:
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com:
On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > Hi, > > On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > > Hi, > > > > This records my thoughts after a discussion with Ilias & Heinrich re > > memory allocation in U-Boot. > > > > 1. malloc() > > > > malloc() is used for programmatic memory allocation. It allows memory > > to be freed. It is not designed for very large allocations (e.g. a > > 10MB kernel or 100MB ramdisk). > > > > 2. lmb > > > > lmb is used for large blocks of memory, such as those needed for a > > kernel or ramdisk. Allocation is only transitory, for the purposes of > > loading some images and booting. If the boot fails, then all lmb > > allocations go away. > > > > lmb is set up by getting all available memory and then removing what > > is used by U-Boot (code, data, malloc() space, etc.) > > > > lmb reservations have a few flags so that areas of memory can be > > provided with attributes > > > > There are some corner cases...e.g. loading a file does an lmb > > allocation but only for the purpose of avoiding a file being loaded > > over U-Boot code/data. The allocation is dropped immediately after the > > file is loaded. Within the bootm command, or when using standard boot, > > this would be fairly easy to solve. > > > > Linux has renamed lmb to memblock. We should consider doing the same. > > > > 3. EFI > > > > EFI has its own memory-allocation tables. > > > > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > function it can also do smaller allocations similar to malloc(), > > although each one uses at least 4KB at present. > > > > EFI allocations do not go away when a boot fails. > > > > With EFI it is possible to add allocations post facto, in which case > > they are added to the allocation table just as if the memory was > > allocated with EFI to begin with. > > > > The EFI allocations and the lmb allocations use the same memory, so in > > principle could conflict. > > > > EFI allocations are sometimes used to allocate internal U-Boot data as > > well, if needed by the EFI app. For example, while efi_image_parse() > > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > in the app context and may need to access the memory after U-Boot has > > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > mapping as well. > > > > EFI memory has attributes, including what the memory is used for (to > > some degree of granularity). See enum efi_memory_type and struct > > efi_mem_desc. In the latter there are also attribute flags - whether > > memory is cacheable, etc. > > > > EFI also has the x86 idea of 'conventional' memory, meaning (I > > believe) that below 4GB that isn't reserved for the hardware/system. > > This is meaningless, or at least confusing, on ARM systems. > > > > 4. reservations > > > > It is perhaps worth mentioning a fourth method of memory management, > > where U-Boot reserves chunks of memory before relocation (in > > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > region, etc. > > > > > > Problems > > —------- > > > > There are no urgent problems, but here are some things that could be improved: > > > > 1. EFI should attach most of its data structures to driver model. This > > work has started, with the partition support, but more effort would > > help. This would make it easier to see which memory is related to > > devices and which is separate. > > > > 2. Some drivers do EFI reservations today, whether EFI is used for > > booting or not (e.g. rockchip video rk_vop_probe()). > > > > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > armv8's struct mm_region) to EFI ones. > > > > 4. EFI duplicates some code from bootm, some of which relates to > > memory allocation (e.g. FDT fixup). > > > > 5. EFI code is used even if EFI is never used to boot > > > > 6. EFI allocations can result in the same memory being used as has > > already been allocated by lmb. Users may load files which overwrite > > memory allocated by EFI. > > 7. We need to support doing an allocation when a file is loaded (to > ensure files do not overlap), without making it too difficult to load > multiple files to the same place, if desired. > > > > > > > Lifetime > > -------- > > > > We have three different memory allocators with different purposes. Can > > we unify them a little? > > > > Within U-Boot: > > - malloc() space lives forever > > - lmb lives while setting out images for booting > > - EFI (mostly) lives while booting an EFI app > > > > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > some not. EFI allocations stay around forever. This works OK since > > large allocations are normally not done in EFI, so memory isn't really > > consumed to any great degree by the boot process. > > > > What happens to EFI allocations if the app returns? They are still > > present, in case another app is run. This seems fine. > > > > API > > –-- > > Can we unify some APIs? > > > > It should be possible to use lmb for large EFI memory allocations, so > > long as they are only needed for booting. We effectively do this > > today, since EFI does not manage the arrangement of loaded images in > > memory. for the most part. > > > > It would not make sense to use EFI allocation to replace lmb and > > malloc(), of course. > > > > Could we use a common (lower-level) API for allocation, used by both > > lmb and EFI? They do have some similarities. However they have > > different lifetime constraints (EFI allocations are never dropped, > > unlikely lmb). > > > > ** Overall, it seems that the existence of memory allocation in > > boot-time services has created confusion. Memory allocation is > > muddled, with both U-Boot code and boot-time services calling the same > > memory allocator. This just has not been clearly thought out. > > > > > > Proposal > > —------- > > > > Here are some ideas: > > > > 1. For video, use the driver model API to locate the video regions, or > > block off the entire framebuffer memory, for all devices as a whole. > > Use efi_add_memory_map() > > > > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > > 3. Add all EFI reservations just before booting the app, as we do with > > devicetree fixup. With this model, malloc() and lmb are used for all > > allocation. Then efi_add_memory_map() is called for each region in > > turn just before booting. Memory attributes are dealt with above. The > > type (enum efi_memory_type) can be determined simply by the data > > structure stored in it, as is done today. For example, SMBIOS tables > > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > understands the meaning of each. > > > > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > booting. This looks to require very little effort. > > > > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > boot-time services. This solves the problem 6. If memory is needed by > > an app, allocate it with malloc() and see 3. There are only two > > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > efi_allocate_pool(), but most of these seem easy to fix up. For > > example, efi_init_event_log() allocates a buffer, but this can be > > allocated in normal malloc() space or in a bloblist. > > > > 6. Don't worry too much about whether EFI will be used for booting. > > The cost is likely not that great: use bootstage to measure it as is > > done for driver model. Try to minmise the cost of its tables, > > particularly for execution time, but otherwise just rely on the > > ability to disable EFI_LOADER. > > 7. Add a flag to the 'load' command: > > -m <type> - make an lmb allocation for the file > <type> is the image type to use (kernel, ramdisk, flat_dt) > > any existing allocation for that type will be automatically freed > first. If <type> is "none" then no freeing is possible: any loaded > images just stack up in lmb. > > Add an 'lmb' (or memblock) command to allow listing and clearing allocations.
I would really not like to change the user interface and instead simply handle this with flags to whatever mark/allocation function is called. You can always overwrite things that are brought in to memory, you cannot overwrite U-Boot or our internals. Optionally noting that some previous load to memory has been at least partially overwritten could be helpful, if it's not too much extra logic.
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
I'm not sure if we still want the proposed lmb command? Perhaps it is not needed.
Please lets not call it "lmb" because the current lmb isn't what we want anymore and it's otherwise probably leading to some of the confusion. The kernel evolved it to memblocks, so perhaps that terminology too here. And yes, a command for if nothing else debug / investigation would be good.
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote:
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote:
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote:
Date: Fri, 29 Dec 2023 11:17:44 -0500 From: Tom Rini trini@konsulko.com
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote:
Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: >> Hi, >> >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: >> > >> > Hi, >> > >> > This records my thoughts after a discussion with Ilias & Heinrich re >> > memory allocation in U-Boot. >> > >> > 1. malloc() >> > >> > malloc() is used for programmatic memory allocation. It allows memory >> > to be freed. It is not designed for very large allocations (e.g. a >> > 10MB kernel or 100MB ramdisk). >> > >> > 2. lmb >> > >> > lmb is used for large blocks of memory, such as those needed for a >> > kernel or ramdisk. Allocation is only transitory, for the purposes of >> > loading some images and booting. If the boot fails, then all lmb >> > allocations go away. >> > >> > lmb is set up by getting all available memory and then removing what >> > is used by U-Boot (code, data, malloc() space, etc.) >> > >> > lmb reservations have a few flags so that areas of memory can be >> > provided with attributes >> > >> > There are some corner cases...e.g. loading a file does an lmb >> > allocation but only for the purpose of avoiding a file being loaded >> > over U-Boot code/data. The allocation is dropped immediately after the >> > file is loaded. Within the bootm command, or when using standard boot, >> > this would be fairly easy to solve. >> > >> > Linux has renamed lmb to memblock. We should consider doing the same. >> > >> > 3. EFI >> > >> > EFI has its own memory-allocation tables. >> > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' >> > function it can also do smaller allocations similar to malloc(), >> > although each one uses at least 4KB at present. >> > >> > EFI allocations do not go away when a boot fails. >> > >> > With EFI it is possible to add allocations post facto, in which case >> > they are added to the allocation table just as if the memory was >> > allocated with EFI to begin with. >> > >> > The EFI allocations and the lmb allocations use the same memory, so in >> > principle could conflict. >> > >> > EFI allocations are sometimes used to allocate internal U-Boot data as >> > well, if needed by the EFI app. For example, while efi_image_parse() >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs >> > in the app context and may need to access the memory after U-Boot has >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new >> > mapping as well. >> > >> > EFI memory has attributes, including what the memory is used for (to >> > some degree of granularity). See enum efi_memory_type and struct >> > efi_mem_desc. In the latter there are also attribute flags - whether >> > memory is cacheable, etc. >> > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I >> > believe) that below 4GB that isn't reserved for the hardware/system. >> > This is meaningless, or at least confusing, on ARM systems. >> > >> > 4. reservations >> > >> > It is perhaps worth mentioning a fourth method of memory management, >> > where U-Boot reserves chunks of memory before relocation (in >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() >> > region, etc. >> > >> > >> > Problems >> > —------- >> > >> > There are no urgent problems, but here are some things that could be improved: >> > >> > 1. EFI should attach most of its data structures to driver model. This >> > work has started, with the partition support, but more effort would >> > help. This would make it easier to see which memory is related to >> > devices and which is separate. >> > >> > 2. Some drivers do EFI reservations today, whether EFI is used for >> > booting or not (e.g. rockchip video rk_vop_probe()). >> > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. >> > armv8's struct mm_region) to EFI ones. >> > >> > 4. EFI duplicates some code from bootm, some of which relates to >> > memory allocation (e.g. FDT fixup). >> > >> > 5. EFI code is used even if EFI is never used to boot >> > >> > 6. EFI allocations can result in the same memory being used as has >> > already been allocated by lmb. Users may load files which overwrite >> > memory allocated by EFI. >> >> 7. We need to support doing an allocation when a file is loaded (to >> ensure files do not overlap), without making it too difficult to load >> multiple files to the same place, if desired. >> >> > >> > >> > Lifetime >> > -------- >> > >> > We have three different memory allocators with different purposes. Can >> > we unify them a little? >> > >> > Within U-Boot: >> > - malloc() space lives forever >> > - lmb lives while setting out images for booting >> > - EFI (mostly) lives while booting an EFI app >> > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, >> > some not. EFI allocations stay around forever. This works OK since >> > large allocations are normally not done in EFI, so memory isn't really >> > consumed to any great degree by the boot process. >> > >> > What happens to EFI allocations if the app returns? They are still >> > present, in case another app is run. This seems fine. >> > >> > API >> > –-- >> > Can we unify some APIs? >> > >> > It should be possible to use lmb for large EFI memory allocations, so >> > long as they are only needed for booting. We effectively do this >> > today, since EFI does not manage the arrangement of loaded images in >> > memory. for the most part. >> > >> > It would not make sense to use EFI allocation to replace lmb and >> > malloc(), of course. >> > >> > Could we use a common (lower-level) API for allocation, used by both >> > lmb and EFI? They do have some similarities. However they have >> > different lifetime constraints (EFI allocations are never dropped, >> > unlikely lmb). >> > >> > ** Overall, it seems that the existence of memory allocation in >> > boot-time services has created confusion. Memory allocation is >> > muddled, with both U-Boot code and boot-time services calling the same >> > memory allocator. This just has not been clearly thought out. >> > >> > >> > Proposal >> > —------- >> > >> > Here are some ideas: >> > >> > 1. For video, use the driver model API to locate the video regions, or >> > block off the entire framebuffer memory, for all devices as a whole. >> > Use efi_add_memory_map() >> > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. >> > >> > 3. Add all EFI reservations just before booting the app, as we do with >> > devicetree fixup. With this model, malloc() and lmb are used for all >> > allocation. Then efi_add_memory_map() is called for each region in >> > turn just before booting. Memory attributes are dealt with above. The >> > type (enum efi_memory_type) can be determined simply by the data >> > structure stored in it, as is done today. For example, SMBIOS tables >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code >> > understands the meaning of each. >> > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when >> > booting. This looks to require very little effort. >> > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside >> > boot-time services. This solves the problem 6. If memory is needed by >> > an app, allocate it with malloc() and see 3. There are only two >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of >> > efi_allocate_pool(), but most of these seem easy to fix up. For >> > example, efi_init_event_log() allocates a buffer, but this can be >> > allocated in normal malloc() space or in a bloblist. >> > >> > 6. Don't worry too much about whether EFI will be used for booting. >> > The cost is likely not that great: use bootstage to measure it as is >> > done for driver model. Try to minmise the cost of its tables, >> > particularly for execution time, but otherwise just rely on the >> > ability to disable EFI_LOADER. >> >> 7. Add a flag to the 'load' command: >> >> -m <type> - make an lmb allocation for the file >> <type> is the image type to use (kernel, ramdisk, flat_dt) >> >> any existing allocation for that type will be automatically freed >> first. If <type> is "none" then no freeing is possible: any loaded >> images just stack up in lmb. >> >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > >I would really not like to change the user interface and instead simply >handle this with flags to whatever mark/allocation function is called. >You can always overwrite things that are brought in to memory, you >cannot overwrite U-Boot or our internals. Optionally noting that some >previous load to memory has been at least partially overwritten could be >helpful, if it's not too much extra logic. >
In most use cases users load exactly one file at each address. An unload command would be the cleanest way for a user to indicate that he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
I'm not sure if we still want the proposed lmb command? Perhaps it is not needed.
Please lets not call it "lmb" because the current lmb isn't what we want anymore and it's otherwise probably leading to some of the confusion. The kernel evolved it to memblocks, so perhaps that terminology too here. And yes, a command for if nothing else debug / investigation would be good.
OK, I would prefer memblock too.
Regards, Simon
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote:
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote:
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote:
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote:
Date: Fri, 29 Dec 2023 11:17:44 -0500 From: Tom Rini trini@konsulko.com
On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > >> Hi, > >> > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > >> > > >> > Hi, > >> > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > >> > memory allocation in U-Boot. > >> > > >> > 1. malloc() > >> > > >> > malloc() is used for programmatic memory allocation. It allows memory > >> > to be freed. It is not designed for very large allocations (e.g. a > >> > 10MB kernel or 100MB ramdisk). > >> > > >> > 2. lmb > >> > > >> > lmb is used for large blocks of memory, such as those needed for a > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > >> > loading some images and booting. If the boot fails, then all lmb > >> > allocations go away. > >> > > >> > lmb is set up by getting all available memory and then removing what > >> > is used by U-Boot (code, data, malloc() space, etc.) > >> > > >> > lmb reservations have a few flags so that areas of memory can be > >> > provided with attributes > >> > > >> > There are some corner cases...e.g. loading a file does an lmb > >> > allocation but only for the purpose of avoiding a file being loaded > >> > over U-Boot code/data. The allocation is dropped immediately after the > >> > file is loaded. Within the bootm command, or when using standard boot, > >> > this would be fairly easy to solve. > >> > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > >> > > >> > 3. EFI > >> > > >> > EFI has its own memory-allocation tables. > >> > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > >> > function it can also do smaller allocations similar to malloc(), > >> > although each one uses at least 4KB at present. > >> > > >> > EFI allocations do not go away when a boot fails. > >> > > >> > With EFI it is possible to add allocations post facto, in which case > >> > they are added to the allocation table just as if the memory was > >> > allocated with EFI to begin with. > >> > > >> > The EFI allocations and the lmb allocations use the same memory, so in > >> > principle could conflict. > >> > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > >> > well, if needed by the EFI app. For example, while efi_image_parse() > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > >> > in the app context and may need to access the memory after U-Boot has > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > >> > mapping as well. > >> > > >> > EFI memory has attributes, including what the memory is used for (to > >> > some degree of granularity). See enum efi_memory_type and struct > >> > efi_mem_desc. In the latter there are also attribute flags - whether > >> > memory is cacheable, etc. > >> > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > >> > believe) that below 4GB that isn't reserved for the hardware/system. > >> > This is meaningless, or at least confusing, on ARM systems. > >> > > >> > 4. reservations > >> > > >> > It is perhaps worth mentioning a fourth method of memory management, > >> > where U-Boot reserves chunks of memory before relocation (in > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > >> > region, etc. > >> > > >> > > >> > Problems > >> > —------- > >> > > >> > There are no urgent problems, but here are some things that could be improved: > >> > > >> > 1. EFI should attach most of its data structures to driver model. This > >> > work has started, with the partition support, but more effort would > >> > help. This would make it easier to see which memory is related to > >> > devices and which is separate. > >> > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > >> > booting or not (e.g. rockchip video rk_vop_probe()). > >> > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > >> > armv8's struct mm_region) to EFI ones. > >> > > >> > 4. EFI duplicates some code from bootm, some of which relates to > >> > memory allocation (e.g. FDT fixup). > >> > > >> > 5. EFI code is used even if EFI is never used to boot > >> > > >> > 6. EFI allocations can result in the same memory being used as has > >> > already been allocated by lmb. Users may load files which overwrite > >> > memory allocated by EFI. > >> > >> 7. We need to support doing an allocation when a file is loaded (to > >> ensure files do not overlap), without making it too difficult to load > >> multiple files to the same place, if desired. > >> > >> > > >> > > >> > Lifetime > >> > -------- > >> > > >> > We have three different memory allocators with different purposes. Can > >> > we unify them a little? > >> > > >> > Within U-Boot: > >> > - malloc() space lives forever > >> > - lmb lives while setting out images for booting > >> > - EFI (mostly) lives while booting an EFI app > >> > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > >> > some not. EFI allocations stay around forever. This works OK since > >> > large allocations are normally not done in EFI, so memory isn't really > >> > consumed to any great degree by the boot process. > >> > > >> > What happens to EFI allocations if the app returns? They are still > >> > present, in case another app is run. This seems fine. > >> > > >> > API > >> > –-- > >> > Can we unify some APIs? > >> > > >> > It should be possible to use lmb for large EFI memory allocations, so > >> > long as they are only needed for booting. We effectively do this > >> > today, since EFI does not manage the arrangement of loaded images in > >> > memory. for the most part. > >> > > >> > It would not make sense to use EFI allocation to replace lmb and > >> > malloc(), of course. > >> > > >> > Could we use a common (lower-level) API for allocation, used by both > >> > lmb and EFI? They do have some similarities. However they have > >> > different lifetime constraints (EFI allocations are never dropped, > >> > unlikely lmb). > >> > > >> > ** Overall, it seems that the existence of memory allocation in > >> > boot-time services has created confusion. Memory allocation is > >> > muddled, with both U-Boot code and boot-time services calling the same > >> > memory allocator. This just has not been clearly thought out. > >> > > >> > > >> > Proposal > >> > —------- > >> > > >> > Here are some ideas: > >> > > >> > 1. For video, use the driver model API to locate the video regions, or > >> > block off the entire framebuffer memory, for all devices as a whole. > >> > Use efi_add_memory_map() > >> > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > >> > > >> > 3. Add all EFI reservations just before booting the app, as we do with > >> > devicetree fixup. With this model, malloc() and lmb are used for all > >> > allocation. Then efi_add_memory_map() is called for each region in > >> > turn just before booting. Memory attributes are dealt with above. The > >> > type (enum efi_memory_type) can be determined simply by the data > >> > structure stored in it, as is done today. For example, SMBIOS tables > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > >> > understands the meaning of each. > >> > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > >> > booting. This looks to require very little effort. > >> > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > >> > boot-time services. This solves the problem 6. If memory is needed by > >> > an app, allocate it with malloc() and see 3. There are only two > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > >> > example, efi_init_event_log() allocates a buffer, but this can be > >> > allocated in normal malloc() space or in a bloblist. > >> > > >> > 6. Don't worry too much about whether EFI will be used for booting. > >> > The cost is likely not that great: use bootstage to measure it as is > >> > done for driver model. Try to minmise the cost of its tables, > >> > particularly for execution time, but otherwise just rely on the > >> > ability to disable EFI_LOADER. > >> > >> 7. Add a flag to the 'load' command: > >> > >> -m <type> - make an lmb allocation for the file > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > >> > >> any existing allocation for that type will be automatically freed > >> first. If <type> is "none" then no freeing is possible: any loaded > >> images just stack up in lmb. > >> > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > >I would really not like to change the user interface and instead simply > >handle this with flags to whatever mark/allocation function is called. > >You can always overwrite things that are brought in to memory, you > >cannot overwrite U-Boot or our internals. Optionally noting that some > >previous load to memory has been at least partially overwritten could be > >helpful, if it's not too much extra logic. > > > > In most use cases users load exactly one file at each address. An > unload command would be the cleanest way for a user to indicate that > he wants to reuse the memory.
I very much do not want to change the API. There's untold numbers of scripts out there and they should continue to work. I mentioned to Ilias off list just now that I'm not against adding a command to add flags to these areas, but I don't think it's worthwhile to prevent overwrites the user did early. The biggest long running problem in this space was that for 32bit ARM we couldn't know where the kernel BSS was going to be and so would have ramdisk at the wrong spot and get partially eaten, and this was hard to figure out. The current example is "ooops, decompression buffer for Image.gz/etc is too close to other things" which ends up failing nice and loudly, and in the future once this proposal is done we can just dynamically find and use a spot, since we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote:
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote:
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote:
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote:
> Date: Fri, 29 Dec 2023 11:17:44 -0500 > From: Tom Rini trini@konsulko.com > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > >> Hi, > > >> > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > >> > > > >> > Hi, > > >> > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > >> > memory allocation in U-Boot. > > >> > > > >> > 1. malloc() > > >> > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > >> > to be freed. It is not designed for very large allocations (e.g. a > > >> > 10MB kernel or 100MB ramdisk). > > >> > > > >> > 2. lmb > > >> > > > >> > lmb is used for large blocks of memory, such as those needed for a > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > >> > loading some images and booting. If the boot fails, then all lmb > > >> > allocations go away. > > >> > > > >> > lmb is set up by getting all available memory and then removing what > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > >> > > > >> > lmb reservations have a few flags so that areas of memory can be > > >> > provided with attributes > > >> > > > >> > There are some corner cases...e.g. loading a file does an lmb > > >> > allocation but only for the purpose of avoiding a file being loaded > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > >> > file is loaded. Within the bootm command, or when using standard boot, > > >> > this would be fairly easy to solve. > > >> > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > >> > > > >> > 3. EFI > > >> > > > >> > EFI has its own memory-allocation tables. > > >> > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > >> > function it can also do smaller allocations similar to malloc(), > > >> > although each one uses at least 4KB at present. > > >> > > > >> > EFI allocations do not go away when a boot fails. > > >> > > > >> > With EFI it is possible to add allocations post facto, in which case > > >> > they are added to the allocation table just as if the memory was > > >> > allocated with EFI to begin with. > > >> > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > >> > principle could conflict. > > >> > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > >> > in the app context and may need to access the memory after U-Boot has > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > >> > mapping as well. > > >> > > > >> > EFI memory has attributes, including what the memory is used for (to > > >> > some degree of granularity). See enum efi_memory_type and struct > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > >> > memory is cacheable, etc. > > >> > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > >> > This is meaningless, or at least confusing, on ARM systems. > > >> > > > >> > 4. reservations > > >> > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > >> > where U-Boot reserves chunks of memory before relocation (in > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > >> > region, etc. > > >> > > > >> > > > >> > Problems > > >> > —------- > > >> > > > >> > There are no urgent problems, but here are some things that could be improved: > > >> > > > >> > 1. EFI should attach most of its data structures to driver model. This > > >> > work has started, with the partition support, but more effort would > > >> > help. This would make it easier to see which memory is related to > > >> > devices and which is separate. > > >> > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > >> > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > >> > armv8's struct mm_region) to EFI ones. > > >> > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > >> > memory allocation (e.g. FDT fixup). > > >> > > > >> > 5. EFI code is used even if EFI is never used to boot > > >> > > > >> > 6. EFI allocations can result in the same memory being used as has > > >> > already been allocated by lmb. Users may load files which overwrite > > >> > memory allocated by EFI. > > >> > > >> 7. We need to support doing an allocation when a file is loaded (to > > >> ensure files do not overlap), without making it too difficult to load > > >> multiple files to the same place, if desired. > > >> > > >> > > > >> > > > >> > Lifetime > > >> > -------- > > >> > > > >> > We have three different memory allocators with different purposes. Can > > >> > we unify them a little? > > >> > > > >> > Within U-Boot: > > >> > - malloc() space lives forever > > >> > - lmb lives while setting out images for booting > > >> > - EFI (mostly) lives while booting an EFI app > > >> > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > >> > some not. EFI allocations stay around forever. This works OK since > > >> > large allocations are normally not done in EFI, so memory isn't really > > >> > consumed to any great degree by the boot process. > > >> > > > >> > What happens to EFI allocations if the app returns? They are still > > >> > present, in case another app is run. This seems fine. > > >> > > > >> > API > > >> > –-- > > >> > Can we unify some APIs? > > >> > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > >> > long as they are only needed for booting. We effectively do this > > >> > today, since EFI does not manage the arrangement of loaded images in > > >> > memory. for the most part. > > >> > > > >> > It would not make sense to use EFI allocation to replace lmb and > > >> > malloc(), of course. > > >> > > > >> > Could we use a common (lower-level) API for allocation, used by both > > >> > lmb and EFI? They do have some similarities. However they have > > >> > different lifetime constraints (EFI allocations are never dropped, > > >> > unlikely lmb). > > >> > > > >> > ** Overall, it seems that the existence of memory allocation in > > >> > boot-time services has created confusion. Memory allocation is > > >> > muddled, with both U-Boot code and boot-time services calling the same > > >> > memory allocator. This just has not been clearly thought out. > > >> > > > >> > > > >> > Proposal > > >> > —------- > > >> > > > >> > Here are some ideas: > > >> > > > >> > 1. For video, use the driver model API to locate the video regions, or > > >> > block off the entire framebuffer memory, for all devices as a whole. > > >> > Use efi_add_memory_map() > > >> > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > >> > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > >> > allocation. Then efi_add_memory_map() is called for each region in > > >> > turn just before booting. Memory attributes are dealt with above. The > > >> > type (enum efi_memory_type) can be determined simply by the data > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > >> > understands the meaning of each. > > >> > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > >> > booting. This looks to require very little effort. > > >> > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > >> > boot-time services. This solves the problem 6. If memory is needed by > > >> > an app, allocate it with malloc() and see 3. There are only two > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > >> > allocated in normal malloc() space or in a bloblist. > > >> > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > >> > The cost is likely not that great: use bootstage to measure it as is > > >> > done for driver model. Try to minmise the cost of its tables, > > >> > particularly for execution time, but otherwise just rely on the > > >> > ability to disable EFI_LOADER. > > >> > > >> 7. Add a flag to the 'load' command: > > >> > > >> -m <type> - make an lmb allocation for the file > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > >> > > >> any existing allocation for that type will be automatically freed > > >> first. If <type> is "none" then no freeing is possible: any loaded > > >> images just stack up in lmb. > > >> > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > >I would really not like to change the user interface and instead simply > > >handle this with flags to whatever mark/allocation function is called. > > >You can always overwrite things that are brought in to memory, you > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > >previous load to memory has been at least partially overwritten could be > > >helpful, if it's not too much extra logic. > > > > > > > In most use cases users load exactly one file at each address. An > > unload command would be the cleanest way for a user to indicate that > > he wants to reuse the memory. > > I very much do not want to change the API. There's untold numbers of > scripts out there and they should continue to work. I mentioned to Ilias > off list just now that I'm not against adding a command to add flags to > these areas, but I don't think it's worthwhile to prevent overwrites the > user did early. The biggest long running problem in this space was that > for 32bit ARM we couldn't know where the kernel BSS was going to be and > so would have ramdisk at the wrong spot and get partially eaten, and > this was hard to figure out. The current example is "ooops, > decompression buffer for Image.gz/etc is too close to other things" > which ends up failing nice and loudly, and in the future once this > proposal is done we can just dynamically find and use a spot, since > we'll have that ability finally.
In order to keep the existing interfaces we need lmb to keep track of (at least ) three different states. I think of those as "free", "allocated" and "reserved". The load command would "reserve" memory insteaf "allocate". And it would pass a flag to lmb when reserving memory to indicate that reserving memory that is already reserved is ok. Both "reserved" and "allocated" memory should show up as not free in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
Best regards
Heinrich
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote:
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote:
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote:
On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > From: Tom Rini trini@konsulko.com > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > > >> Hi, > > > >> > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > >> > > > > >> > Hi, > > > >> > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > > >> > memory allocation in U-Boot. > > > >> > > > > >> > 1. malloc() > > > >> > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > > >> > 10MB kernel or 100MB ramdisk). > > > >> > > > > >> > 2. lmb > > > >> > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > > >> > loading some images and booting. If the boot fails, then all lmb > > > >> > allocations go away. > > > >> > > > > >> > lmb is set up by getting all available memory and then removing what > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > > >> > > > > >> > lmb reservations have a few flags so that areas of memory can be > > > >> > provided with attributes > > > >> > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > > >> > allocation but only for the purpose of avoiding a file being loaded > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > > >> > this would be fairly easy to solve. > > > >> > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > > >> > > > > >> > 3. EFI > > > >> > > > > >> > EFI has its own memory-allocation tables. > > > >> > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > > >> > function it can also do smaller allocations similar to malloc(), > > > >> > although each one uses at least 4KB at present. > > > >> > > > > >> > EFI allocations do not go away when a boot fails. > > > >> > > > > >> > With EFI it is possible to add allocations post facto, in which case > > > >> > they are added to the allocation table just as if the memory was > > > >> > allocated with EFI to begin with. > > > >> > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > > >> > principle could conflict. > > > >> > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > > >> > in the app context and may need to access the memory after U-Boot has > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > > >> > mapping as well. > > > >> > > > > >> > EFI memory has attributes, including what the memory is used for (to > > > >> > some degree of granularity). See enum efi_memory_type and struct > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > > >> > memory is cacheable, etc. > > > >> > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > > >> > This is meaningless, or at least confusing, on ARM systems. > > > >> > > > > >> > 4. reservations > > > >> > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > > >> > where U-Boot reserves chunks of memory before relocation (in > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > > >> > region, etc. > > > >> > > > > >> > > > > >> > Problems > > > >> > —------- > > > >> > > > > >> > There are no urgent problems, but here are some things that could be improved: > > > >> > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > > >> > work has started, with the partition support, but more effort would > > > >> > help. This would make it easier to see which memory is related to > > > >> > devices and which is separate. > > > >> > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > > >> > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > > >> > armv8's struct mm_region) to EFI ones. > > > >> > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > > >> > memory allocation (e.g. FDT fixup). > > > >> > > > > >> > 5. EFI code is used even if EFI is never used to boot > > > >> > > > > >> > 6. EFI allocations can result in the same memory being used as has > > > >> > already been allocated by lmb. Users may load files which overwrite > > > >> > memory allocated by EFI. > > > >> > > > >> 7. We need to support doing an allocation when a file is loaded (to > > > >> ensure files do not overlap), without making it too difficult to load > > > >> multiple files to the same place, if desired. > > > >> > > > >> > > > > >> > > > > >> > Lifetime > > > >> > -------- > > > >> > > > > >> > We have three different memory allocators with different purposes. Can > > > >> > we unify them a little? > > > >> > > > > >> > Within U-Boot: > > > >> > - malloc() space lives forever > > > >> > - lmb lives while setting out images for booting > > > >> > - EFI (mostly) lives while booting an EFI app > > > >> > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > > >> > some not. EFI allocations stay around forever. This works OK since > > > >> > large allocations are normally not done in EFI, so memory isn't really > > > >> > consumed to any great degree by the boot process. > > > >> > > > > >> > What happens to EFI allocations if the app returns? They are still > > > >> > present, in case another app is run. This seems fine. > > > >> > > > > >> > API > > > >> > –-- > > > >> > Can we unify some APIs? > > > >> > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > > >> > long as they are only needed for booting. We effectively do this > > > >> > today, since EFI does not manage the arrangement of loaded images in > > > >> > memory. for the most part. > > > >> > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > > >> > malloc(), of course. > > > >> > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > > >> > lmb and EFI? They do have some similarities. However they have > > > >> > different lifetime constraints (EFI allocations are never dropped, > > > >> > unlikely lmb). > > > >> > > > > >> > ** Overall, it seems that the existence of memory allocation in > > > >> > boot-time services has created confusion. Memory allocation is > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > > >> > memory allocator. This just has not been clearly thought out. > > > >> > > > > >> > > > > >> > Proposal > > > >> > —------- > > > >> > > > > >> > Here are some ideas: > > > >> > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > > >> > Use efi_add_memory_map() > > > >> > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > >> > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > > >> > turn just before booting. Memory attributes are dealt with above. The > > > >> > type (enum efi_memory_type) can be determined simply by the data > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > > >> > understands the meaning of each. > > > >> > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > > >> > booting. This looks to require very little effort. > > > >> > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > > >> > an app, allocate it with malloc() and see 3. There are only two > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > > >> > allocated in normal malloc() space or in a bloblist. > > > >> > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > > >> > The cost is likely not that great: use bootstage to measure it as is > > > >> > done for driver model. Try to minmise the cost of its tables, > > > >> > particularly for execution time, but otherwise just rely on the > > > >> > ability to disable EFI_LOADER. > > > >> > > > >> 7. Add a flag to the 'load' command: > > > >> > > > >> -m <type> - make an lmb allocation for the file > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > > >> > > > >> any existing allocation for that type will be automatically freed > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > > >> images just stack up in lmb. > > > >> > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > > > >I would really not like to change the user interface and instead simply > > > >handle this with flags to whatever mark/allocation function is called. > > > >You can always overwrite things that are brought in to memory, you > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > > >previous load to memory has been at least partially overwritten could be > > > >helpful, if it's not too much extra logic. > > > > > > > > > > In most use cases users load exactly one file at each address. An > > > unload command would be the cleanest way for a user to indicate that > > > he wants to reuse the memory. > > > > I very much do not want to change the API. There's untold numbers of > > scripts out there and they should continue to work. I mentioned to Ilias > > off list just now that I'm not against adding a command to add flags to > > these areas, but I don't think it's worthwhile to prevent overwrites the > > user did early. The biggest long running problem in this space was that > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > so would have ramdisk at the wrong spot and get partially eaten, and > > this was hard to figure out. The current example is "ooops, > > decompression buffer for Image.gz/etc is too close to other things" > > which ends up failing nice and loudly, and in the future once this > > proposal is done we can just dynamically find and use a spot, since > > we'll have that ability finally. > > In order to keep the existing interfaces we need lmb to keep track of > (at least ) three different states. I think of those as "free", > "allocated" and "reserved". The load command would "reserve" memory > insteaf "allocate". And it would pass a flag to lmb when reserving > memory to indicate that reserving memory that is already reserved is > ok. Both "reserved" and "allocated" memory should show up as not free > in the EFI memory map (probably as EfiLoaderData).
Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be: - Free. - Readable but not Writable. - Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote:
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote:
Hi,
On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > > From: Tom Rini trini@konsulko.com > > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > > > >> Hi, > > > > >> > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > > >> > > > > > >> > Hi, > > > > >> > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > > > >> > memory allocation in U-Boot. > > > > >> > > > > > >> > 1. malloc() > > > > >> > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > > > >> > 10MB kernel or 100MB ramdisk). > > > > >> > > > > > >> > 2. lmb > > > > >> > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > > > >> > loading some images and booting. If the boot fails, then all lmb > > > > >> > allocations go away. > > > > >> > > > > > >> > lmb is set up by getting all available memory and then removing what > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > > > >> > > > > > >> > lmb reservations have a few flags so that areas of memory can be > > > > >> > provided with attributes > > > > >> > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > > > >> > allocation but only for the purpose of avoiding a file being loaded > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > > > >> > this would be fairly easy to solve. > > > > >> > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > > > >> > > > > > >> > 3. EFI > > > > >> > > > > > >> > EFI has its own memory-allocation tables. > > > > >> > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > > > >> > function it can also do smaller allocations similar to malloc(), > > > > >> > although each one uses at least 4KB at present. > > > > >> > > > > > >> > EFI allocations do not go away when a boot fails. > > > > >> > > > > > >> > With EFI it is possible to add allocations post facto, in which case > > > > >> > they are added to the allocation table just as if the memory was > > > > >> > allocated with EFI to begin with. > > > > >> > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > > > >> > principle could conflict. > > > > >> > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > > > >> > in the app context and may need to access the memory after U-Boot has > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > > > >> > mapping as well. > > > > >> > > > > > >> > EFI memory has attributes, including what the memory is used for (to > > > > >> > some degree of granularity). See enum efi_memory_type and struct > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > > > >> > memory is cacheable, etc. > > > > >> > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > > > >> > This is meaningless, or at least confusing, on ARM systems. > > > > >> > > > > > >> > 4. reservations > > > > >> > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > > > >> > where U-Boot reserves chunks of memory before relocation (in > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > > > >> > region, etc. > > > > >> > > > > > >> > > > > > >> > Problems > > > > >> > —------- > > > > >> > > > > > >> > There are no urgent problems, but here are some things that could be improved: > > > > >> > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > > > >> > work has started, with the partition support, but more effort would > > > > >> > help. This would make it easier to see which memory is related to > > > > >> > devices and which is separate. > > > > >> > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > > > >> > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > > > >> > armv8's struct mm_region) to EFI ones. > > > > >> > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > > > >> > memory allocation (e.g. FDT fixup). > > > > >> > > > > > >> > 5. EFI code is used even if EFI is never used to boot > > > > >> > > > > > >> > 6. EFI allocations can result in the same memory being used as has > > > > >> > already been allocated by lmb. Users may load files which overwrite > > > > >> > memory allocated by EFI. > > > > >> > > > > >> 7. We need to support doing an allocation when a file is loaded (to > > > > >> ensure files do not overlap), without making it too difficult to load > > > > >> multiple files to the same place, if desired. > > > > >> > > > > >> > > > > > >> > > > > > >> > Lifetime > > > > >> > -------- > > > > >> > > > > > >> > We have three different memory allocators with different purposes. Can > > > > >> > we unify them a little? > > > > >> > > > > > >> > Within U-Boot: > > > > >> > - malloc() space lives forever > > > > >> > - lmb lives while setting out images for booting > > > > >> > - EFI (mostly) lives while booting an EFI app > > > > >> > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > > > >> > some not. EFI allocations stay around forever. This works OK since > > > > >> > large allocations are normally not done in EFI, so memory isn't really > > > > >> > consumed to any great degree by the boot process. > > > > >> > > > > > >> > What happens to EFI allocations if the app returns? They are still > > > > >> > present, in case another app is run. This seems fine. > > > > >> > > > > > >> > API > > > > >> > –-- > > > > >> > Can we unify some APIs? > > > > >> > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > > > >> > long as they are only needed for booting. We effectively do this > > > > >> > today, since EFI does not manage the arrangement of loaded images in > > > > >> > memory. for the most part. > > > > >> > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > > > >> > malloc(), of course. > > > > >> > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > > > >> > lmb and EFI? They do have some similarities. However they have > > > > >> > different lifetime constraints (EFI allocations are never dropped, > > > > >> > unlikely lmb). > > > > >> > > > > > >> > ** Overall, it seems that the existence of memory allocation in > > > > >> > boot-time services has created confusion. Memory allocation is > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > > > >> > memory allocator. This just has not been clearly thought out. > > > > >> > > > > > >> > > > > > >> > Proposal > > > > >> > —------- > > > > >> > > > > > >> > Here are some ideas: > > > > >> > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > > > >> > Use efi_add_memory_map() > > > > >> > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > > >> > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > > > >> > turn just before booting. Memory attributes are dealt with above. The > > > > >> > type (enum efi_memory_type) can be determined simply by the data > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > > > >> > understands the meaning of each. > > > > >> > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > > > >> > booting. This looks to require very little effort. > > > > >> > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > > > >> > an app, allocate it with malloc() and see 3. There are only two > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > > > >> > allocated in normal malloc() space or in a bloblist. > > > > >> > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > > > >> > The cost is likely not that great: use bootstage to measure it as is > > > > >> > done for driver model. Try to minmise the cost of its tables, > > > > >> > particularly for execution time, but otherwise just rely on the > > > > >> > ability to disable EFI_LOADER. > > > > >> > > > > >> 7. Add a flag to the 'load' command: > > > > >> > > > > >> -m <type> - make an lmb allocation for the file > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > > > >> > > > > >> any existing allocation for that type will be automatically freed > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > > > >> images just stack up in lmb. > > > > >> > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > > > > > >I would really not like to change the user interface and instead simply > > > > >handle this with flags to whatever mark/allocation function is called. > > > > >You can always overwrite things that are brought in to memory, you > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > > > >previous load to memory has been at least partially overwritten could be > > > > >helpful, if it's not too much extra logic. > > > > > > > > > > > > > In most use cases users load exactly one file at each address. An > > > > unload command would be the cleanest way for a user to indicate that > > > > he wants to reuse the memory. > > > > > > I very much do not want to change the API. There's untold numbers of > > > scripts out there and they should continue to work. I mentioned to Ilias > > > off list just now that I'm not against adding a command to add flags to > > > these areas, but I don't think it's worthwhile to prevent overwrites the > > > user did early. The biggest long running problem in this space was that > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > > so would have ramdisk at the wrong spot and get partially eaten, and > > > this was hard to figure out. The current example is "ooops, > > > decompression buffer for Image.gz/etc is too close to other things" > > > which ends up failing nice and loudly, and in the future once this > > > proposal is done we can just dynamically find and use a spot, since > > > we'll have that ability finally. > > > > In order to keep the existing interfaces we need lmb to keep track of > > (at least ) three different states. I think of those as "free", > > "allocated" and "reserved". The load command would "reserve" memory > > insteaf "allocate". And it would pass a flag to lmb when reserving > > memory to indicate that reserving memory that is already reserved is > > ok. Both "reserved" and "allocated" memory should show up as not free > > in the EFI memory map (probably as EfiLoaderData). > > Yes, something like this is what I was getting at, thanks.
Yes, that is a good way of putting it. There is definitely a distinction there.
If we don't want this flag, we could make U-Boot always do a reservation on load, with a '-f' command to force loading over an existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
-- Tom
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote:
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: > Hi, > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > > > From: Tom Rini trini@konsulko.com > > > > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > > > > >> Hi, > > > > > >> > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > > > >> > > > > > > >> > Hi, > > > > > >> > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > > > > >> > memory allocation in U-Boot. > > > > > >> > > > > > > >> > 1. malloc() > > > > > >> > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > > > > >> > 10MB kernel or 100MB ramdisk). > > > > > >> > > > > > > >> > 2. lmb > > > > > >> > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > > > > >> > loading some images and booting. If the boot fails, then all lmb > > > > > >> > allocations go away. > > > > > >> > > > > > > >> > lmb is set up by getting all available memory and then removing what > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > > > > >> > > > > > > >> > lmb reservations have a few flags so that areas of memory can be > > > > > >> > provided with attributes > > > > > >> > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > > > > >> > allocation but only for the purpose of avoiding a file being loaded > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > > > > >> > this would be fairly easy to solve. > > > > > >> > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > > > > >> > > > > > > >> > 3. EFI > > > > > >> > > > > > > >> > EFI has its own memory-allocation tables. > > > > > >> > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > > > > >> > function it can also do smaller allocations similar to malloc(), > > > > > >> > although each one uses at least 4KB at present. > > > > > >> > > > > > > >> > EFI allocations do not go away when a boot fails. > > > > > >> > > > > > > >> > With EFI it is possible to add allocations post facto, in which case > > > > > >> > they are added to the allocation table just as if the memory was > > > > > >> > allocated with EFI to begin with. > > > > > >> > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > > > > >> > principle could conflict. > > > > > >> > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > > > > >> > in the app context and may need to access the memory after U-Boot has > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > > > > >> > mapping as well. > > > > > >> > > > > > > >> > EFI memory has attributes, including what the memory is used for (to > > > > > >> > some degree of granularity). See enum efi_memory_type and struct > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > > > > >> > memory is cacheable, etc. > > > > > >> > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > > > > >> > This is meaningless, or at least confusing, on ARM systems. > > > > > >> > > > > > > >> > 4. reservations > > > > > >> > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > > > > >> > where U-Boot reserves chunks of memory before relocation (in > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > > > > >> > region, etc. > > > > > >> > > > > > > >> > > > > > > >> > Problems > > > > > >> > —------- > > > > > >> > > > > > > >> > There are no urgent problems, but here are some things that could be improved: > > > > > >> > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > > > > >> > work has started, with the partition support, but more effort would > > > > > >> > help. This would make it easier to see which memory is related to > > > > > >> > devices and which is separate. > > > > > >> > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > > > > >> > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > > > > >> > armv8's struct mm_region) to EFI ones. > > > > > >> > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > > > > >> > memory allocation (e.g. FDT fixup). > > > > > >> > > > > > > >> > 5. EFI code is used even if EFI is never used to boot > > > > > >> > > > > > > >> > 6. EFI allocations can result in the same memory being used as has > > > > > >> > already been allocated by lmb. Users may load files which overwrite > > > > > >> > memory allocated by EFI. > > > > > >> > > > > > >> 7. We need to support doing an allocation when a file is loaded (to > > > > > >> ensure files do not overlap), without making it too difficult to load > > > > > >> multiple files to the same place, if desired. > > > > > >> > > > > > >> > > > > > > >> > > > > > > >> > Lifetime > > > > > >> > -------- > > > > > >> > > > > > > >> > We have three different memory allocators with different purposes. Can > > > > > >> > we unify them a little? > > > > > >> > > > > > > >> > Within U-Boot: > > > > > >> > - malloc() space lives forever > > > > > >> > - lmb lives while setting out images for booting > > > > > >> > - EFI (mostly) lives while booting an EFI app > > > > > >> > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > > > > >> > some not. EFI allocations stay around forever. This works OK since > > > > > >> > large allocations are normally not done in EFI, so memory isn't really > > > > > >> > consumed to any great degree by the boot process. > > > > > >> > > > > > > >> > What happens to EFI allocations if the app returns? They are still > > > > > >> > present, in case another app is run. This seems fine. > > > > > >> > > > > > > >> > API > > > > > >> > –-- > > > > > >> > Can we unify some APIs? > > > > > >> > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > > > > >> > long as they are only needed for booting. We effectively do this > > > > > >> > today, since EFI does not manage the arrangement of loaded images in > > > > > >> > memory. for the most part. > > > > > >> > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > > > > >> > malloc(), of course. > > > > > >> > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > > > > >> > lmb and EFI? They do have some similarities. However they have > > > > > >> > different lifetime constraints (EFI allocations are never dropped, > > > > > >> > unlikely lmb). > > > > > >> > > > > > > >> > ** Overall, it seems that the existence of memory allocation in > > > > > >> > boot-time services has created confusion. Memory allocation is > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > > > > >> > memory allocator. This just has not been clearly thought out. > > > > > >> > > > > > > >> > > > > > > >> > Proposal > > > > > >> > —------- > > > > > >> > > > > > > >> > Here are some ideas: > > > > > >> > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > > > > >> > Use efi_add_memory_map() > > > > > >> > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > > > >> > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > > > > >> > turn just before booting. Memory attributes are dealt with above. The > > > > > >> > type (enum efi_memory_type) can be determined simply by the data > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > > > > >> > understands the meaning of each. > > > > > >> > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > > > > >> > booting. This looks to require very little effort. > > > > > >> > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > > > > >> > an app, allocate it with malloc() and see 3. There are only two > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > > > > >> > allocated in normal malloc() space or in a bloblist. > > > > > >> > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > > > > >> > The cost is likely not that great: use bootstage to measure it as is > > > > > >> > done for driver model. Try to minmise the cost of its tables, > > > > > >> > particularly for execution time, but otherwise just rely on the > > > > > >> > ability to disable EFI_LOADER. > > > > > >> > > > > > >> 7. Add a flag to the 'load' command: > > > > > >> > > > > > >> -m <type> - make an lmb allocation for the file > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > > > > >> > > > > > >> any existing allocation for that type will be automatically freed > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > > > > >> images just stack up in lmb. > > > > > >> > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > > > > > > > >I would really not like to change the user interface and instead simply > > > > > >handle this with flags to whatever mark/allocation function is called. > > > > > >You can always overwrite things that are brought in to memory, you > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > > > > >previous load to memory has been at least partially overwritten could be > > > > > >helpful, if it's not too much extra logic. > > > > > > > > > > > > > > > > In most use cases users load exactly one file at each address. An > > > > > unload command would be the cleanest way for a user to indicate that > > > > > he wants to reuse the memory. > > > > > > > > I very much do not want to change the API. There's untold numbers of > > > > scripts out there and they should continue to work. I mentioned to Ilias > > > > off list just now that I'm not against adding a command to add flags to > > > > these areas, but I don't think it's worthwhile to prevent overwrites the > > > > user did early. The biggest long running problem in this space was that > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > > > so would have ramdisk at the wrong spot and get partially eaten, and > > > > this was hard to figure out. The current example is "ooops, > > > > decompression buffer for Image.gz/etc is too close to other things" > > > > which ends up failing nice and loudly, and in the future once this > > > > proposal is done we can just dynamically find and use a spot, since > > > > we'll have that ability finally. > > > > > > In order to keep the existing interfaces we need lmb to keep track of > > > (at least ) three different states. I think of those as "free", > > > "allocated" and "reserved". The load command would "reserve" memory > > > insteaf "allocate". And it would pass a flag to lmb when reserving > > > memory to indicate that reserving memory that is already reserved is > > > ok. Both "reserved" and "allocated" memory should show up as not free > > > in the EFI memory map (probably as EfiLoaderData). > > > > Yes, something like this is what I was getting at, thanks. > > Yes, that is a good way of putting it. There is definitely a distinction there. > > If we don't want this flag, we could make U-Boot always do a > reservation on load, with a '-f' command to force loading over an > existing reservation / releasing it first?
Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
-sughosh
[1] - https://docs.google.com/document/d/1LJj4f1oBaxPIALnMlIy8Bf-3pESjecTyKhrQoeCf...
-- Tom
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote:
Hi Tom,
On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: > > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: > > Hi, > > > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > > > > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > > > > From: Tom Rini trini@konsulko.com > > > > > > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > > > > > >> Hi, > > > > > > >> > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > > > > >> > > > > > > > >> > Hi, > > > > > > >> > > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > > > > > >> > memory allocation in U-Boot. > > > > > > >> > > > > > > > >> > 1. malloc() > > > > > > >> > > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > > > > > >> > 10MB kernel or 100MB ramdisk). > > > > > > >> > > > > > > > >> > 2. lmb > > > > > > >> > > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > > > > > >> > loading some images and booting. If the boot fails, then all lmb > > > > > > >> > allocations go away. > > > > > > >> > > > > > > > >> > lmb is set up by getting all available memory and then removing what > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > > > > > >> > > > > > > > >> > lmb reservations have a few flags so that areas of memory can be > > > > > > >> > provided with attributes > > > > > > >> > > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > > > > > >> > this would be fairly easy to solve. > > > > > > >> > > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > > > > > >> > > > > > > > >> > 3. EFI > > > > > > >> > > > > > > > >> > EFI has its own memory-allocation tables. > > > > > > >> > > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > > > > > >> > function it can also do smaller allocations similar to malloc(), > > > > > > >> > although each one uses at least 4KB at present. > > > > > > >> > > > > > > > >> > EFI allocations do not go away when a boot fails. > > > > > > >> > > > > > > > >> > With EFI it is possible to add allocations post facto, in which case > > > > > > >> > they are added to the allocation table just as if the memory was > > > > > > >> > allocated with EFI to begin with. > > > > > > >> > > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > > > > > >> > principle could conflict. > > > > > > >> > > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > > > > > >> > in the app context and may need to access the memory after U-Boot has > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > > > > > >> > mapping as well. > > > > > > >> > > > > > > > >> > EFI memory has attributes, including what the memory is used for (to > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > > > > > >> > memory is cacheable, etc. > > > > > > >> > > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. > > > > > > >> > > > > > > > >> > 4. reservations > > > > > > >> > > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > > > > > >> > region, etc. > > > > > > >> > > > > > > > >> > > > > > > > >> > Problems > > > > > > >> > —------- > > > > > > >> > > > > > > > >> > There are no urgent problems, but here are some things that could be improved: > > > > > > >> > > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > > > > > >> > work has started, with the partition support, but more effort would > > > > > > >> > help. This would make it easier to see which memory is related to > > > > > > >> > devices and which is separate. > > > > > > >> > > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > > > > > >> > > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > > > > > >> > armv8's struct mm_region) to EFI ones. > > > > > > >> > > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > > > > > >> > memory allocation (e.g. FDT fixup). > > > > > > >> > > > > > > > >> > 5. EFI code is used even if EFI is never used to boot > > > > > > >> > > > > > > > >> > 6. EFI allocations can result in the same memory being used as has > > > > > > >> > already been allocated by lmb. Users may load files which overwrite > > > > > > >> > memory allocated by EFI. > > > > > > >> > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to > > > > > > >> ensure files do not overlap), without making it too difficult to load > > > > > > >> multiple files to the same place, if desired. > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > > > >> > Lifetime > > > > > > >> > -------- > > > > > > >> > > > > > > > >> > We have three different memory allocators with different purposes. Can > > > > > > >> > we unify them a little? > > > > > > >> > > > > > > > >> > Within U-Boot: > > > > > > >> > - malloc() space lives forever > > > > > > >> > - lmb lives while setting out images for booting > > > > > > >> > - EFI (mostly) lives while booting an EFI app > > > > > > >> > > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > > > > > >> > some not. EFI allocations stay around forever. This works OK since > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really > > > > > > >> > consumed to any great degree by the boot process. > > > > > > >> > > > > > > > >> > What happens to EFI allocations if the app returns? They are still > > > > > > >> > present, in case another app is run. This seems fine. > > > > > > >> > > > > > > > >> > API > > > > > > >> > –-- > > > > > > >> > Can we unify some APIs? > > > > > > >> > > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > > > > > >> > long as they are only needed for booting. We effectively do this > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in > > > > > > >> > memory. for the most part. > > > > > > >> > > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > > > > > >> > malloc(), of course. > > > > > > >> > > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > > > > > >> > lmb and EFI? They do have some similarities. However they have > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, > > > > > > >> > unlikely lmb). > > > > > > >> > > > > > > > >> > ** Overall, it seems that the existence of memory allocation in > > > > > > >> > boot-time services has created confusion. Memory allocation is > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > > > > > >> > memory allocator. This just has not been clearly thought out. > > > > > > >> > > > > > > > >> > > > > > > > >> > Proposal > > > > > > >> > —------- > > > > > > >> > > > > > > > >> > Here are some ideas: > > > > > > >> > > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > > > > > >> > Use efi_add_memory_map() > > > > > > >> > > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > > > > >> > > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > > > > > >> > understands the meaning of each. > > > > > > >> > > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > > > > > >> > booting. This looks to require very little effort. > > > > > > >> > > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > > > > > >> > allocated in normal malloc() space or in a bloblist. > > > > > > >> > > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is > > > > > > >> > done for driver model. Try to minmise the cost of its tables, > > > > > > >> > particularly for execution time, but otherwise just rely on the > > > > > > >> > ability to disable EFI_LOADER. > > > > > > >> > > > > > > >> 7. Add a flag to the 'load' command: > > > > > > >> > > > > > > >> -m <type> - make an lmb allocation for the file > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > > > > > >> > > > > > > >> any existing allocation for that type will be automatically freed > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > > > > > >> images just stack up in lmb. > > > > > > >> > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > > > > > > > > > >I would really not like to change the user interface and instead simply > > > > > > >handle this with flags to whatever mark/allocation function is called. > > > > > > >You can always overwrite things that are brought in to memory, you > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > > > > > >previous load to memory has been at least partially overwritten could be > > > > > > >helpful, if it's not too much extra logic. > > > > > > > > > > > > > > > > > > > In most use cases users load exactly one file at each address. An > > > > > > unload command would be the cleanest way for a user to indicate that > > > > > > he wants to reuse the memory. > > > > > > > > > > I very much do not want to change the API. There's untold numbers of > > > > > scripts out there and they should continue to work. I mentioned to Ilias > > > > > off list just now that I'm not against adding a command to add flags to > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the > > > > > user did early. The biggest long running problem in this space was that > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > > > > so would have ramdisk at the wrong spot and get partially eaten, and > > > > > this was hard to figure out. The current example is "ooops, > > > > > decompression buffer for Image.gz/etc is too close to other things" > > > > > which ends up failing nice and loudly, and in the future once this > > > > > proposal is done we can just dynamically find and use a spot, since > > > > > we'll have that ability finally. > > > > > > > > In order to keep the existing interfaces we need lmb to keep track of > > > > (at least ) three different states. I think of those as "free", > > > > "allocated" and "reserved". The load command would "reserve" memory > > > > insteaf "allocate". And it would pass a flag to lmb when reserving > > > > memory to indicate that reserving memory that is already reserved is > > > > ok. Both "reserved" and "allocated" memory should show up as not free > > > > in the EFI memory map (probably as EfiLoaderData). > > > > > > Yes, something like this is what I was getting at, thanks. > > > > Yes, that is a good way of putting it. There is definitely a distinction there. > > > > If we don't want this flag, we could make U-Boot always do a > > reservation on load, with a '-f' command to force loading over an > > existing reservation / releasing it first? > > Again, this is an API change and I don't want to change the API.
The flag is only needed to drop a reservation, since we apparently want the 'load' command to create a permanent reservation. It should not affect existing boot scripts since they won't load overlapping images.
Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
Regards, Simon
-sughosh
[1] - https://docs.google.com/document/d/1LJj4f1oBaxPIALnMlIy8Bf-3pESjecTyKhrQoeCf...
-- Tom
hi Simon,
On Mon, 10 Jun 2024 at 19:25, Simon Glass sjg@chromium.org wrote:
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote: > Hi Tom, > > On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: > > > > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: > > > Hi, > > > > > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > > > > > > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > > > > > From: Tom Rini trini@konsulko.com > > > > > > > > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > > > > > > >> Hi, > > > > > > > >> > > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > > > > > >> > > > > > > > > >> > Hi, > > > > > > > >> > > > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > > > > > > >> > memory allocation in U-Boot. > > > > > > > >> > > > > > > > > >> > 1. malloc() > > > > > > > >> > > > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > > > > > > >> > 10MB kernel or 100MB ramdisk). > > > > > > > >> > > > > > > > > >> > 2. lmb > > > > > > > >> > > > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > > > > > > >> > loading some images and booting. If the boot fails, then all lmb > > > > > > > >> > allocations go away. > > > > > > > >> > > > > > > > > >> > lmb is set up by getting all available memory and then removing what > > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > > > > > > >> > > > > > > > > >> > lmb reservations have a few flags so that areas of memory can be > > > > > > > >> > provided with attributes > > > > > > > >> > > > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded > > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > > > > > > >> > this would be fairly easy to solve. > > > > > > > >> > > > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > > > > > > >> > > > > > > > > >> > 3. EFI > > > > > > > >> > > > > > > > > >> > EFI has its own memory-allocation tables. > > > > > > > >> > > > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > > > > > > >> > function it can also do smaller allocations similar to malloc(), > > > > > > > >> > although each one uses at least 4KB at present. > > > > > > > >> > > > > > > > > >> > EFI allocations do not go away when a boot fails. > > > > > > > >> > > > > > > > > >> > With EFI it is possible to add allocations post facto, in which case > > > > > > > >> > they are added to the allocation table just as if the memory was > > > > > > > >> > allocated with EFI to begin with. > > > > > > > >> > > > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > > > > > > >> > principle could conflict. > > > > > > > >> > > > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > > > > > > >> > in the app context and may need to access the memory after U-Boot has > > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > > > > > > >> > mapping as well. > > > > > > > >> > > > > > > > > >> > EFI memory has attributes, including what the memory is used for (to > > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct > > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > > > > > > >> > memory is cacheable, etc. > > > > > > > >> > > > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. > > > > > > > >> > > > > > > > > >> > 4. reservations > > > > > > > >> > > > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in > > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > > > > > > >> > region, etc. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Problems > > > > > > > >> > —------- > > > > > > > >> > > > > > > > > >> > There are no urgent problems, but here are some things that could be improved: > > > > > > > >> > > > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > > > > > > >> > work has started, with the partition support, but more effort would > > > > > > > >> > help. This would make it easier to see which memory is related to > > > > > > > >> > devices and which is separate. > > > > > > > >> > > > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > > > > > > >> > > > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > > > > > > >> > armv8's struct mm_region) to EFI ones. > > > > > > > >> > > > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > > > > > > >> > memory allocation (e.g. FDT fixup). > > > > > > > >> > > > > > > > > >> > 5. EFI code is used even if EFI is never used to boot > > > > > > > >> > > > > > > > > >> > 6. EFI allocations can result in the same memory being used as has > > > > > > > >> > already been allocated by lmb. Users may load files which overwrite > > > > > > > >> > memory allocated by EFI. > > > > > > > >> > > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to > > > > > > > >> ensure files do not overlap), without making it too difficult to load > > > > > > > >> multiple files to the same place, if desired. > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Lifetime > > > > > > > >> > -------- > > > > > > > >> > > > > > > > > >> > We have three different memory allocators with different purposes. Can > > > > > > > >> > we unify them a little? > > > > > > > >> > > > > > > > > >> > Within U-Boot: > > > > > > > >> > - malloc() space lives forever > > > > > > > >> > - lmb lives while setting out images for booting > > > > > > > >> > - EFI (mostly) lives while booting an EFI app > > > > > > > >> > > > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > > > > > > >> > some not. EFI allocations stay around forever. This works OK since > > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really > > > > > > > >> > consumed to any great degree by the boot process. > > > > > > > >> > > > > > > > > >> > What happens to EFI allocations if the app returns? They are still > > > > > > > >> > present, in case another app is run. This seems fine. > > > > > > > >> > > > > > > > > >> > API > > > > > > > >> > –-- > > > > > > > >> > Can we unify some APIs? > > > > > > > >> > > > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > > > > > > >> > long as they are only needed for booting. We effectively do this > > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in > > > > > > > >> > memory. for the most part. > > > > > > > >> > > > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > > > > > > >> > malloc(), of course. > > > > > > > >> > > > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > > > > > > >> > lmb and EFI? They do have some similarities. However they have > > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, > > > > > > > >> > unlikely lmb). > > > > > > > >> > > > > > > > > >> > ** Overall, it seems that the existence of memory allocation in > > > > > > > >> > boot-time services has created confusion. Memory allocation is > > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > > > > > > >> > memory allocator. This just has not been clearly thought out. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Proposal > > > > > > > >> > —------- > > > > > > > >> > > > > > > > > >> > Here are some ideas: > > > > > > > >> > > > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > > > > > > >> > Use efi_add_memory_map() > > > > > > > >> > > > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > > > > > >> > > > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The > > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data > > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > > > > > > >> > understands the meaning of each. > > > > > > > >> > > > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > > > > > > >> > booting. This looks to require very little effort. > > > > > > > >> > > > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two > > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > > > > > > >> > allocated in normal malloc() space or in a bloblist. > > > > > > > >> > > > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is > > > > > > > >> > done for driver model. Try to minmise the cost of its tables, > > > > > > > >> > particularly for execution time, but otherwise just rely on the > > > > > > > >> > ability to disable EFI_LOADER. > > > > > > > >> > > > > > > > >> 7. Add a flag to the 'load' command: > > > > > > > >> > > > > > > > >> -m <type> - make an lmb allocation for the file > > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > > > > > > >> > > > > > > > >> any existing allocation for that type will be automatically freed > > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > > > > > > >> images just stack up in lmb. > > > > > > > >> > > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > > > > > > > > > > > >I would really not like to change the user interface and instead simply > > > > > > > >handle this with flags to whatever mark/allocation function is called. > > > > > > > >You can always overwrite things that are brought in to memory, you > > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > > > > > > >previous load to memory has been at least partially overwritten could be > > > > > > > >helpful, if it's not too much extra logic. > > > > > > > > > > > > > > > > > > > > > > In most use cases users load exactly one file at each address. An > > > > > > > unload command would be the cleanest way for a user to indicate that > > > > > > > he wants to reuse the memory. > > > > > > > > > > > > I very much do not want to change the API. There's untold numbers of > > > > > > scripts out there and they should continue to work. I mentioned to Ilias > > > > > > off list just now that I'm not against adding a command to add flags to > > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the > > > > > > user did early. The biggest long running problem in this space was that > > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > > > > > so would have ramdisk at the wrong spot and get partially eaten, and > > > > > > this was hard to figure out. The current example is "ooops, > > > > > > decompression buffer for Image.gz/etc is too close to other things" > > > > > > which ends up failing nice and loudly, and in the future once this > > > > > > proposal is done we can just dynamically find and use a spot, since > > > > > > we'll have that ability finally. > > > > > > > > > > In order to keep the existing interfaces we need lmb to keep track of > > > > > (at least ) three different states. I think of those as "free", > > > > > "allocated" and "reserved". The load command would "reserve" memory > > > > > insteaf "allocate". And it would pass a flag to lmb when reserving > > > > > memory to indicate that reserving memory that is already reserved is > > > > > ok. Both "reserved" and "allocated" memory should show up as not free > > > > > in the EFI memory map (probably as EfiLoaderData). > > > > > > > > Yes, something like this is what I was getting at, thanks. > > > > > > Yes, that is a good way of putting it. There is definitely a distinction there. > > > > > > If we don't want this flag, we could make U-Boot always do a > > > reservation on load, with a '-f' command to force loading over an > > > existing reservation / releasing it first? > > > > Again, this is an API change and I don't want to change the API. > > The flag is only needed to drop a reservation, since we apparently > want the 'load' command to create a permanent reservation. It should > not affect existing boot scripts since they won't load overlapping > images. > > Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Can you please describe in a little more detail what "whole new class of problems" are being introduced here. If there are problems being introduced, it is better if they are enumerated, so that I can try to fix them.
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
I have gone through this thread multiple times, and my understanding was that there was no conclusive agreement that was arrived at. I saw a reply from Tom where he broadly mentioned what kind of a solution he is looking for. I am not sure if you have gone through the cover-letter of my RFC series [1], which clearly states the problem my patch series is trying to fix. Like I said above, it would help if you can describe in a little more detail the kind of issues you see with the RFC series. Thanks.
-sughosh
Regards, Simon
-sughosh
[1] - https://docs.google.com/document/d/1LJj4f1oBaxPIALnMlIy8Bf-3pESjecTyKhrQoeCf...
-- Tom
Hi Sughosh,
On Mon, 10 Jun 2024 at 08:40, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Mon, 10 Jun 2024 at 19:25, Simon Glass sjg@chromium.org wrote:
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com: >On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote: >> Hi Tom, >> >> On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: >> > >> > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: >> > > Hi, >> > > >> > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: >> > > > >> > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: >> > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 >> > > > > > From: Tom Rini trini@konsulko.com >> > > > > > >> > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: >> > > > > > > >> > > > > > > >> > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: >> > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: >> > > > > > > >> Hi, >> > > > > > > >> >> > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: >> > > > > > > >> > >> > > > > > > >> > Hi, >> > > > > > > >> > >> > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re >> > > > > > > >> > memory allocation in U-Boot. >> > > > > > > >> > >> > > > > > > >> > 1. malloc() >> > > > > > > >> > >> > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory >> > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a >> > > > > > > >> > 10MB kernel or 100MB ramdisk). >> > > > > > > >> > >> > > > > > > >> > 2. lmb >> > > > > > > >> > >> > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a >> > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of >> > > > > > > >> > loading some images and booting. If the boot fails, then all lmb >> > > > > > > >> > allocations go away. >> > > > > > > >> > >> > > > > > > >> > lmb is set up by getting all available memory and then removing what >> > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) >> > > > > > > >> > >> > > > > > > >> > lmb reservations have a few flags so that areas of memory can be >> > > > > > > >> > provided with attributes >> > > > > > > >> > >> > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb >> > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded >> > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the >> > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, >> > > > > > > >> > this would be fairly easy to solve. >> > > > > > > >> > >> > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. >> > > > > > > >> > >> > > > > > > >> > 3. EFI >> > > > > > > >> > >> > > > > > > >> > EFI has its own memory-allocation tables. >> > > > > > > >> > >> > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' >> > > > > > > >> > function it can also do smaller allocations similar to malloc(), >> > > > > > > >> > although each one uses at least 4KB at present. >> > > > > > > >> > >> > > > > > > >> > EFI allocations do not go away when a boot fails. >> > > > > > > >> > >> > > > > > > >> > With EFI it is possible to add allocations post facto, in which case >> > > > > > > >> > they are added to the allocation table just as if the memory was >> > > > > > > >> > allocated with EFI to begin with. >> > > > > > > >> > >> > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in >> > > > > > > >> > principle could conflict. >> > > > > > > >> > >> > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as >> > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() >> > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs >> > > > > > > >> > in the app context and may need to access the memory after U-Boot has >> > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new >> > > > > > > >> > mapping as well. >> > > > > > > >> > >> > > > > > > >> > EFI memory has attributes, including what the memory is used for (to >> > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct >> > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether >> > > > > > > >> > memory is cacheable, etc. >> > > > > > > >> > >> > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I >> > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. >> > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. >> > > > > > > >> > >> > > > > > > >> > 4. reservations >> > > > > > > >> > >> > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, >> > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in >> > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() >> > > > > > > >> > region, etc. >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > Problems >> > > > > > > >> > —------- >> > > > > > > >> > >> > > > > > > >> > There are no urgent problems, but here are some things that could be improved: >> > > > > > > >> > >> > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This >> > > > > > > >> > work has started, with the partition support, but more effort would >> > > > > > > >> > help. This would make it easier to see which memory is related to >> > > > > > > >> > devices and which is separate. >> > > > > > > >> > >> > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for >> > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). >> > > > > > > >> > >> > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. >> > > > > > > >> > armv8's struct mm_region) to EFI ones. >> > > > > > > >> > >> > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to >> > > > > > > >> > memory allocation (e.g. FDT fixup). >> > > > > > > >> > >> > > > > > > >> > 5. EFI code is used even if EFI is never used to boot >> > > > > > > >> > >> > > > > > > >> > 6. EFI allocations can result in the same memory being used as has >> > > > > > > >> > already been allocated by lmb. Users may load files which overwrite >> > > > > > > >> > memory allocated by EFI. >> > > > > > > >> >> > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to >> > > > > > > >> ensure files do not overlap), without making it too difficult to load >> > > > > > > >> multiple files to the same place, if desired. >> > > > > > > >> >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > Lifetime >> > > > > > > >> > -------- >> > > > > > > >> > >> > > > > > > >> > We have three different memory allocators with different purposes. Can >> > > > > > > >> > we unify them a little? >> > > > > > > >> > >> > > > > > > >> > Within U-Boot: >> > > > > > > >> > - malloc() space lives forever >> > > > > > > >> > - lmb lives while setting out images for booting >> > > > > > > >> > - EFI (mostly) lives while booting an EFI app >> > > > > > > >> > >> > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, >> > > > > > > >> > some not. EFI allocations stay around forever. This works OK since >> > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really >> > > > > > > >> > consumed to any great degree by the boot process. >> > > > > > > >> > >> > > > > > > >> > What happens to EFI allocations if the app returns? They are still >> > > > > > > >> > present, in case another app is run. This seems fine. >> > > > > > > >> > >> > > > > > > >> > API >> > > > > > > >> > –-- >> > > > > > > >> > Can we unify some APIs? >> > > > > > > >> > >> > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so >> > > > > > > >> > long as they are only needed for booting. We effectively do this >> > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in >> > > > > > > >> > memory. for the most part. >> > > > > > > >> > >> > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and >> > > > > > > >> > malloc(), of course. >> > > > > > > >> > >> > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both >> > > > > > > >> > lmb and EFI? They do have some similarities. However they have >> > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, >> > > > > > > >> > unlikely lmb). >> > > > > > > >> > >> > > > > > > >> > ** Overall, it seems that the existence of memory allocation in >> > > > > > > >> > boot-time services has created confusion. Memory allocation is >> > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same >> > > > > > > >> > memory allocator. This just has not been clearly thought out. >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > Proposal >> > > > > > > >> > —------- >> > > > > > > >> > >> > > > > > > >> > Here are some ideas: >> > > > > > > >> > >> > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or >> > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. >> > > > > > > >> > Use efi_add_memory_map() >> > > > > > > >> > >> > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to >> > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. >> > > > > > > >> > >> > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with >> > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all >> > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in >> > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The >> > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data >> > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables >> > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code >> > > > > > > >> > understands the meaning of each. >> > > > > > > >> > >> > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when >> > > > > > > >> > booting. This looks to require very little effort. >> > > > > > > >> > >> > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside >> > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by >> > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two >> > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of >> > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For >> > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be >> > > > > > > >> > allocated in normal malloc() space or in a bloblist. >> > > > > > > >> > >> > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. >> > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is >> > > > > > > >> > done for driver model. Try to minmise the cost of its tables, >> > > > > > > >> > particularly for execution time, but otherwise just rely on the >> > > > > > > >> > ability to disable EFI_LOADER. >> > > > > > > >> >> > > > > > > >> 7. Add a flag to the 'load' command: >> > > > > > > >> >> > > > > > > >> -m <type> - make an lmb allocation for the file >> > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) >> > > > > > > >> >> > > > > > > >> any existing allocation for that type will be automatically freed >> > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded >> > > > > > > >> images just stack up in lmb. >> > > > > > > >> >> > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. >> > > > > > > > >> > > > > > > >I would really not like to change the user interface and instead simply >> > > > > > > >handle this with flags to whatever mark/allocation function is called. >> > > > > > > >You can always overwrite things that are brought in to memory, you >> > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some >> > > > > > > >previous load to memory has been at least partially overwritten could be >> > > > > > > >helpful, if it's not too much extra logic. >> > > > > > > > >> > > > > > > >> > > > > > > In most use cases users load exactly one file at each address. An >> > > > > > > unload command would be the cleanest way for a user to indicate that >> > > > > > > he wants to reuse the memory. >> > > > > > >> > > > > > I very much do not want to change the API. There's untold numbers of >> > > > > > scripts out there and they should continue to work. I mentioned to Ilias >> > > > > > off list just now that I'm not against adding a command to add flags to >> > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the >> > > > > > user did early. The biggest long running problem in this space was that >> > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and >> > > > > > so would have ramdisk at the wrong spot and get partially eaten, and >> > > > > > this was hard to figure out. The current example is "ooops, >> > > > > > decompression buffer for Image.gz/etc is too close to other things" >> > > > > > which ends up failing nice and loudly, and in the future once this >> > > > > > proposal is done we can just dynamically find and use a spot, since >> > > > > > we'll have that ability finally. >> > > > > >> > > > > In order to keep the existing interfaces we need lmb to keep track of >> > > > > (at least ) three different states. I think of those as "free", >> > > > > "allocated" and "reserved". The load command would "reserve" memory >> > > > > insteaf "allocate". And it would pass a flag to lmb when reserving >> > > > > memory to indicate that reserving memory that is already reserved is >> > > > > ok. Both "reserved" and "allocated" memory should show up as not free >> > > > > in the EFI memory map (probably as EfiLoaderData). >> > > > >> > > > Yes, something like this is what I was getting at, thanks. >> > > >> > > Yes, that is a good way of putting it. There is definitely a distinction there. >> > > >> > > If we don't want this flag, we could make U-Boot always do a >> > > reservation on load, with a '-f' command to force loading over an >> > > existing reservation / releasing it first? >> > >> > Again, this is an API change and I don't want to change the API. >> >> The flag is only needed to drop a reservation, since we apparently >> want the 'load' command to create a permanent reservation. It should >> not affect existing boot scripts since they won't load overlapping >> images. >> >> Anyway, what do you suggest? > >That "load" (and sf read and nand read and tftp and wget and ...) >"reserve" memory but not "allocate" memory and "reserve" means something >is there and "allocate" means that it can't be modified again. For >example, running U-Boot and our malloc pool are "allocated" but just >loading a file to memory is "reserved". And then yes, I can see use for >the command where some cases might want to "reserve" memory to fiddle >with it and then "allocate" it so something else can't change it. > >This is one of those cases where english is terrible to discuss things >in as both reserve and allocate can mean similar things. >
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Can you please describe in a little more detail what "whole new class of problems" are being introduced here. If there are problems being introduced, it is better if they are enumerated, so that I can try to fix them.
Please see my point immediately below:
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
I have gone through this thread multiple times, and my understanding was that there was no conclusive agreement that was arrived at. I saw a reply from Tom where he broadly mentioned what kind of a solution he is looking for. I am not sure if you have gone through the cover-letter of my RFC series [1], which clearly states the problem my patch series is trying to fix. Like I said above, it would help if you can describe in a little more detail the kind of issues you see with the RFC series. Thanks.
My request was for you to reply to the thread with any thoughts so we can agree what to do about it. I believe my original email mostly stands as a summary of where things are and what the problems are. If it helps, I can update it based on the discussion above. It is important to avoid over-complicating U-Boot.
My fundamental expectation is that LMB reservations only apply for a particular boot. They should not be global, nor need freeing, nor should they be similar to memory allocations. We discussed it in this thread.
Regards, Simon
hi Simon,
On Mon, 10 Jun 2024 at 20:33, Simon Glass sjg@chromium.org wrote:
Hi Sughosh,
On Mon, 10 Jun 2024 at 08:40, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Mon, 10 Jun 2024 at 19:25, Simon Glass sjg@chromium.org wrote:
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote: > > > Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com: > >On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote: > >> Hi Tom, > >> > >> On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: > >> > > >> > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: > >> > > Hi, > >> > > > >> > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > >> > > > > >> > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > >> > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > >> > > > > > From: Tom Rini trini@konsulko.com > >> > > > > > > >> > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > >> > > > > > > > >> > > > > > > > >> > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > >> > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > >> > > > > > > >> Hi, > >> > > > > > > >> > >> > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > >> > > > > > > >> > > >> > > > > > > >> > Hi, > >> > > > > > > >> > > >> > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > >> > > > > > > >> > memory allocation in U-Boot. > >> > > > > > > >> > > >> > > > > > > >> > 1. malloc() > >> > > > > > > >> > > >> > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > >> > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > >> > > > > > > >> > 10MB kernel or 100MB ramdisk). > >> > > > > > > >> > > >> > > > > > > >> > 2. lmb > >> > > > > > > >> > > >> > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > >> > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > >> > > > > > > >> > loading some images and booting. If the boot fails, then all lmb > >> > > > > > > >> > allocations go away. > >> > > > > > > >> > > >> > > > > > > >> > lmb is set up by getting all available memory and then removing what > >> > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > >> > > > > > > >> > > >> > > > > > > >> > lmb reservations have a few flags so that areas of memory can be > >> > > > > > > >> > provided with attributes > >> > > > > > > >> > > >> > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > >> > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded > >> > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > >> > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > >> > > > > > > >> > this would be fairly easy to solve. > >> > > > > > > >> > > >> > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > >> > > > > > > >> > > >> > > > > > > >> > 3. EFI > >> > > > > > > >> > > >> > > > > > > >> > EFI has its own memory-allocation tables. > >> > > > > > > >> > > >> > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > >> > > > > > > >> > function it can also do smaller allocations similar to malloc(), > >> > > > > > > >> > although each one uses at least 4KB at present. > >> > > > > > > >> > > >> > > > > > > >> > EFI allocations do not go away when a boot fails. > >> > > > > > > >> > > >> > > > > > > >> > With EFI it is possible to add allocations post facto, in which case > >> > > > > > > >> > they are added to the allocation table just as if the memory was > >> > > > > > > >> > allocated with EFI to begin with. > >> > > > > > > >> > > >> > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > >> > > > > > > >> > principle could conflict. > >> > > > > > > >> > > >> > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > >> > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > >> > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > >> > > > > > > >> > in the app context and may need to access the memory after U-Boot has > >> > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > >> > > > > > > >> > mapping as well. > >> > > > > > > >> > > >> > > > > > > >> > EFI memory has attributes, including what the memory is used for (to > >> > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct > >> > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > >> > > > > > > >> > memory is cacheable, etc. > >> > > > > > > >> > > >> > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > >> > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > >> > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. > >> > > > > > > >> > > >> > > > > > > >> > 4. reservations > >> > > > > > > >> > > >> > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > >> > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in > >> > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > >> > > > > > > >> > region, etc. > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > Problems > >> > > > > > > >> > —------- > >> > > > > > > >> > > >> > > > > > > >> > There are no urgent problems, but here are some things that could be improved: > >> > > > > > > >> > > >> > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > >> > > > > > > >> > work has started, with the partition support, but more effort would > >> > > > > > > >> > help. This would make it easier to see which memory is related to > >> > > > > > > >> > devices and which is separate. > >> > > > > > > >> > > >> > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > >> > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > >> > > > > > > >> > > >> > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > >> > > > > > > >> > armv8's struct mm_region) to EFI ones. > >> > > > > > > >> > > >> > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > >> > > > > > > >> > memory allocation (e.g. FDT fixup). > >> > > > > > > >> > > >> > > > > > > >> > 5. EFI code is used even if EFI is never used to boot > >> > > > > > > >> > > >> > > > > > > >> > 6. EFI allocations can result in the same memory being used as has > >> > > > > > > >> > already been allocated by lmb. Users may load files which overwrite > >> > > > > > > >> > memory allocated by EFI. > >> > > > > > > >> > >> > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to > >> > > > > > > >> ensure files do not overlap), without making it too difficult to load > >> > > > > > > >> multiple files to the same place, if desired. > >> > > > > > > >> > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > Lifetime > >> > > > > > > >> > -------- > >> > > > > > > >> > > >> > > > > > > >> > We have three different memory allocators with different purposes. Can > >> > > > > > > >> > we unify them a little? > >> > > > > > > >> > > >> > > > > > > >> > Within U-Boot: > >> > > > > > > >> > - malloc() space lives forever > >> > > > > > > >> > - lmb lives while setting out images for booting > >> > > > > > > >> > - EFI (mostly) lives while booting an EFI app > >> > > > > > > >> > > >> > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > >> > > > > > > >> > some not. EFI allocations stay around forever. This works OK since > >> > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really > >> > > > > > > >> > consumed to any great degree by the boot process. > >> > > > > > > >> > > >> > > > > > > >> > What happens to EFI allocations if the app returns? They are still > >> > > > > > > >> > present, in case another app is run. This seems fine. > >> > > > > > > >> > > >> > > > > > > >> > API > >> > > > > > > >> > –-- > >> > > > > > > >> > Can we unify some APIs? > >> > > > > > > >> > > >> > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > >> > > > > > > >> > long as they are only needed for booting. We effectively do this > >> > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in > >> > > > > > > >> > memory. for the most part. > >> > > > > > > >> > > >> > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > >> > > > > > > >> > malloc(), of course. > >> > > > > > > >> > > >> > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > >> > > > > > > >> > lmb and EFI? They do have some similarities. However they have > >> > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, > >> > > > > > > >> > unlikely lmb). > >> > > > > > > >> > > >> > > > > > > >> > ** Overall, it seems that the existence of memory allocation in > >> > > > > > > >> > boot-time services has created confusion. Memory allocation is > >> > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > >> > > > > > > >> > memory allocator. This just has not been clearly thought out. > >> > > > > > > >> > > >> > > > > > > >> > > >> > > > > > > >> > Proposal > >> > > > > > > >> > —------- > >> > > > > > > >> > > >> > > > > > > >> > Here are some ideas: > >> > > > > > > >> > > >> > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > >> > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > >> > > > > > > >> > Use efi_add_memory_map() > >> > > > > > > >> > > >> > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > >> > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > >> > > > > > > >> > > >> > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > >> > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > >> > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > >> > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The > >> > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data > >> > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > >> > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > >> > > > > > > >> > understands the meaning of each. > >> > > > > > > >> > > >> > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > >> > > > > > > >> > booting. This looks to require very little effort. > >> > > > > > > >> > > >> > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > >> > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > >> > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two > >> > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > >> > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > >> > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > >> > > > > > > >> > allocated in normal malloc() space or in a bloblist. > >> > > > > > > >> > > >> > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > >> > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is > >> > > > > > > >> > done for driver model. Try to minmise the cost of its tables, > >> > > > > > > >> > particularly for execution time, but otherwise just rely on the > >> > > > > > > >> > ability to disable EFI_LOADER. > >> > > > > > > >> > >> > > > > > > >> 7. Add a flag to the 'load' command: > >> > > > > > > >> > >> > > > > > > >> -m <type> - make an lmb allocation for the file > >> > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > >> > > > > > > >> > >> > > > > > > >> any existing allocation for that type will be automatically freed > >> > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > >> > > > > > > >> images just stack up in lmb. > >> > > > > > > >> > >> > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > >> > > > > > > > > >> > > > > > > >I would really not like to change the user interface and instead simply > >> > > > > > > >handle this with flags to whatever mark/allocation function is called. > >> > > > > > > >You can always overwrite things that are brought in to memory, you > >> > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > >> > > > > > > >previous load to memory has been at least partially overwritten could be > >> > > > > > > >helpful, if it's not too much extra logic. > >> > > > > > > > > >> > > > > > > > >> > > > > > > In most use cases users load exactly one file at each address. An > >> > > > > > > unload command would be the cleanest way for a user to indicate that > >> > > > > > > he wants to reuse the memory. > >> > > > > > > >> > > > > > I very much do not want to change the API. There's untold numbers of > >> > > > > > scripts out there and they should continue to work. I mentioned to Ilias > >> > > > > > off list just now that I'm not against adding a command to add flags to > >> > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the > >> > > > > > user did early. The biggest long running problem in this space was that > >> > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > >> > > > > > so would have ramdisk at the wrong spot and get partially eaten, and > >> > > > > > this was hard to figure out. The current example is "ooops, > >> > > > > > decompression buffer for Image.gz/etc is too close to other things" > >> > > > > > which ends up failing nice and loudly, and in the future once this > >> > > > > > proposal is done we can just dynamically find and use a spot, since > >> > > > > > we'll have that ability finally. > >> > > > > > >> > > > > In order to keep the existing interfaces we need lmb to keep track of > >> > > > > (at least ) three different states. I think of those as "free", > >> > > > > "allocated" and "reserved". The load command would "reserve" memory > >> > > > > insteaf "allocate". And it would pass a flag to lmb when reserving > >> > > > > memory to indicate that reserving memory that is already reserved is > >> > > > > ok. Both "reserved" and "allocated" memory should show up as not free > >> > > > > in the EFI memory map (probably as EfiLoaderData). > >> > > > > >> > > > Yes, something like this is what I was getting at, thanks. > >> > > > >> > > Yes, that is a good way of putting it. There is definitely a distinction there. > >> > > > >> > > If we don't want this flag, we could make U-Boot always do a > >> > > reservation on load, with a '-f' command to force loading over an > >> > > existing reservation / releasing it first? > >> > > >> > Again, this is an API change and I don't want to change the API. > >> > >> The flag is only needed to drop a reservation, since we apparently > >> want the 'load' command to create a permanent reservation. It should > >> not affect existing boot scripts since they won't load overlapping > >> images. > >> > >> Anyway, what do you suggest? > > > >That "load" (and sf read and nand read and tftp and wget and ...) > >"reserve" memory but not "allocate" memory and "reserve" means something > >is there and "allocate" means that it can't be modified again. For > >example, running U-Boot and our malloc pool are "allocated" but just > >loading a file to memory is "reserved". And then yes, I can see use for > >the command where some cases might want to "reserve" memory to fiddle > >with it and then "allocate" it so something else can't change it. > > > >This is one of those cases where english is terrible to discuss things > >in as both reserve and allocate can mean similar things. > > > > I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A. > > Who can and who cannot write into "reserved" memory? > > What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Can you please describe in a little more detail what "whole new class of problems" are being introduced here. If there are problems being introduced, it is better if they are enumerated, so that I can try to fix them.
Please see my point immediately below:
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
I have gone through this thread multiple times, and my understanding was that there was no conclusive agreement that was arrived at. I saw a reply from Tom where he broadly mentioned what kind of a solution he is looking for. I am not sure if you have gone through the cover-letter of my RFC series [1], which clearly states the problem my patch series is trying to fix. Like I said above, it would help if you can describe in a little more detail the kind of issues you see with the RFC series. Thanks.
My request was for you to reply to the thread with any thoughts so we can agree what to do about it. I believe my original email mostly stands as a summary of where things are and what the problems are. If it helps, I can update it based on the discussion above. It is important to avoid over-complicating U-Boot.
My fundamental expectation is that LMB reservations only apply for a particular boot. They should not be global, nor need freeing, nor should they be similar to memory allocations. We discussed it in this thread.
Please note that I have run the patches through CI, and they pass the CI run. There are numerous examples in the CI test scripts where the LMB memory is re-used, and these patches are not changing that behaviour in any which way. The only reason why the LMB memory needs to be persistent and global is because it needs to work cohesively with the EFI memory allocator.
At this point, I guess the EFI maintainers and Tom have to provide their input on how we go about this. But if we have to have EFI allocations live alongside LMB, the *only* other solution that I see is to then break up the memory into two sections, one for LMB and one for EFI. But then again, that introduces it's own set of issues, for e.g. a platform might not be able to use some memory region simply because it is allocated to the other subsystem. But I would let the maintainers chime in at this point. Thanks.
-sughosh
Regards, Simon
Hi Sughosh,
On Mon, 10 Jun 2024 at 09:17, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Mon, 10 Jun 2024 at 20:33, Simon Glass sjg@chromium.org wrote:
Hi Sughosh,
On Mon, 10 Jun 2024 at 08:40, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Mon, 10 Jun 2024 at 19:25, Simon Glass sjg@chromium.org wrote:
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote: > > On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote: > > > > > > Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com: > > >On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote: > > >> Hi Tom, > > >> > > >> On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: > > >> > > > >> > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: > > >> > > Hi, > > >> > > > > >> > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > > >> > > > > > >> > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > >> > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > >> > > > > > From: Tom Rini trini@konsulko.com > > >> > > > > > > > >> > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > >> > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > >> > > > > > > >> Hi, > > >> > > > > > > >> > > >> > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > >> > > > > > > >> > > > >> > > > > > > >> > Hi, > > >> > > > > > > >> > > > >> > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > >> > > > > > > >> > memory allocation in U-Boot. > > >> > > > > > > >> > > > >> > > > > > > >> > 1. malloc() > > >> > > > > > > >> > > > >> > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > >> > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > >> > > > > > > >> > 10MB kernel or 100MB ramdisk). > > >> > > > > > > >> > > > >> > > > > > > >> > 2. lmb > > >> > > > > > > >> > > > >> > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > >> > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > >> > > > > > > >> > loading some images and booting. If the boot fails, then all lmb > > >> > > > > > > >> > allocations go away. > > >> > > > > > > >> > > > >> > > > > > > >> > lmb is set up by getting all available memory and then removing what > > >> > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > >> > > > > > > >> > > > >> > > > > > > >> > lmb reservations have a few flags so that areas of memory can be > > >> > > > > > > >> > provided with attributes > > >> > > > > > > >> > > > >> > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > >> > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded > > >> > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > >> > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > >> > > > > > > >> > this would be fairly easy to solve. > > >> > > > > > > >> > > > >> > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > >> > > > > > > >> > > > >> > > > > > > >> > 3. EFI > > >> > > > > > > >> > > > >> > > > > > > >> > EFI has its own memory-allocation tables. > > >> > > > > > > >> > > > >> > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > >> > > > > > > >> > function it can also do smaller allocations similar to malloc(), > > >> > > > > > > >> > although each one uses at least 4KB at present. > > >> > > > > > > >> > > > >> > > > > > > >> > EFI allocations do not go away when a boot fails. > > >> > > > > > > >> > > > >> > > > > > > >> > With EFI it is possible to add allocations post facto, in which case > > >> > > > > > > >> > they are added to the allocation table just as if the memory was > > >> > > > > > > >> > allocated with EFI to begin with. > > >> > > > > > > >> > > > >> > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > >> > > > > > > >> > principle could conflict. > > >> > > > > > > >> > > > >> > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > >> > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > >> > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > >> > > > > > > >> > in the app context and may need to access the memory after U-Boot has > > >> > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > >> > > > > > > >> > mapping as well. > > >> > > > > > > >> > > > >> > > > > > > >> > EFI memory has attributes, including what the memory is used for (to > > >> > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct > > >> > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > >> > > > > > > >> > memory is cacheable, etc. > > >> > > > > > > >> > > > >> > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > >> > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > >> > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. > > >> > > > > > > >> > > > >> > > > > > > >> > 4. reservations > > >> > > > > > > >> > > > >> > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > >> > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in > > >> > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > >> > > > > > > >> > region, etc. > > >> > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > > >> > Problems > > >> > > > > > > >> > —------- > > >> > > > > > > >> > > > >> > > > > > > >> > There are no urgent problems, but here are some things that could be improved: > > >> > > > > > > >> > > > >> > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > >> > > > > > > >> > work has started, with the partition support, but more effort would > > >> > > > > > > >> > help. This would make it easier to see which memory is related to > > >> > > > > > > >> > devices and which is separate. > > >> > > > > > > >> > > > >> > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > >> > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > >> > > > > > > >> > > > >> > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > >> > > > > > > >> > armv8's struct mm_region) to EFI ones. > > >> > > > > > > >> > > > >> > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > >> > > > > > > >> > memory allocation (e.g. FDT fixup). > > >> > > > > > > >> > > > >> > > > > > > >> > 5. EFI code is used even if EFI is never used to boot > > >> > > > > > > >> > > > >> > > > > > > >> > 6. EFI allocations can result in the same memory being used as has > > >> > > > > > > >> > already been allocated by lmb. Users may load files which overwrite > > >> > > > > > > >> > memory allocated by EFI. > > >> > > > > > > >> > > >> > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to > > >> > > > > > > >> ensure files do not overlap), without making it too difficult to load > > >> > > > > > > >> multiple files to the same place, if desired. > > >> > > > > > > >> > > >> > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > > >> > Lifetime > > >> > > > > > > >> > -------- > > >> > > > > > > >> > > > >> > > > > > > >> > We have three different memory allocators with different purposes. Can > > >> > > > > > > >> > we unify them a little? > > >> > > > > > > >> > > > >> > > > > > > >> > Within U-Boot: > > >> > > > > > > >> > - malloc() space lives forever > > >> > > > > > > >> > - lmb lives while setting out images for booting > > >> > > > > > > >> > - EFI (mostly) lives while booting an EFI app > > >> > > > > > > >> > > > >> > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > >> > > > > > > >> > some not. EFI allocations stay around forever. This works OK since > > >> > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really > > >> > > > > > > >> > consumed to any great degree by the boot process. > > >> > > > > > > >> > > > >> > > > > > > >> > What happens to EFI allocations if the app returns? They are still > > >> > > > > > > >> > present, in case another app is run. This seems fine. > > >> > > > > > > >> > > > >> > > > > > > >> > API > > >> > > > > > > >> > –-- > > >> > > > > > > >> > Can we unify some APIs? > > >> > > > > > > >> > > > >> > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > >> > > > > > > >> > long as they are only needed for booting. We effectively do this > > >> > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in > > >> > > > > > > >> > memory. for the most part. > > >> > > > > > > >> > > > >> > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > >> > > > > > > >> > malloc(), of course. > > >> > > > > > > >> > > > >> > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > >> > > > > > > >> > lmb and EFI? They do have some similarities. However they have > > >> > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, > > >> > > > > > > >> > unlikely lmb). > > >> > > > > > > >> > > > >> > > > > > > >> > ** Overall, it seems that the existence of memory allocation in > > >> > > > > > > >> > boot-time services has created confusion. Memory allocation is > > >> > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > >> > > > > > > >> > memory allocator. This just has not been clearly thought out. > > >> > > > > > > >> > > > >> > > > > > > >> > > > >> > > > > > > >> > Proposal > > >> > > > > > > >> > —------- > > >> > > > > > > >> > > > >> > > > > > > >> > Here are some ideas: > > >> > > > > > > >> > > > >> > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > >> > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > >> > > > > > > >> > Use efi_add_memory_map() > > >> > > > > > > >> > > > >> > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > >> > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > >> > > > > > > >> > > > >> > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > >> > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > >> > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > >> > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The > > >> > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data > > >> > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > >> > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > >> > > > > > > >> > understands the meaning of each. > > >> > > > > > > >> > > > >> > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > >> > > > > > > >> > booting. This looks to require very little effort. > > >> > > > > > > >> > > > >> > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > >> > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > >> > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two > > >> > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > >> > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > >> > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > >> > > > > > > >> > allocated in normal malloc() space or in a bloblist. > > >> > > > > > > >> > > > >> > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > >> > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is > > >> > > > > > > >> > done for driver model. Try to minmise the cost of its tables, > > >> > > > > > > >> > particularly for execution time, but otherwise just rely on the > > >> > > > > > > >> > ability to disable EFI_LOADER. > > >> > > > > > > >> > > >> > > > > > > >> 7. Add a flag to the 'load' command: > > >> > > > > > > >> > > >> > > > > > > >> -m <type> - make an lmb allocation for the file > > >> > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > >> > > > > > > >> > > >> > > > > > > >> any existing allocation for that type will be automatically freed > > >> > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > >> > > > > > > >> images just stack up in lmb. > > >> > > > > > > >> > > >> > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > >> > > > > > > > > > >> > > > > > > >I would really not like to change the user interface and instead simply > > >> > > > > > > >handle this with flags to whatever mark/allocation function is called. > > >> > > > > > > >You can always overwrite things that are brought in to memory, you > > >> > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > >> > > > > > > >previous load to memory has been at least partially overwritten could be > > >> > > > > > > >helpful, if it's not too much extra logic. > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > In most use cases users load exactly one file at each address. An > > >> > > > > > > unload command would be the cleanest way for a user to indicate that > > >> > > > > > > he wants to reuse the memory. > > >> > > > > > > > >> > > > > > I very much do not want to change the API. There's untold numbers of > > >> > > > > > scripts out there and they should continue to work. I mentioned to Ilias > > >> > > > > > off list just now that I'm not against adding a command to add flags to > > >> > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the > > >> > > > > > user did early. The biggest long running problem in this space was that > > >> > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > >> > > > > > so would have ramdisk at the wrong spot and get partially eaten, and > > >> > > > > > this was hard to figure out. The current example is "ooops, > > >> > > > > > decompression buffer for Image.gz/etc is too close to other things" > > >> > > > > > which ends up failing nice and loudly, and in the future once this > > >> > > > > > proposal is done we can just dynamically find and use a spot, since > > >> > > > > > we'll have that ability finally. > > >> > > > > > > >> > > > > In order to keep the existing interfaces we need lmb to keep track of > > >> > > > > (at least ) three different states. I think of those as "free", > > >> > > > > "allocated" and "reserved". The load command would "reserve" memory > > >> > > > > insteaf "allocate". And it would pass a flag to lmb when reserving > > >> > > > > memory to indicate that reserving memory that is already reserved is > > >> > > > > ok. Both "reserved" and "allocated" memory should show up as not free > > >> > > > > in the EFI memory map (probably as EfiLoaderData). > > >> > > > > > >> > > > Yes, something like this is what I was getting at, thanks. > > >> > > > > >> > > Yes, that is a good way of putting it. There is definitely a distinction there. > > >> > > > > >> > > If we don't want this flag, we could make U-Boot always do a > > >> > > reservation on load, with a '-f' command to force loading over an > > >> > > existing reservation / releasing it first? > > >> > > > >> > Again, this is an API change and I don't want to change the API. > > >> > > >> The flag is only needed to drop a reservation, since we apparently > > >> want the 'load' command to create a permanent reservation. It should > > >> not affect existing boot scripts since they won't load overlapping > > >> images. > > >> > > >> Anyway, what do you suggest? > > > > > >That "load" (and sf read and nand read and tftp and wget and ...) > > >"reserve" memory but not "allocate" memory and "reserve" means something > > >is there and "allocate" means that it can't be modified again. For > > >example, running U-Boot and our malloc pool are "allocated" but just > > >loading a file to memory is "reserved". And then yes, I can see use for > > >the command where some cases might want to "reserve" memory to fiddle > > >with it and then "allocate" it so something else can't change it. > > > > > >This is one of those cases where english is terrible to discuss things > > >in as both reserve and allocate can mean similar things. > > > > > > > I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A. > > > > Who can and who cannot write into "reserved" memory? > > > > What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory? > > So, historically (lets say mid 2010s), you could use "load" to bring a > file in to memory, anywhere, and it could even overwrite part of U-Boot > (running, or malloc pool or whatever). You could also use "load" to > bring a file in to memory and then bring another file in to that same > location in memory for whatever reason (assorted development cases). > > Then later someone noted that using "load" to overwrite U-Boot should > get a CVE and instead of ignoring it we decided to use "lmb" to try and > make sure that we couldn't use "load" to overwrite U-Boot itself, and > that that's a pre-check. This still allows overwriting a previously > loaded file in memory. > > Overwriting something the user put in memory is part of the ABI and has > some use cases. > > So for whatever future system we setup, a memory location can be: > - Free. > - Readable but not Writable. > - Readable and Writable. > > Something like $loadaddr starts as Free. If someone then uses "load" to > bring in an OS image, it's now "Readable and Writable". But if someone > does an API call to ask for a new region memory, it wouldn't return > $loadaddr because it's not Free. On the other hand, $relocaddr where > U-Boot is, at least in terms of the API is that it's "Readable but not > Writable". > > Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Can you please describe in a little more detail what "whole new class of problems" are being introduced here. If there are problems being introduced, it is better if they are enumerated, so that I can try to fix them.
Please see my point immediately below:
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
I have gone through this thread multiple times, and my understanding was that there was no conclusive agreement that was arrived at. I saw a reply from Tom where he broadly mentioned what kind of a solution he is looking for. I am not sure if you have gone through the cover-letter of my RFC series [1], which clearly states the problem my patch series is trying to fix. Like I said above, it would help if you can describe in a little more detail the kind of issues you see with the RFC series. Thanks.
My request was for you to reply to the thread with any thoughts so we can agree what to do about it. I believe my original email mostly stands as a summary of where things are and what the problems are. If it helps, I can update it based on the discussion above. It is important to avoid over-complicating U-Boot.
My fundamental expectation is that LMB reservations only apply for a particular boot. They should not be global, nor need freeing, nor should they be similar to memory allocations. We discussed it in this thread.
Please note that I have run the patches through CI, and they pass the CI run. There are numerous examples in the CI test scripts where the LMB memory is re-used, and these patches are not changing that behaviour in any which way. The only reason why the LMB memory needs to be persistent and global is because it needs to work cohesively with the EFI memory allocator.
I believe that is the central misunderstanding here. The EFI memory allocator was written as if it has access to all the memory in U-Boot. It is sort-of an allocator and sort-of a classifier, for memory. We need to be very careful to avoid conflating those two things.
Here I am repeating myself, but let me try.
Within U-Boot, there is a malloc() pool and that should be used for all allocations, including those which purport to be EFI allocations while U-Boot itself is running (and has not booted into an EFI app). I believe this is the case today, but we don't have a test for it, and the early init of EFI memory could be causing issues / confusion.
The EFI 'allocator' should be set up just before booting an EFI application, and only be used after boot, to handle requests from EFI applications.
LMB should also be set up on a per-boot basis. It is just there to avoid conflicts.
At this point, I guess the EFI maintainers and Tom have to provide their input on how we go about this. But if we have to have EFI allocations live alongside LMB, the *only* other solution that I see is to then break up the memory into two sections, one for LMB and one for EFI. But then again, that introduces it's own set of issues, for e.g. a platform might not be able to use some memory region simply because it is allocated to the other subsystem. But I would let the maintainers chime in at this point. Thanks.
As in the other thread, LMB is not an allocator, really. We should perhaps rename the functions to avoid any confusion.
I'd like to come back to the problems we are trying to solve. I listed 6...just to repeat:
There are no urgent problems, but here are some things that could be improved:
- EFI should attach most of its data structures to driver model. This
work has started, with the partition support, but more effort would help. This would make it easier to see which memory is related to devices and which is separate.
- Some drivers do EFI reservations today, whether EFI is used for
booting or not (e.g. rockchip video rk_vop_probe()).
- U-Boot doesn't really map arch-specific memory attributes (e.g.
armv8's struct mm_region) to EFI ones.
- EFI duplicates some code from bootm, some of which relates to
memory allocation (e.g. FDT fixup).
EFI code is used even if EFI is never used to boot
EFI allocations can result in the same memory being used as has
already been allocated by lmb. Users may load files which overwrite memory allocated by EFI.
Do you agree with those? Would you add others? It is this discussion which I believe is very important, before we end up re-inventing Tianocore :-)
Regards, SImon
On Mon, Jun 10, 2024 at 07:55:44AM -0600, Simon Glass wrote:
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com:
On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote: > Hi Tom, > > On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: > > > > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: > > > Hi, > > > > > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: > > > > > > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: > > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 > > > > > > From: Tom Rini trini@konsulko.com > > > > > > > > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: > > > > > > > > > > > > > > > > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: > > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: > > > > > > > >> Hi, > > > > > > > >> > > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: > > > > > > > >> > > > > > > > > >> > Hi, > > > > > > > >> > > > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re > > > > > > > >> > memory allocation in U-Boot. > > > > > > > >> > > > > > > > > >> > 1. malloc() > > > > > > > >> > > > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory > > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a > > > > > > > >> > 10MB kernel or 100MB ramdisk). > > > > > > > >> > > > > > > > > >> > 2. lmb > > > > > > > >> > > > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a > > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of > > > > > > > >> > loading some images and booting. If the boot fails, then all lmb > > > > > > > >> > allocations go away. > > > > > > > >> > > > > > > > > >> > lmb is set up by getting all available memory and then removing what > > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) > > > > > > > >> > > > > > > > > >> > lmb reservations have a few flags so that areas of memory can be > > > > > > > >> > provided with attributes > > > > > > > >> > > > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb > > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded > > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the > > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, > > > > > > > >> > this would be fairly easy to solve. > > > > > > > >> > > > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. > > > > > > > >> > > > > > > > > >> > 3. EFI > > > > > > > >> > > > > > > > > >> > EFI has its own memory-allocation tables. > > > > > > > >> > > > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' > > > > > > > >> > function it can also do smaller allocations similar to malloc(), > > > > > > > >> > although each one uses at least 4KB at present. > > > > > > > >> > > > > > > > > >> > EFI allocations do not go away when a boot fails. > > > > > > > >> > > > > > > > > >> > With EFI it is possible to add allocations post facto, in which case > > > > > > > >> > they are added to the allocation table just as if the memory was > > > > > > > >> > allocated with EFI to begin with. > > > > > > > >> > > > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in > > > > > > > >> > principle could conflict. > > > > > > > >> > > > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as > > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() > > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs > > > > > > > >> > in the app context and may need to access the memory after U-Boot has > > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new > > > > > > > >> > mapping as well. > > > > > > > >> > > > > > > > > >> > EFI memory has attributes, including what the memory is used for (to > > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct > > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether > > > > > > > >> > memory is cacheable, etc. > > > > > > > >> > > > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I > > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. > > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. > > > > > > > >> > > > > > > > > >> > 4. reservations > > > > > > > >> > > > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, > > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in > > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() > > > > > > > >> > region, etc. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Problems > > > > > > > >> > —------- > > > > > > > >> > > > > > > > > >> > There are no urgent problems, but here are some things that could be improved: > > > > > > > >> > > > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This > > > > > > > >> > work has started, with the partition support, but more effort would > > > > > > > >> > help. This would make it easier to see which memory is related to > > > > > > > >> > devices and which is separate. > > > > > > > >> > > > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for > > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). > > > > > > > >> > > > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. > > > > > > > >> > armv8's struct mm_region) to EFI ones. > > > > > > > >> > > > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to > > > > > > > >> > memory allocation (e.g. FDT fixup). > > > > > > > >> > > > > > > > > >> > 5. EFI code is used even if EFI is never used to boot > > > > > > > >> > > > > > > > > >> > 6. EFI allocations can result in the same memory being used as has > > > > > > > >> > already been allocated by lmb. Users may load files which overwrite > > > > > > > >> > memory allocated by EFI. > > > > > > > >> > > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to > > > > > > > >> ensure files do not overlap), without making it too difficult to load > > > > > > > >> multiple files to the same place, if desired. > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Lifetime > > > > > > > >> > -------- > > > > > > > >> > > > > > > > > >> > We have three different memory allocators with different purposes. Can > > > > > > > >> > we unify them a little? > > > > > > > >> > > > > > > > > >> > Within U-Boot: > > > > > > > >> > - malloc() space lives forever > > > > > > > >> > - lmb lives while setting out images for booting > > > > > > > >> > - EFI (mostly) lives while booting an EFI app > > > > > > > >> > > > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, > > > > > > > >> > some not. EFI allocations stay around forever. This works OK since > > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really > > > > > > > >> > consumed to any great degree by the boot process. > > > > > > > >> > > > > > > > > >> > What happens to EFI allocations if the app returns? They are still > > > > > > > >> > present, in case another app is run. This seems fine. > > > > > > > >> > > > > > > > > >> > API > > > > > > > >> > –-- > > > > > > > >> > Can we unify some APIs? > > > > > > > >> > > > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so > > > > > > > >> > long as they are only needed for booting. We effectively do this > > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in > > > > > > > >> > memory. for the most part. > > > > > > > >> > > > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and > > > > > > > >> > malloc(), of course. > > > > > > > >> > > > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both > > > > > > > >> > lmb and EFI? They do have some similarities. However they have > > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, > > > > > > > >> > unlikely lmb). > > > > > > > >> > > > > > > > > >> > ** Overall, it seems that the existence of memory allocation in > > > > > > > >> > boot-time services has created confusion. Memory allocation is > > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same > > > > > > > >> > memory allocator. This just has not been clearly thought out. > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > Proposal > > > > > > > >> > —------- > > > > > > > >> > > > > > > > > >> > Here are some ideas: > > > > > > > >> > > > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or > > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. > > > > > > > >> > Use efi_add_memory_map() > > > > > > > >> > > > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to > > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. > > > > > > > >> > > > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with > > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all > > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in > > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The > > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data > > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables > > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code > > > > > > > >> > understands the meaning of each. > > > > > > > >> > > > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when > > > > > > > >> > booting. This looks to require very little effort. > > > > > > > >> > > > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside > > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by > > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two > > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of > > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For > > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be > > > > > > > >> > allocated in normal malloc() space or in a bloblist. > > > > > > > >> > > > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. > > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is > > > > > > > >> > done for driver model. Try to minmise the cost of its tables, > > > > > > > >> > particularly for execution time, but otherwise just rely on the > > > > > > > >> > ability to disable EFI_LOADER. > > > > > > > >> > > > > > > > >> 7. Add a flag to the 'load' command: > > > > > > > >> > > > > > > > >> -m <type> - make an lmb allocation for the file > > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) > > > > > > > >> > > > > > > > >> any existing allocation for that type will be automatically freed > > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded > > > > > > > >> images just stack up in lmb. > > > > > > > >> > > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. > > > > > > > > > > > > > > > >I would really not like to change the user interface and instead simply > > > > > > > >handle this with flags to whatever mark/allocation function is called. > > > > > > > >You can always overwrite things that are brought in to memory, you > > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some > > > > > > > >previous load to memory has been at least partially overwritten could be > > > > > > > >helpful, if it's not too much extra logic. > > > > > > > > > > > > > > > > > > > > > > In most use cases users load exactly one file at each address. An > > > > > > > unload command would be the cleanest way for a user to indicate that > > > > > > > he wants to reuse the memory. > > > > > > > > > > > > I very much do not want to change the API. There's untold numbers of > > > > > > scripts out there and they should continue to work. I mentioned to Ilias > > > > > > off list just now that I'm not against adding a command to add flags to > > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the > > > > > > user did early. The biggest long running problem in this space was that > > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and > > > > > > so would have ramdisk at the wrong spot and get partially eaten, and > > > > > > this was hard to figure out. The current example is "ooops, > > > > > > decompression buffer for Image.gz/etc is too close to other things" > > > > > > which ends up failing nice and loudly, and in the future once this > > > > > > proposal is done we can just dynamically find and use a spot, since > > > > > > we'll have that ability finally. > > > > > > > > > > In order to keep the existing interfaces we need lmb to keep track of > > > > > (at least ) three different states. I think of those as "free", > > > > > "allocated" and "reserved". The load command would "reserve" memory > > > > > insteaf "allocate". And it would pass a flag to lmb when reserving > > > > > memory to indicate that reserving memory that is already reserved is > > > > > ok. Both "reserved" and "allocated" memory should show up as not free > > > > > in the EFI memory map (probably as EfiLoaderData). > > > > > > > > Yes, something like this is what I was getting at, thanks. > > > > > > Yes, that is a good way of putting it. There is definitely a distinction there. > > > > > > If we don't want this flag, we could make U-Boot always do a > > > reservation on load, with a '-f' command to force loading over an > > > existing reservation / releasing it first? > > > > Again, this is an API change and I don't want to change the API. > > The flag is only needed to drop a reservation, since we apparently > want the 'load' command to create a permanent reservation. It should > not affect existing boot scripts since they won't load overlapping > images. > > Anyway, what do you suggest?
That "load" (and sf read and nand read and tftp and wget and ...) "reserve" memory but not "allocate" memory and "reserve" means something is there and "allocate" means that it can't be modified again. For example, running U-Boot and our malloc pool are "allocated" but just loading a file to memory is "reserved". And then yes, I can see use for the command where some cases might want to "reserve" memory to fiddle with it and then "allocate" it so something else can't change it.
This is one of those cases where english is terrible to discuss things in as both reserve and allocate can mean similar things.
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
Again, the fundamental problem is that LMB as originally introduced was fine enough for the time it was introduced and the limited sets of problems we were caring about, which was essentially ONLY "put a few objects in non-conflicting areas for the OS". That's no longer the case, so we need something different that IS persistent from caller to caller. I do need to read Sughosh's series, but it is the starting point for discussion of an implementation for modern systems and problems, not a call to go back to the drawing board.
Hi Tom,
On Mon, 10 Jun 2024 at 10:29, Tom Rini trini@konsulko.com wrote:
On Mon, Jun 10, 2024 at 07:55:44AM -0600, Simon Glass wrote:
Hi Sughosh,
On Thu, 6 Jun 2024 at 13:18, Sughosh Ganu sughosh.ganu@linaro.org wrote:
hi Simon,
On Wed, 29 May 2024 at 22:00, Simon Glass sjg@chromium.org wrote:
+Sughosh Ganu for reference
On Sun, 31 Dec 2023 at 09:16, Tom Rini trini@konsulko.com wrote:
On Sun, Dec 31, 2023 at 04:40:06PM +0100, Heinrich Schuchardt wrote:
Am 31. Dezember 2023 16:11:44 MEZ schrieb Tom Rini trini@konsulko.com: >On Sun, Dec 31, 2023 at 07:22:10AM -0700, Simon Glass wrote: >> Hi Tom, >> >> On Sun, Dec 31, 2023 at 6:54 AM Tom Rini trini@konsulko.com wrote: >> > >> > On Sun, Dec 31, 2023 at 05:48:23AM -0700, Simon Glass wrote: >> > > Hi, >> > > >> > > On Fri, Dec 29, 2023 at 10:52 AM Tom Rini trini@konsulko.com wrote: >> > > > >> > > > On Fri, Dec 29, 2023 at 06:44:15PM +0100, Mark Kettenis wrote: >> > > > > > Date: Fri, 29 Dec 2023 11:17:44 -0500 >> > > > > > From: Tom Rini trini@konsulko.com >> > > > > > >> > > > > > On Fri, Dec 29, 2023 at 05:05:17PM +0100, Heinrich Schuchardt wrote: >> > > > > > > >> > > > > > > >> > > > > > > Am 29. Dezember 2023 16:43:07 MEZ schrieb Tom Rini trini@konsulko.com: >> > > > > > > >On Fri, Dec 29, 2023 at 05:36:09AM +0000, Simon Glass wrote: >> > > > > > > >> Hi, >> > > > > > > >> >> > > > > > > >> On Sat, Dec 16, 2023 at 6:01 PM Simon Glass sjg@chromium.org wrote: >> > > > > > > >> > >> > > > > > > >> > Hi, >> > > > > > > >> > >> > > > > > > >> > This records my thoughts after a discussion with Ilias & Heinrich re >> > > > > > > >> > memory allocation in U-Boot. >> > > > > > > >> > >> > > > > > > >> > 1. malloc() >> > > > > > > >> > >> > > > > > > >> > malloc() is used for programmatic memory allocation. It allows memory >> > > > > > > >> > to be freed. It is not designed for very large allocations (e.g. a >> > > > > > > >> > 10MB kernel or 100MB ramdisk). >> > > > > > > >> > >> > > > > > > >> > 2. lmb >> > > > > > > >> > >> > > > > > > >> > lmb is used for large blocks of memory, such as those needed for a >> > > > > > > >> > kernel or ramdisk. Allocation is only transitory, for the purposes of >> > > > > > > >> > loading some images and booting. If the boot fails, then all lmb >> > > > > > > >> > allocations go away. >> > > > > > > >> > >> > > > > > > >> > lmb is set up by getting all available memory and then removing what >> > > > > > > >> > is used by U-Boot (code, data, malloc() space, etc.) >> > > > > > > >> > >> > > > > > > >> > lmb reservations have a few flags so that areas of memory can be >> > > > > > > >> > provided with attributes >> > > > > > > >> > >> > > > > > > >> > There are some corner cases...e.g. loading a file does an lmb >> > > > > > > >> > allocation but only for the purpose of avoiding a file being loaded >> > > > > > > >> > over U-Boot code/data. The allocation is dropped immediately after the >> > > > > > > >> > file is loaded. Within the bootm command, or when using standard boot, >> > > > > > > >> > this would be fairly easy to solve. >> > > > > > > >> > >> > > > > > > >> > Linux has renamed lmb to memblock. We should consider doing the same. >> > > > > > > >> > >> > > > > > > >> > 3. EFI >> > > > > > > >> > >> > > > > > > >> > EFI has its own memory-allocation tables. >> > > > > > > >> > >> > > > > > > >> > Like lmb, EFI is able to deal with large allocations. But via a 'pool' >> > > > > > > >> > function it can also do smaller allocations similar to malloc(), >> > > > > > > >> > although each one uses at least 4KB at present. >> > > > > > > >> > >> > > > > > > >> > EFI allocations do not go away when a boot fails. >> > > > > > > >> > >> > > > > > > >> > With EFI it is possible to add allocations post facto, in which case >> > > > > > > >> > they are added to the allocation table just as if the memory was >> > > > > > > >> > allocated with EFI to begin with. >> > > > > > > >> > >> > > > > > > >> > The EFI allocations and the lmb allocations use the same memory, so in >> > > > > > > >> > principle could conflict. >> > > > > > > >> > >> > > > > > > >> > EFI allocations are sometimes used to allocate internal U-Boot data as >> > > > > > > >> > well, if needed by the EFI app. For example, while efi_image_parse() >> > > > > > > >> > uses malloc(), efi_var_mem.c uses EFI allocations since the code runs >> > > > > > > >> > in the app context and may need to access the memory after U-Boot has >> > > > > > > >> > exited. Also efi_smbios.c uses allocate_pages() and then adds a new >> > > > > > > >> > mapping as well. >> > > > > > > >> > >> > > > > > > >> > EFI memory has attributes, including what the memory is used for (to >> > > > > > > >> > some degree of granularity). See enum efi_memory_type and struct >> > > > > > > >> > efi_mem_desc. In the latter there are also attribute flags - whether >> > > > > > > >> > memory is cacheable, etc. >> > > > > > > >> > >> > > > > > > >> > EFI also has the x86 idea of 'conventional' memory, meaning (I >> > > > > > > >> > believe) that below 4GB that isn't reserved for the hardware/system. >> > > > > > > >> > This is meaningless, or at least confusing, on ARM systems. >> > > > > > > >> > >> > > > > > > >> > 4. reservations >> > > > > > > >> > >> > > > > > > >> > It is perhaps worth mentioning a fourth method of memory management, >> > > > > > > >> > where U-Boot reserves chunks of memory before relocation (in >> > > > > > > >> > board_init_f.c), e.g. for the framebuffer, U-Boot code, the malloc() >> > > > > > > >> > region, etc. >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > Problems >> > > > > > > >> > —------- >> > > > > > > >> > >> > > > > > > >> > There are no urgent problems, but here are some things that could be improved: >> > > > > > > >> > >> > > > > > > >> > 1. EFI should attach most of its data structures to driver model. This >> > > > > > > >> > work has started, with the partition support, but more effort would >> > > > > > > >> > help. This would make it easier to see which memory is related to >> > > > > > > >> > devices and which is separate. >> > > > > > > >> > >> > > > > > > >> > 2. Some drivers do EFI reservations today, whether EFI is used for >> > > > > > > >> > booting or not (e.g. rockchip video rk_vop_probe()). >> > > > > > > >> > >> > > > > > > >> > 3. U-Boot doesn't really map arch-specific memory attributes (e.g. >> > > > > > > >> > armv8's struct mm_region) to EFI ones. >> > > > > > > >> > >> > > > > > > >> > 4. EFI duplicates some code from bootm, some of which relates to >> > > > > > > >> > memory allocation (e.g. FDT fixup). >> > > > > > > >> > >> > > > > > > >> > 5. EFI code is used even if EFI is never used to boot >> > > > > > > >> > >> > > > > > > >> > 6. EFI allocations can result in the same memory being used as has >> > > > > > > >> > already been allocated by lmb. Users may load files which overwrite >> > > > > > > >> > memory allocated by EFI. >> > > > > > > >> >> > > > > > > >> 7. We need to support doing an allocation when a file is loaded (to >> > > > > > > >> ensure files do not overlap), without making it too difficult to load >> > > > > > > >> multiple files to the same place, if desired. >> > > > > > > >> >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > Lifetime >> > > > > > > >> > -------- >> > > > > > > >> > >> > > > > > > >> > We have three different memory allocators with different purposes. Can >> > > > > > > >> > we unify them a little? >> > > > > > > >> > >> > > > > > > >> > Within U-Boot: >> > > > > > > >> > - malloc() space lives forever >> > > > > > > >> > - lmb lives while setting out images for booting >> > > > > > > >> > - EFI (mostly) lives while booting an EFI app >> > > > > > > >> > >> > > > > > > >> > In practice, EFI is set up early in U-Boot. Some of this is necessary, >> > > > > > > >> > some not. EFI allocations stay around forever. This works OK since >> > > > > > > >> > large allocations are normally not done in EFI, so memory isn't really >> > > > > > > >> > consumed to any great degree by the boot process. >> > > > > > > >> > >> > > > > > > >> > What happens to EFI allocations if the app returns? They are still >> > > > > > > >> > present, in case another app is run. This seems fine. >> > > > > > > >> > >> > > > > > > >> > API >> > > > > > > >> > –-- >> > > > > > > >> > Can we unify some APIs? >> > > > > > > >> > >> > > > > > > >> > It should be possible to use lmb for large EFI memory allocations, so >> > > > > > > >> > long as they are only needed for booting. We effectively do this >> > > > > > > >> > today, since EFI does not manage the arrangement of loaded images in >> > > > > > > >> > memory. for the most part. >> > > > > > > >> > >> > > > > > > >> > It would not make sense to use EFI allocation to replace lmb and >> > > > > > > >> > malloc(), of course. >> > > > > > > >> > >> > > > > > > >> > Could we use a common (lower-level) API for allocation, used by both >> > > > > > > >> > lmb and EFI? They do have some similarities. However they have >> > > > > > > >> > different lifetime constraints (EFI allocations are never dropped, >> > > > > > > >> > unlikely lmb). >> > > > > > > >> > >> > > > > > > >> > ** Overall, it seems that the existence of memory allocation in >> > > > > > > >> > boot-time services has created confusion. Memory allocation is >> > > > > > > >> > muddled, with both U-Boot code and boot-time services calling the same >> > > > > > > >> > memory allocator. This just has not been clearly thought out. >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > Proposal >> > > > > > > >> > —------- >> > > > > > > >> > >> > > > > > > >> > Here are some ideas: >> > > > > > > >> > >> > > > > > > >> > 1. For video, use the driver model API to locate the video regions, or >> > > > > > > >> > block off the entire framebuffer memory, for all devices as a whole. >> > > > > > > >> > Use efi_add_memory_map() >> > > > > > > >> > >> > > > > > > >> > 2. Add memory attributes to UCLASS_RAM and use them in EFI, mapping to >> > > > > > > >> > the EFI_MEMORY_... attributes in struct efi_mem_desc. >> > > > > > > >> > >> > > > > > > >> > 3. Add all EFI reservations just before booting the app, as we do with >> > > > > > > >> > devicetree fixup. With this model, malloc() and lmb are used for all >> > > > > > > >> > allocation. Then efi_add_memory_map() is called for each region in >> > > > > > > >> > turn just before booting. Memory attributes are dealt with above. The >> > > > > > > >> > type (enum efi_memory_type) can be determined simply by the data >> > > > > > > >> > structure stored in it, as is done today. For example, SMBIOS tables >> > > > > > > >> > can use EFI_ACPI_RECLAIM_MEMORY. Very few types are used and EFI code >> > > > > > > >> > understands the meaning of each. >> > > > > > > >> > >> > > > > > > >> > 4. Avoid setting up EFI memory at the start of U-Boot. Do it only when >> > > > > > > >> > booting. This looks to require very little effort. >> > > > > > > >> > >> > > > > > > >> > 5. Avoid calling efi_allocate_pages() and efi_allocate_pool() outside >> > > > > > > >> > boot-time services. This solves the problem 6. If memory is needed by >> > > > > > > >> > an app, allocate it with malloc() and see 3. There are only two >> > > > > > > >> > efi_allocate_pages() (smbios and efi_runtime). There are more calls of >> > > > > > > >> > efi_allocate_pool(), but most of these seem easy to fix up. For >> > > > > > > >> > example, efi_init_event_log() allocates a buffer, but this can be >> > > > > > > >> > allocated in normal malloc() space or in a bloblist. >> > > > > > > >> > >> > > > > > > >> > 6. Don't worry too much about whether EFI will be used for booting. >> > > > > > > >> > The cost is likely not that great: use bootstage to measure it as is >> > > > > > > >> > done for driver model. Try to minmise the cost of its tables, >> > > > > > > >> > particularly for execution time, but otherwise just rely on the >> > > > > > > >> > ability to disable EFI_LOADER. >> > > > > > > >> >> > > > > > > >> 7. Add a flag to the 'load' command: >> > > > > > > >> >> > > > > > > >> -m <type> - make an lmb allocation for the file >> > > > > > > >> <type> is the image type to use (kernel, ramdisk, flat_dt) >> > > > > > > >> >> > > > > > > >> any existing allocation for that type will be automatically freed >> > > > > > > >> first. If <type> is "none" then no freeing is possible: any loaded >> > > > > > > >> images just stack up in lmb. >> > > > > > > >> >> > > > > > > >> Add an 'lmb' (or memblock) command to allow listing and clearing allocations. >> > > > > > > > >> > > > > > > >I would really not like to change the user interface and instead simply >> > > > > > > >handle this with flags to whatever mark/allocation function is called. >> > > > > > > >You can always overwrite things that are brought in to memory, you >> > > > > > > >cannot overwrite U-Boot or our internals. Optionally noting that some >> > > > > > > >previous load to memory has been at least partially overwritten could be >> > > > > > > >helpful, if it's not too much extra logic. >> > > > > > > > >> > > > > > > >> > > > > > > In most use cases users load exactly one file at each address. An >> > > > > > > unload command would be the cleanest way for a user to indicate that >> > > > > > > he wants to reuse the memory. >> > > > > > >> > > > > > I very much do not want to change the API. There's untold numbers of >> > > > > > scripts out there and they should continue to work. I mentioned to Ilias >> > > > > > off list just now that I'm not against adding a command to add flags to >> > > > > > these areas, but I don't think it's worthwhile to prevent overwrites the >> > > > > > user did early. The biggest long running problem in this space was that >> > > > > > for 32bit ARM we couldn't know where the kernel BSS was going to be and >> > > > > > so would have ramdisk at the wrong spot and get partially eaten, and >> > > > > > this was hard to figure out. The current example is "ooops, >> > > > > > decompression buffer for Image.gz/etc is too close to other things" >> > > > > > which ends up failing nice and loudly, and in the future once this >> > > > > > proposal is done we can just dynamically find and use a spot, since >> > > > > > we'll have that ability finally. >> > > > > >> > > > > In order to keep the existing interfaces we need lmb to keep track of >> > > > > (at least ) three different states. I think of those as "free", >> > > > > "allocated" and "reserved". The load command would "reserve" memory >> > > > > insteaf "allocate". And it would pass a flag to lmb when reserving >> > > > > memory to indicate that reserving memory that is already reserved is >> > > > > ok. Both "reserved" and "allocated" memory should show up as not free >> > > > > in the EFI memory map (probably as EfiLoaderData). >> > > > >> > > > Yes, something like this is what I was getting at, thanks. >> > > >> > > Yes, that is a good way of putting it. There is definitely a distinction there. >> > > >> > > If we don't want this flag, we could make U-Boot always do a >> > > reservation on load, with a '-f' command to force loading over an >> > > existing reservation / releasing it first? >> > >> > Again, this is an API change and I don't want to change the API. >> >> The flag is only needed to drop a reservation, since we apparently >> want the 'load' command to create a permanent reservation. It should >> not affect existing boot scripts since they won't load overlapping >> images. >> >> Anyway, what do you suggest? > >That "load" (and sf read and nand read and tftp and wget and ...) >"reserve" memory but not "allocate" memory and "reserve" means something >is there and "allocate" means that it can't be modified again. For >example, running U-Boot and our malloc pool are "allocated" but just >loading a file to memory is "reserved". And then yes, I can see use for >the command where some cases might want to "reserve" memory to fiddle >with it and then "allocate" it so something else can't change it. > >This is one of those cases where english is terrible to discuss things >in as both reserve and allocate can mean similar things. >
I have no clue what the semantics of the mentioned "reserved" state might be. Up to now memory reservations designated address ranges that U-Boot must not use at all, e.g. the memory used by OpenSBI or TF-A.
Who can and who cannot write into "reserved" memory?
What is wrong about allocating memory for files to forbid any other use until you are done with the file and free the memory?
So, historically (lets say mid 2010s), you could use "load" to bring a file in to memory, anywhere, and it could even overwrite part of U-Boot (running, or malloc pool or whatever). You could also use "load" to bring a file in to memory and then bring another file in to that same location in memory for whatever reason (assorted development cases).
Then later someone noted that using "load" to overwrite U-Boot should get a CVE and instead of ignoring it we decided to use "lmb" to try and make sure that we couldn't use "load" to overwrite U-Boot itself, and that that's a pre-check. This still allows overwriting a previously loaded file in memory.
Overwriting something the user put in memory is part of the ABI and has some use cases.
So for whatever future system we setup, a memory location can be:
- Free.
- Readable but not Writable.
- Readable and Writable.
Something like $loadaddr starts as Free. If someone then uses "load" to bring in an OS image, it's now "Readable and Writable". But if someone does an API call to ask for a new region memory, it wouldn't return $loadaddr because it's not Free. On the other hand, $relocaddr where U-Boot is, at least in terms of the API is that it's "Readable but not Writable".
Does that help?
Thanks for sharing this with me. I have been working on a solution for this issue [1], and plan to send a rfc series by the end of this week. My primary motivation behind this work is to have some kind of synchronisation between the allocations done by the LMB module and the EFI subsystem. As you are aware, the way things stand today, either of the two can make memory allocations overwriting any earlier allocated memory. The primary reasons behind this is 1) the LMB allocations are private to their respective callers, so the LMB memory map is not global and persistent, and 2) the EFI allocation routines do not have any view of the allocations that would have been made by the LMB module.
To fix these issues, I am working on a solution which 1) makes LMB memory map persistent and global, and 2) have notifications between the two modules when either of the two has made any changes to it's memory map. This way, it would be possible to prohibit one module from allocating memory which is in use.
I am aware that we have a whole bunch of tests and user scripts which work with an assumption that the memory can be re-used. The solution that I would be proposing takes care of this assumption. Hopefully I should be in a position to send a rfc series this week. We can discuss on the patches. Thanks.
I actually am not sure what we are on the same page here. I described the problems in my original email but your series seems to solve somewhat different problems, while IMO introducing a whole new class of problems.
Making LMB persistent and global is a solution looking for a problem, IMO. Can we go up a level and figure out exactly what is broken here? That is what I attempted to do in my original email. Perhaps you could reply to that and suggest where you agree / disagree?
Again, the fundamental problem is that LMB as originally introduced was fine enough for the time it was introduced and the limited sets of problems we were caring about, which was essentially ONLY "put a few objects in non-conflicting areas for the OS". That's no longer the case, so we need something different that IS persistent from caller to caller. I do need to read Sughosh's series, but it is the starting point for discussion of an implementation for modern systems and problems, not a call to go back to the drawing board.
OK I will make a few comments there.
Regards, Simon
participants (6)
-
Heinrich Schuchardt -
Ilias Apalodimas -
Mark Kettenis -
Simon Glass -
Sughosh Ganu -
Tom Rini