Am 27. Mai 2022 09:52:04 MESZ schrieb Oleksii Kurochko oleksii.kurochko@gmail.com:
Hello Heinrich,
get_maintainer.pl script told me that you are a maintainer of dlmalloc.c
Hello Oleksii,
I am not the maintainer of dlmalloc.c but only a contributor.
so if you don't mind I would like to ask you for advice about how could debugging of dlmalloc allocation be done?
I faced the issue that hanging happens during the freeing of resources [ https://elixir.bootlin.com/u-boot/latest/source/arch/arm/lib/bootm.c#L77] before U-boot finally will transfer control to Linux Kernel. I did a little investigation and found out that hanging happens here [ https://elixir.bootlin.com/u-boot/latest/source/common/dlmalloc.c#L940] and it happens because FD->fd points to itself so a cycle occurs.
Does the problem exist in mainline U-Boot? How can it be reproduced in mainline U-Boot?
If it is a Bootlin specific problem, please, consider contacting that company.
Possible reasons for running into problems in dlmalloc.c are double frees and out of bound writing to memory.
Best regards
Heinrich
Should be a check added here which will check that FD->fd does not point to itself or does a guarantee exist that the cycle can't be at all?
Also, I tried to do "#define DEBUG" at the top of dlmalloc.c and I started to be asserted here: https://elixir.bootlin.com/u-boot/latest/source/common/dlmalloc.c#L842 during the mentioned transfer from U-boot to kernel.
I also tried to understand why it is in the used state and I couldn't find where it is actually switched to the unused state. At least clear_inuse() macros aren't used elsewhere in file dlmalloc.c.
Interesting behavior has been obtained. It looks like that for the number of lines in CONFIG_EXTRA_ENV_SETTINGS there is no hanging at all during mentioned above transfer. So I can assume that it might be allocated a buffer or pointer to its buffer is overwritten because of the memory alignment of an array that uses CONFIG_EXTRA_ENV_SETTINGS.
Could you please give me some advice on how such behavior could be debugged?
Thanks in advance.
Best regards,
Oleksii