[U-Boot] [RFC] FAT errors
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
Best regards
Heinrich
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
While I don't deny shifting to other code base for FAT, I'm willing to debug the current code if you send me binary data of corrupted file system. The first 1MB or so, which will contains directory meta data, would be good enough as I said before.
I think that most errors stem from wrong long-file-name handling.
-Takahiro Akashi
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
Best regards
Heinrich
On 9/5/19 10:37 AM, AKASHI Takahiro wrote:
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
While I don't deny shifting to other code base for FAT, I'm willing to debug the current code if you send me binary data of corrupted file system. The first 1MB or so, which will contains directory meta data, would be good enough as I said before.
I think that most errors stem from wrong long-file-name handling.
-Takahiro Akashi
The SCT test that really shows the trouble is for EFI_FILE_PROTOCOL.GetInfo(). On FAT16 it hits the 512 files per directory limit.
Both on FAT16 and FAT32 it crashes when saving the log.
Observed on qemu_arm64_defconfig with edk2-test HEAD.
Best regards
Heinrich
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
Best regards
Heinrich
On Sun, Sep 08, 2019 at 01:49:30AM +0200, Heinrich Schuchardt wrote:
On 9/5/19 10:37 AM, AKASHI Takahiro wrote:
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
While I don't deny shifting to other code base for FAT, I'm willing to debug the current code if you send me binary data of corrupted file system. The first 1MB or so, which will contains directory meta data, would be good enough as I said before.
I think that most errors stem from wrong long-file-name handling.
-Takahiro Akashi
The SCT test that really shows the trouble is for EFI_FILE_PROTOCOL.GetInfo(). On FAT16 it hits the 512 files per directory limit.
Please note that, on FAT16, the root directory can have a fixed number of blocks and that there is definitely an upper limit of maximum number of entries allowed under it.
Both on FAT16 and FAT32 it crashes when saving the log.
Did you see any message, or the file system got corrupted silently?
-Takahiro Akashi
Observed on qemu_arm64_defconfig with edk2-test HEAD.
Best regards
Heinrich
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
Best regards
Heinrich
On 9/10/19 2:53 AM, AKASHI Takahiro wrote:
On Sun, Sep 08, 2019 at 01:49:30AM +0200, Heinrich Schuchardt wrote:
On 9/5/19 10:37 AM, AKASHI Takahiro wrote:
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
While I don't deny shifting to other code base for FAT, I'm willing to debug the current code if you send me binary data of corrupted file system. The first 1MB or so, which will contains directory meta data, would be good enough as I said before.
I think that most errors stem from wrong long-file-name handling.
-Takahiro Akashi
The SCT test that really shows the trouble is for EFI_FILE_PROTOCOL.GetInfo(). On FAT16 it hits the 512 files per directory limit.
Please note that, on FAT16, the root directory can have a fixed number of blocks and that there is definitely an upper limit of maximum number of entries allowed under it.
Both on FAT16 and FAT32 it crashes when saving the log.
Did you see any message, or the file system got corrupted silently?
On FAT16 I saw messages indicating that the directory capacity was exhausted ("Error: allocating new dir entry"). So I switched to FAT32.
When using FAT32 and running appended getinfo.seq I do not see anything special except for the crash in guidcmp() due to some illegal address passed (see trace.txt).
After the test:
$ sudo losetup -o 1048576 /dev/loop1 sct-arm64.img $ sudo dosfsck /dev/loop1 fsck.fat 4.1 (2017-01-24) Orphaned long file name part "Sct.log" 1: Delete. 2: Leave it. ? 2 Orphaned long file name part "verbose.mode" 1: Delete. 2: Leave it. ? 2 Orphaned long file name part "[abc123]][()].Aa [b" 1: Delete. 2: Leave it. ? 2 Orphaned long file name part "[abc123]][()].Aa [b" 1: Delete. 2: Leave it. ? 2 Orphaned long file name part "BBTestGetInfoBasicTestCheckpoint3_File_0" 1: Delete. 2: Leave it.
Best regards
Heinrich
-Takahiro Akashi
Observed on qemu_arm64_defconfig with edk2-test HEAD.
Best regards
Heinrich
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log
Duplicate directory entry.
First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
Best regards
Heinrich
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
I think that or similar has been suggested before. Are you volunteering to maintain the wrapper and keep it in-sync? Thanks!
On 9/5/19 2:06 PM, Tom Rini wrote:
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
I think that or similar has been suggested before. Are you volunteering to maintain the wrapper and keep it in-sync? Thanks!
Here is the reference
Stephen Warren [PATCH 0/9] Replace the FAT filesystem code https://lists.denx.de/pipermail/u-boot/2015-August/222545.html
Regards
Heinrich
On 9/5/19 6:22 AM, Heinrich Schuchardt wrote:
On 9/5/19 2:06 PM, Tom Rini wrote:
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log
Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
I think that or similar has been suggested before. Are you volunteering to maintain the wrapper and keep it in-sync? Thanks!
Here is the reference
Stephen Warren [PATCH 0/9] Replace the FAT filesystem code https://lists.denx.de/pipermail/u-boot/2015-August/222545.html
IIRC, we abandoned that effort since ff turned out to be very slow. I'm not sure whether adding a disk caching library underneath it would have helped or not. Probably you can find those discussions from the mailing list post above?
Another FAT implementation that might be worth looking at is from FreeRTOS:
https://freertos.org/FreeRTOS-Plus/FreeRTOS_Plus_FAT/index.html
FreeRTOS itself recently got bought by Amazon and switched to an MIT license. I assume the same is true of their FAT code. So, should be OK to integrate.
On 9/5/19 5:23 PM, Stephen Warren wrote:
On 9/5/19 6:22 AM, Heinrich Schuchardt wrote:
On 9/5/19 2:06 PM, Tom Rini wrote:
On Thu, Sep 05, 2019 at 08:43:43AM +0200, Heinrich Schuchardt wrote:
Currently we do no have a maintainer for the FAT file system. Takahiro has done a great job fixing some of the most prominent deficiencies. But still the driver is not in good shape:
I once again ran upon errors in FAT when executing the UEFI SCT.
Here is some of the output of dosfsck -w -r -l -a -v -t The full output has hundreds of errors recorded.
Orphaned long file name part "Sct.log" Auto-deleting. Orphaned long file name part "Sct.log" Auto-deleting. /Log/RuntimeServicesTest/VariableServicesTest0/QueryVariableInfo_Conf_0_0_61758774-91A3-47DD-BDBD-B81094A5F62D.log
Duplicate directory entry. First Size 4712 bytes, date 01:00:00 Dec 31 1979 Second Size 5086 bytes, date 01:00:00 Dec 31 1979 Auto-renaming second. Renamed to FSCK0000.008
BareBox is using a (somewhat outdated) copy of this library with a little bit of wrapper code:
FatFs - Generic FAT Filesystem Module http://elm-chan.org/fsw/ff/00index_e.html http://elm-chan.org/fsw/ff/arc/ff13c.zip
The same library is also used for Arduinos: https://github.com/stm32duino/FatFs
Shouldn't we try just the same?
I think that or similar has been suggested before. Are you volunteering to maintain the wrapper and keep it in-sync? Thanks!
Here is the reference
Stephen Warren [PATCH 0/9] Replace the FAT filesystem code https://lists.denx.de/pipermail/u-boot/2015-August/222545.html
IIRC, we abandoned that effort since ff turned out to be very slow. I'm not sure whether adding a disk caching library underneath it would have helped or not. Probably you can find those discussions from the mailing list post above?
Another FAT implementation that might be worth looking at is from FreeRTOS:
https://freertos.org/FreeRTOS-Plus/FreeRTOS_Plus_FAT/index.html
FreeRTOS itself recently got bought by Amazon and switched to an MIT license. I assume the same is true of their FAT code. So, should be OK to integrate.
Thanks for the link. The MIT license has been applied to Free RTOS plus FAT. It seems to much more code than what ElmChan wrote. So we will have to look check the increase of U-Boot.
Best regards
Heinrich
participants (4)
-
AKASHI Takahiro -
Heinrich Schuchardt -
Stephen Warren -
Tom Rini