[U-Boot-Users] Little endian
How much testing has been done on little-endian mode of u-boot? Does everyone pretty much just use big-endian, or is there a mix? I am working on getting the Compact Flash to work properly on the Au1500, and I know there is some weirdness with this particular chip, but I just wondered if anyone else has similar problems in little-endian mode. I noticed that there was special handling done in common/cmd_ide.c for input_swap_data(), but not for input_data(). It would seem to me that if one of them was in the wrong endian mode, the other other would be as well, or did I miss the point? Same thing for input_data_short()? And if they bytes need to be swapped in the input routines, don't they need to be swapped on the output routines as well? Does all of this only need to be done for the AU1X00 series processors?
Ed Okerson
Dear Ed,
in message Pine.LNX.4.44.0311261130170.20886-100000@dallas.texasconnect.net you wrote:
How much testing has been done on little-endian mode of u-boot? Does everyone pretty much just use big-endian, or is there a mix? I am working
Frankly, I don't understand your question. Both endianesses are equally well supported - PowerPC is by default bigendian, most ARM systems are little-endian.
there was special handling done in common/cmd_ide.c for input_swap_data(), but not for input_data(). It would seem to me that if one of them was in the wrong endian mode, the other other would be as well, or did I miss
IIRC this has been discussed before.
output routines as well? Does all of this only need to be done for the AU1X00 series processors?
Sorery, I never worked with such a processor yet, so I cannot comment here.
Best regards,
Wolfgang Denk
On Wed, 26 Nov 2003, Wolfgang Denk wrote:
Dear Ed,
in message Pine.LNX.4.44.0311261130170.20886-100000@dallas.texasconnect.net you wrote:
How much testing has been done on little-endian mode of u-boot? Does everyone pretty much just use big-endian, or is there a mix? I am working
Frankly, I don't understand your question. Both endianesses are equally well supported - PowerPC is by default bigendian, most ARM systems are little-endian.
That is all I was asking.
there was special handling done in common/cmd_ide.c for input_swap_data(), but not for input_data(). It would seem to me that if one of them was in the wrong endian mode, the other other would be as well, or did I miss
IIRC this has been discussed before.
Yes it has, but the current workaround requires byteswapping the entire CF card on the PC platform before it can be used by the AU1500. This will be very inconvenient for my application. I want to correct that so the CF card can be moved between the PC and AU1500 with standard partitions and filesystems. In looking at the code I only saw where the little-endian issue was addressed for only one function and I didn't know if that was really the only place it needed to be addressed. My initial experiments seem to suggest that at least the input_data_shorts() function also needs to handle things differently for little-endian, at least on the AU1500. Making it swap bytes makes the AU1500 able to read at least the partition table on a CF card without the bytes being swapped. This led me to wonder if anyone else was having these problems in little-endian mode or if it was just me.
Ed Okerson
Hey Ed,
On Wed, 26 Nov 2003, Ed Okerson wrote:
Yes it has, but the current workaround requires byteswapping the entire CF card on the PC platform before it can be used by the AU1500. This will be very inconvenient for my application. I want to correct that so the CF card can be moved between the PC and AU1500 with standard partitions and filesystems. In looking at the code I only saw where the little-endian issue was addressed for only one function and I didn't know if that was really the only place it needed to be addressed. My initial experiments seem to suggest that at least the input_data_shorts() function also needs to handle things differently for little-endian, at least on the AU1500. Making it swap bytes makes the AU1500 able to read at least the partition table on a CF card without the bytes being swapped. This led me to wonder if anyone else was having these problems in little-endian mode or if it was just me.
I recently added PCMCIA support for PXA/WEPEP250 and have successfully used CompactFlash on it.
I don't know AU1500 processor, but the comment in cmd_ide.c says that processor swaps something automaticaly.
Paste: /* We only need to swap data if we are running on a big endian cpu. */ /* But Au1x00 cpu:s already swaps data in big endian mode! */ #if defined(__LITTLE_ENDIAN) || defined(CONFIG_AU1X00) #define input_swap_data(x,y,z) input_data(x,y,z) #else End of Paste
You will prolly know what has to be swapped and what not for AU1500.
Our PXA runs in little endian mode and all I had to tweak was the following:
Convert PCMCIA Firmware Revision and Model Number into little endian because they are big endian strings and take care of lba_capacity which is little endian but has been swapped, which worked fine on big endian CPU's but on little endian, gave wrong number of sectors.
Everything else seemed to work fine.
PS: I have verified it by using the same CompactFlash card on a PC and it works.
Hope that helps.
Leon,
On Thu, 27 Nov 2003, Leon KUKOVEC wrote:
Hey Ed,
table on a CF card without the bytes being swapped. This led me to wonder if anyone else was having these problems in little-endian mode or if it was just me.
I recently added PCMCIA support for PXA/WEPEP250 and have successfully used CompactFlash on it.
Congrats!
I don't know AU1500 processor, but the comment in cmd_ide.c says that processor swaps something automaticaly.
Paste: /* We only need to swap data if we are running on a big endian cpu. */ /* But Au1x00 cpu:s already swaps data in big endian mode! */ #if defined(__LITTLE_ENDIAN) || defined(CONFIG_AU1X00) #define input_swap_data(x,y,z) input_data(x,y,z) #else End of Paste
Yes, I saw that as well.
You will prolly know what has to be swapped and what not for AU1500.
This is, or was, the problem. I didn't know exactly what the problem was, mostly because of my own cofusion, but I still think it is a bug in the hardware, as Thomas pointed out earlier.
Our PXA runs in little endian mode and all I had to tweak was the following:
Convert PCMCIA Firmware Revision and Model Number into little endian because they are big endian strings and take care of lba_capacity which is little endian but has been swapped, which worked fine on big endian CPU's but on little endian, gave wrong number of sectors.
I have tweaked these as well for the Au1500, same problem as yours.
Everything else seemed to work fine.
What partition types are you using? I am trying to do a partition table with some linux and at least one FAT16 partition, but even without the FAT partition the system was showing a single partition as:
Partition Map for IDE device 0 -- Partition Type: DOS
Partition Start Sector Num Sectors Type 1 0 501760 f8
I poked around alot in disk/part_dos.c and found that if I remarked out the return after the check for PBR in print_partition_extended, then the rest of the partitions would print out:
Partition Map for IDE device 0 -- Partition Type: DOS
Partition Start Sector Num Sectors Type 1 0 501760 f8 1 32 4576 83 2 4608 4608 83 3 9216 492544 83
So this morning I started going through and figuring exactly what that meant. It appears that the first line shows up if the boot sector of the compact flash card has a DOS boot signature that is what you get. So apparently some idiot (probably me) at some point in time formated that CF card in a DOS machine with the /s option and it happily copied a DOS boot record into the beginning of the card. Boy do I feel stupid. So I popped the CF into a linux machine and typed "cat /dev/zero > /dev/hdg" re-ran fdisk to create my partitions, and now u-boot happily reports:
DbAu1x00 # ide part
Partition Map for IDE device 0 -- Partition Type: DOS
Partition Start Sector Num Sectors Type 1 32 4576 83 2 4608 4608 83 3 9216 492544 83 DbAu1x00 #
Which is what I wanted. I have been using #if defined(CONFIG_AU1X00) around my code mods, but it sounds like your changes and my changes are about the same in the end, so perhaps we should come up with another define for ENDIAN_CHALLENGED or something.
PS: I have verified it by using the same CompactFlash card on a PC and it works.
Hope that helps.
It helped a lot. Thanks. I guess the moral of the story is when you are having problems, make sure there isn't a short between the keyboard and the chair first. :)
Ed Okerson
Hello Ed,
On Thu, 27 Nov 2003, Ed Okerson wrote:
What partition types are you using? I am trying to do a partition table with some linux and at least one FAT16 partition, but even without the FAT partition the system was showing a single partition as:
I have setup two partitions using fdisk on target. fdisk however shows some warnings, but I suspect this is a bug in busybox's fdisk (I might be wrong).
Here's what I get:
------------ paste -------------- # fdisk /dev/discs/disc0/disc /dev/ide/host0/bus0/target0/lun0: p1 p2
Command (m for help): p
Disk /dev/discs/disc0/disc: 128 MB, 128450560 bytes 8 heads, 32 sectors/track, 980 cylinders Units = cylinders of 256 * 512 = 131072 bytes
Device Boot Start End Blocks Id System /dev/discs/disc0/part1 1 77 9840 c Win95 FAT32 (LBA) /dev/discs/disc0/part2 78 980 115584 83 Linux /dev/discs/disc0/part3 1 1 0 0 Empty Partition 3 has different physical/logical beginnings (non-Linux?): phys=(0, 0, 0) logical=(0, 0, 1) Partition 3 has different physical/logical endings: phys=(0, 0, 0) logical=(16777215, 7, 32) Partition 3 does not end on cylinder boundary.
Command (m for help): --------- end of paste -----------
First partition is VFAT (primary), second is Linux(primary). I have not tested extended partitions since I did not find any reason for having them - currently.
However U-Boot ide part shows two partitions:
------------ paste -------------- WEP> ide reset
Reset IDE: SanDisk SDP 5/3 0.6 Fixed Disk Card IDE interface [silicon] [unique] [single] [sleep] [standby] [idle] [low power] Bus 0: OK Device 0: Model: SanDisk SDCFB-128 Firm: Vdg 1.25 Ser#: 1181701L00X20512 Type: Removable Hard Disk Capacity: 122.5 MB = 0.1 GB (250880 x 512) WEP> ide part
Partition Map for IDE device 0 -- Partition Type: DOS
Partition Start Sector Num Sectors Type 1 32 19680 c 2 19712 231168 83 WEP> --------- end of paste -----------
I have not investigated which one is wrong/correct because I can access primary - VFAT - partition on both systems and also on a PC. After I finish VFAT write support in U-Boot, I might go and see which "fdisk" is wrong or correct.
So this morning I started going through and figuring exactly what that meant. It appears that the first line shows up if the boot sector of the compact flash card has a DOS boot signature that is what you get. So apparently some idiot (probably me) at some point in time formated that CF card in a DOS machine with the /s option and it happily copied a DOS boot record into the beginning of the card. Boy do I feel stupid. So I popped the CF into a linux machine and typed "cat /dev/zero > /dev/hdg" re-ran fdisk to create my partitions, and now u-boot happily reports:
DbAu1x00 # ide part
Partition Map for IDE device 0 -- Partition Type: DOS
Partition Start Sector Num Sectors Type 1 32 4576 83 2 4608 4608 83 3 9216 492544 83 DbAu1x00 #
Which is what I wanted. I have been using #if defined(CONFIG_AU1X00) around my code mods, but it sounds like your changes and my changes are about the same in the end, so perhaps we should come up with another define for ENDIAN_CHALLENGED or something.
I think we should stick to defines that are already used. The endian issues should be wraped with __LITTLE_ENDIAN or __BIG_ENDIAN, specific board issues with CONFIG_AU1X00 or CONFIG_WEPEP250 in my case. I have already submitted the patch for PCMCIA on PXA and that patch fixes the issues with Firmware Revision, Model Number and lba_capacity wrapped with endian #ifdef.
When Wolfgang gets some time for U-Boot and examines the patch and hopefully applies it, you will be able to distinguish between "common" patch and your board specific one.
It helped a lot. Thanks. I guess the moral of the story is when you are having problems, make sure there isn't a short between the keyboard and the chair first. :)
Keep up the good work!
Dear Leon,
in message Pine.CYG.4.55.0311280939530.3976@sniper.ultra.si you wrote:
Here's what I get:
Device Boot Start End Blocks Id System/dev/discs/disc0/part1 1 77 9840 c Win95 FAT32 (LBA) /dev/discs/disc0/part2 78 980 115584 83 Linux /dev/discs/disc0/part3 1 1 0 0 Empty Partition 3 has different physical/logical beginnings (non-Linux?): phys=(0, 0, 0) logical=(0, 0, 1) Partition 3 has different physical/logical endings: phys=(0, 0, 0) logical=(16777215, 7, 32) Partition 3 does not end on cylinder boundary.
"Partition 3" seems pretty bogus to me: a partition that occupies 0 blocks is - um - nonexistent to me.
However U-Boot ide part shows two partitions:
...
Partition Map for IDE device 0 -- Partition Type: DOS
Partition Start Sector Num Sectors Type 1 32 19680 c 2 19712 231168 83
Looks ok to me, and fits exactly to the data above.
I have not investigated which one is wrong/correct because I can access
Why do you think anything is wrong?
primary - VFAT - partition on both systems and also on a PC. After I finish VFAT write support in U-Boot, I might go and see which "fdisk" is wrong or correct.
Keep your eyes open while you're going. It seems that there are some serious hidden bugs (writing through rogue pointers) in the VFAT code.
Best regards,
Wolfgang Denk
Hi Wolfgang,
On Fri, 28 Nov 2003, Wolfgang Denk wrote:
Here's what I get:
Device Boot Start End Blocks Id System/dev/discs/disc0/part1 1 77 9840 c Win95 FAT32 (LBA) /dev/discs/disc0/part2 78 980 115584 83 Linux /dev/discs/disc0/part3 1 1 0 0 Empty Partition 3 has different physical/logical beginnings (non-Linux?): phys=(0, 0, 0) logical=(0, 0, 1) Partition 3 has different physical/logical endings: phys=(0, 0, 0) logical=(16777215, 7, 32) Partition 3 does not end on cylinder boundary.
"Partition 3" seems pretty bogus to me: a partition that occupies 0 blocks is - um - nonexistent to me.
I used busybox's fdisk and have created two partitions. After "p", it showed me three. That's why I suspect busybox's fdisk port problem. I assume it's the busybox's fdisk "p" command problem.
primary - VFAT - partition on both systems and also on a PC. After I finish VFAT write support in U-Boot, I might go and see which "fdisk" is wrong or correct.
Keep your eyes open while you're going. It seems that there are some serious hidden bugs (writing through rogue pointers) in the VFAT code.
Thanks for the notice. Does anyone recommend any document about how data is physicaly stored on the hard drive ? I came across documents describing FAT16/FAT32 but not how an actual file (the data) is stored in clusters. Is a file data always spread troughout clusters continuosly or can a file be spread around multiple group of clusters, say one part of file starting at 1 to 50 and the other part continuing at 100 to 120 ? Note that I am very new to filesystem internals but I am going through Linux fat/vfat module's code in order to understand what's happening.
Thanks.
In message Pine.CYG.4.55.0311281315120.3560@sniper.ultra.si you wrote:
Thanks for the notice. Does anyone recommend any document about how data is physicaly stored on the hard drive ? I came across documents describing
The Linux filesystem code? ;-)
FAT16/FAT32 but not how an actual file (the data) is stored in clusters. Is a file data always spread troughout clusters continuosly or can a file
No, or your disks would be full very soon.
be spread around multiple group of clusters, say one part of file starting at 1 to 50 and the other part continuing at 100 to 120 ? Note that I am very new
It can be spread over many different clusters and groups of clusters - that's what they call "fragmentation", and for waht DOS filesystems recommend to use defragmentation tools to speed up file access.
Best regards,
Wolfgang Denk
participants (3)
-
Ed Okerson -
Leon KUKOVEC -
Wolfgang Denk