[U-Boot-Users] CramFS endianness
Hi everybody,
I recently had a problem with cramfs endianness. cramfs tools generates a host-endian file system, which was incompatible with my big-endian MPC8260 system. I patched mkcramfs to reverse the bit order, and was able to read the cramfs image from Linux.
However, even though Linux doesn't do any byte swapping, U-Boot assumes little-endian byte order and swap bytes on big-endian architectures. Why is that ? U-Boot and Linux are clearly not compatible on big-endian architectures, and I'd like to fix the problem in the right place.
Laurent Pinchart
In message 200607121531.03551.laurent.pinchart@tbox.biz you wrote:
However, even though Linux doesn't do any byte swapping, U-Boot assumes little-endian byte order and swap bytes on big-endian architectures. Why is that ? U-Boot and Linux are clearly not compatible on big-endian architectures, and I'd like to fix the problem in the right place.
Can you please point out which specific parts of the code you are talking of, and which specific prolems you see with this? AFAIK cramfs has been working fine on PPC systems for a long time. [I have to admit that I didn't test it recently, but I am not aware of changes in that area either.]
Best regards,
Wolfgang Denk
On Wednesday 12 July 2006 15:54, Wolfgang Denk wrote:
In message 200607121531.03551.laurent.pinchart@tbox.biz you wrote:
However, even though Linux doesn't do any byte swapping, U-Boot assumes little-endian byte order and swap bytes on big-endian architectures. Why is that ? U-Boot and Linux are clearly not compatible on big-endian architectures, and I'd like to fix the problem in the right place.
Can you please point out which specific parts of the code you are talking of, and which specific prolems you see with this? AFAIK cramfs has been working fine on PPC systems for a long time. [I have to admit that I didn't test it recently, but I am not aware of changes in that area either.]
U-Boot defines the following set of macros in include/cramfs/cramfs_fs.h:
CRAMFS_16(x) CRAMFS_24(x) CRAMFS_32(x) CRAMFS_GET_NAMELEN(x) CRAMFS_GET_OFFSET(x) CRAMFS_SET_OFFSET(x,y) CRAMFS_SET_NAMELEN(x,y)
The actual definitions depend on the target byte order: the CRAMFS_{16,24,32} macros are no-op on little-endian systems, and swap bytes on big-endian systems. This seems to imply that the cramfs image is in little-endian.
The mkcramfs tool generates a host-endian image by default (it doesn't perform any byte swapping). If the host is a little-endian machine, the image will be little-endian and will be read properly by U-Boot. This won't be true if the host is a big-endian machine though.
The Linux kernel doesn't swap bytes when accessing cramfs images. For little-endian images on little-endian targets, no problem there. On big-endian targets, mounting the image fails.
There's a patch available on the cramfs sourceforge page which adds a -r option to mkcramfs to reverse the byte order. With the -r option, a big-endian image can be created on a little-endian host, and the image can be accessed by Linux on big-endian target. This breaks U-Boot, as it swaps bytes on big-endian targets.
Hope this helps to understand the problem.
Best regards,
Laurent Pinchart
In message 200607121608.26551.laurent.pinchart@tbox.biz you wrote:
Can you please point out which specific parts of the code you are talking of, and which specific prolems you see with this? AFAIK cramfs has been working fine on PPC systems for a long time. [I have to admit that I didn't test it recently, but I am not aware of changes in that area either.]
U-Boot defines the following set of macros in include/cramfs/cramfs_fs.h:
I do know this code.
The actual definitions depend on the target byte order: the CRAMFS_{16,24,32} macros are no-op on little-endian systems, and swap bytes on big-endian systems. This seems to imply that the cramfs image is in little-endian.
Did you try it out? What was the specific problems you saw?
Hope this helps to understand the problem.
No, it doesn't as you fail to explain what the problem is.
Best regards,
Wolfgang Denk
On Wednesday 12 July 2006 16:47, Wolfgang Denk wrote:
In message 200607121608.26551.laurent.pinchart@tbox.biz you wrote:
Can you please point out which specific parts of the code you are talking of, and which specific prolems you see with this? AFAIK cramfs has been working fine on PPC systems for a long time. [I have to admit that I didn't test it recently, but I am not aware of changes in that area either.]
U-Boot defines the following set of macros in include/cramfs/cramfs_fs.h:
I do know this code.
The actual definitions depend on the target byte order: the CRAMFS_{16,24,32} macros are no-op on little-endian systems, and swap bytes on big-endian systems. This seems to imply that the cramfs image is in little-endian.
Did you try it out? What was the specific problems you saw?
I tried it out. U-Boot is unable to access an image in big-endian format on a big-endian target (MPC8260). Here is what happens when I try to load a file from the cramfs partition:
=> fsload 00200000 /lib/firmware/fpga/fpga.bit ### JFFS2 loading '/lib/firmware/fpga/fpga.bit' to 0x200000 Scanning JFFS2 FS: done. find_inode failed for name=lib load: Failed to find inode ### JFFS2 LOAD ERROR<0> for /lib/firmware/fpga/fpga.bit!
And here is the output of the same command when removing the byte-swapping macros, all other options being the same:
=> fsload 00200000 /lib/firmware/fpga/fpga.bit ### CRAMFS loading '/lib/firmware/fpga/fpga.bit' to 0x200000 ### CRAMFS load complete: 169287 bytes loaded to 0x200000
The cramfs image is in big-endian format (such an image can be generated on a big-endian host or using a patched mkcramfs on a little-endian host).
Hope this helps to understand the problem.
No, it doesn't as you fail to explain what the problem is.
Please don't cut the part where I explain what the problem is:
"There's a patch available on the cramfs sourceforge page which adds a -r option to mkcramfs to reverse the byte order. With the -r option, a big-endian image can be created on a little-endian host, and the image can be accessed by Linux on big-endian target. This breaks U-Boot, as it swaps bytes on big-endian targets."
To access the cramfs from Linux, the image has to be in host-endian format:
"All data is currently in host-endian format; neither mkcramfs nor the kernel ever do swabbing." (fs/cramfs/README in the Linux tree)
U-Boot fails to access the host-endian cramfs image on big-endian targets, as I showed above. The Linux kernel is able to access the image properly. The situation is reversed for little-endian images on big-endian targets, where U-Boot is able to access the image but the Linux kernel isn't.
Best regards, and thank you for your help.
Laurent Pinchart
Hi Wolfgang,
my previous mail concerning the CramFS endianness problem probably got lost somewhere in a mailbox. Could you have a look at it when time permits ?
Thank you in advance for your help.
Best regards,
Laurent Pinchart
Laurent Pinchart <laurent.pinchart <at> tbox.biz> writes:
I recently had a problem with cramfs endianness. cramfs tools generates a host-endian file system, which was incompatible with my big-endian MPC8260 system. I patched mkcramfs to reverse the bit order, and was able to read the cramfs image from Linux.
However, even though Linux doesn't do any byte swapping, U-Boot assumes little-endian byte order and swap bytes on big-endian architectures. Why is that ? U-Boot and Linux are clearly not compatible on big-endian architectures, and I'd like to fix the problem in the right place.
Searching this list and the U-Boot repository, I found that this problem was already acknowledged by Wolfgang once back in 2004: http://article.gmane.org/gmane.comp.boot-loaders.u-boot/12197
Soon after that, there was this fix ("Fix endianess problem in cramfs code"): http://www.denx.de/cgi-bin/gitweb.cgi?p=u-boot.git;a=commitdiff;h=36c728774e...
But then, this fix got reverted by the following change in 2005: http://www.denx.de/cgi-bin/gitweb.cgi?p=u-boot.git;a=commitdiff;h=b66eb5282d...
At last, the issue popped up on the list again in 2006 (the post that I'm replying to): http://article.gmane.org/gmane.comp.boot-loaders.u-boot/22846
Thus, I suggest that the original fix should be re-applied.
Bernhard
In message loom.20080229T161725-122@post.gmane.org you wrote:
Laurent Pinchart <laurent.pinchart <at> tbox.biz> writes:
I recently had a problem with cramfs endianness. cramfs tools generates a host-endian file system, which was incompatible with my big-endian MPC8260 system. I patched mkcramfs to reverse the bit order, and was able to read the cramfs image from Linux.
However, even though Linux doesn't do any byte swapping, U-Boot assumes little-endian byte order and swap bytes on big-endian architectures. Why is that ? U-Boot and Linux are clearly not compatible on big-endian architectures, and I'd like to fix the problem in the right place.
Searching this list and the U-Boot repository, I found that this problem was already acknowledged by Wolfgang once back in 2004: http://article.gmane.org/gmane.comp.boot-loaders.u-boot/12197
Soon after that, there was this fix ("Fix endianess problem in cramfs code"): http://www.denx.de/cgi-bin/gitweb.cgi?p=u-boot.git;a=commitdiff;h=36c728774e...
But then, this fix got reverted by the following change in 2005: http://www.denx.de/cgi-bin/gitweb.cgi?p=u-boot.git;a=commitdiff;h=b66eb5282d...
Argh... I wish we had had git available by then. Obviously the patch checked in was based on older code (without the fix included), and I missed this.
At last, the issue popped up on the list again in 2006 (the post that I'm replying to): http://article.gmane.org/gmane.comp.boot-loaders.u-boot/22846
Thus, I suggest that the original fix should be re-applied.
I agree. Can somebody please submit a proper patch?
Best regards,
Wolfgang Denk
Wolfgang Denk <wd <at> denx.de> writes:
Thus, I suggest that the original fix should be re-applied.
I agree. Can somebody please submit a proper patch?
voila.
Signed-off-by: Bernhard Nemec <bnemec <at> ganssloser.com>
--- include/cramfs/cramfs_fs.h | 22 ---------------------- 1 files changed, 0 insertions(+), 22 deletions(-)
diff --git a/include/cramfs/cramfs_fs.h b/include/cramfs/cramfs_fs.h index 9f1b1d5..e0c14f0 100644 --- a/include/cramfs/cramfs_fs.h +++ b/include/cramfs/cramfs_fs.h @@ -84,7 +84,6 @@ struct cramfs_super { | CRAMFS_FLAG_WRONG_SIGNATURE \ | CRAMFS_FLAG_SHIFTED_ROOT_OFFSET )
-#if __BYTE_ORDER == __LITTLE_ENDIAN #define CRAMFS_16(x) (x) #define CRAMFS_24(x) (x) #define CRAMFS_32(x) (x) @@ -92,27 +91,6 @@ struct cramfs_super { #define CRAMFS_GET_OFFSET(x) ((x)->offset) #define CRAMFS_SET_OFFSET(x,y) ((x)->offset = (y)) #define CRAMFS_SET_NAMELEN(x,y) ((x)->namelen = (y)) -#elif __BYTE_ORDER == __BIG_ENDIAN -#ifdef __KERNEL__ -#define CRAMFS_16(x) swab16(x) -#define CRAMFS_24(x) ((swab32(x)) >> 8) -#define CRAMFS_32(x) swab32(x) -#else /* not __KERNEL__ */ -#define CRAMFS_16(x) bswap_16(x) -#define CRAMFS_24(x) ((bswap_32(x)) >> 8) -#define CRAMFS_32(x) bswap_32(x) -#endif /* not __KERNEL__ */ -#define CRAMFS_GET_NAMELEN(x) (((u8*)(x))[8] & 0x3f) -#define CRAMFS_GET_OFFSET(x) ((CRAMFS_24(((u32*)(x))[2] & 0xffffff) << 2) |\ - ((((u32*)(x))[2] & 0xc0000000) >> 30)) -#define CRAMFS_SET_NAMELEN(x,y) (((u8*)(x))[8] = (((0x3f & (y))) | \ - (0xc0 & ((u8*)(x))[8]))) -#define CRAMFS_SET_OFFSET(x,y) (((u32*)(x))[2] = (((y) & 3) << 30) | \ - CRAMFS_24((((y) & 0x03ffffff) >> 2)) | \ - (((u32)(((u8*)(x))[8] & 0x3f)) << 24)) -#else -#error "__BYTE_ORDER must be __LITTLE_ENDIAN or __BIG_ENDIAN" -#endif
/* Uncompression interfaces to the underlying zlib */ int cramfs_uncompress_block(void *dst, void *src, int srclen); -- 1.5.3.6
In message loom.20080303T115353-490@post.gmane.org you wrote:
I agree. Can somebody please submit a proper patch?
voila.
Thanks, but:
Applying Fix endianess problem in cramfs code (cramfs is always host-endian in Linux) error: patch failed: include/cramfs/cramfs_fs.h:84 error: include/cramfs/cramfs_fs.h: patch does not apply Using index info to reconstruct a base tree... error: patch failed: include/cramfs/cramfs_fs.h:84 error: include/cramfs/cramfs_fs.h: patch does not apply Did you hand edit your patch? It does not apply to blobs recorded in its index. Cannot fall back to three-way merge. Patch failed at 0001.
I had to apply it manually. Please verify that it is OK.
Best regards,
Wolfgang Denk
Wolfgang Denk <wd <at> denx.de> writes:
I had to apply it manually. Please verify that it is OK.
Strange, but yes, the result looks ok to me.
Thanks, Bernhard
participants (3)
-
Bernhard Nemec -
Laurent Pinchart -
Wolfgang Denk