Hi everyone,
Good news! I disabled compression on kernel build and the resulting boot image starts booting Linux.
This is great but it does not solve the decompression issue. Thanks to Mattijs for pointing me in the compression direction which made me further investigate compression options.
So with compression disabled, let's move on to the next issue: I picked the Quartz64 DTS commits, built and booted it, however there seems to still be an issue: Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x412fd050] [ 0.000000] Linux version 4.19.154-xos (nobody@android-build) (Android (6877366 based on r383902b1) clang version 11.0.2 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), GNU ld (binutils-2.27-bd24d23f) 2.27.0.20170315) #1 SMP PREEMPT Wed Nov 8 00:54:49 CET 2023 [ 0.000000] Machine model: Pine64 RK3566 Quartz64-A Board ...omitted for brevity... [ 0.000007] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns [ 0.001828] Insufficient stack space to handle exception! [ 0.001835] ESR: 0x81054040 -- IABT (lower EL) [ 0.002766] FAR: 0xc08eea4d24faf096 [ 0.003098] Task stack: [0xffffff8009cc1000..0xffffff8009cc5000] [ 0.003697] IRQ stack: [0xffffff8008000000..0xffffff8008004000] [ 0.004299] Overflow stack: [0xffffffc1ff713080..0xffffffc1ff714080] [ 0.004907] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.154-xos #1 [ 0.005516] Hardware name: Pine64 RK3566 Quartz64-A Board (DT) [ 0.006073] pstate: 20400009 (nzCv daif +PAN -UAO) [ 0.006544] pc : start_kernel+0x26c/0x3d0 [ 0.006931] lr : start_kernel+0x25c/0x3d0 ...omitted for brevity... [ 0.098560] Kernel panic - not syncing: kernel stack overflow [ 0.099108] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.19.154-xos #1 [ 0.099717] Hardware name: Pine64 RK3566 Quartz64-A Board (DT) [ 0.100270] Call trace: [ 0.100515] dump_backtrace+0x0/0x158 [ 0.100871] show_stack+0x14/0x1c [ 0.101195] dump_stack+0xb8/0xf0 [ 0.101519] panic+0x12c/0x2a4 [ 0.101815] panic+0x0/0x2a4 [ 0.102095] handle_bad_stack+0x118/0x12c [ 0.102479] __bad_stack+0xc0/0xc4 [ 0.102809] start_kernel+0x26c/0x3d0
Full log: https://termbin.com/yzx6k
Do I need to do anything to reserve stack space for the kernel?
Thanks a lot!
Cheers, Simao
On Tuesday, November 07, 2023 21:42 CET, Simão Gomes Viana devel@superboring.dev wrote: Hi Simon, Hi everyone from the mailing list,
I'm following up to the email you sent today about a particular problem I'm facing trying to boot AOSP 11 (built from source) using U-Boot. I have extended my original email to include additional information and added the mailing list to CC as requested.
In essence, it boils down to this: ## Booting Android Image at 0x00c00800 ... Kernel load addr 0x00c01000 size 15184 KiB Kernel command line: console=ttyS2,1500000 earlycon=uart8250,mmio32,0xfe660000 androidboot.selinux=permissive service_locator.enable=1 kpti=0 loop.max_part=7 firmware_class.path=/vendor/etc/firmware buildvariant=user RAM disk load addr 0x01ad5000 size 751 KiB ## Flattened Device Tree blob at 00600000 Booting using the fdt blob at 0x600000 Working FDT set to 600000 Uncompressing Kernel Image lz4 compressed: uncompress error -71 Must RESET board to recover
I have asked the same question on SO with more details.
The kernel is built in-line with the AOSP sources, just like LineageOS does it. It builds the kernel using make as usual, then takes the Image.lz4 and runs mkbootimg with that.
I found this in scripts/Makefile.lib: quiet_cmd_lz4 = LZ4 $@ cmd_lz4 = (cat $(filter-out FORCE,$^) | \ lz4 -l -12 --favor-decSpeed stdin stdout && \ $(call size_append, $(filter-out FORCE,$^))) > $@ || \ (rm -f $@ ; false)
It is using the -l parameter which is required for LZ4 compression of the kernel image.
This is what is present in my BoardConfig:
# Kernel BOARD_KERNEL_CMDLINE := \ console=ttyS2,1500000 \ earlycon=uart8250,mmio32,0xfe660000 \ androidboot.selinux=permissive \ service_locator.enable=1 \ kpti=0 \ loop.max_part=7 \ firmware_class.path=/vendor/etc/firmware
BOARD_KERNEL_BASE := 0x10000000 BOARD_KERNEL_PAGESIZE := 2048 BOARD_KERNEL_IMAGE_NAME := Image.lz4 TARGET_KERNEL_CLANG_COMPILE := true #TARGET_KERNEL_CLANG_VERSION := latest TARGET_KERNEL_SOURCE := kernel/rockchip/rk356x TARGET_KERNEL_CONFIG := xos_rk356x_defconfig TARGET_PREBUILT_RESOURCE ?= $(TARGET_KERNEL_SOURCE)/resource.img TARGET_BOOTLOADER_IS_2ND := true
This is what the file utility tells me: out/target/product/rk3566_rgo/boot.img: Android bootimg, kernel (0x10008000), ramdisk (0x11000000), second stage (0x10f00000), page size: 2048, cmdline (console=ttyS2,1500000 earlycon=uart8250,mmio32,0xfe660000 androidboot.selinux=permissive service_locator.enable=1 kpti=0 loop.m)
unpack_bootimg shows: boot magic: ANDROID! kernel_size: 15547433 kernel load address: 0x10008000 ramdisk size: 768194 ramdisk load address: 0x11000000 second bootloader size: 200192 second bootloader load address: 0x10f00000 kernel tags load address: 0x10000100 page size: 2048 os version: 11.0.0 os patch level: 2021-09 boot image header version: 2 product name: command line args: console=ttyS2,1500000 earlycon=uart8250,mmio32,0xfe660000 androidboot.selinux=permissive service_locator.enable=1 kpti=0 loop.max_part=7 firmware_class.path=/vendor/etc/firmware buildvariant=user additional command line args: recovery dtbo size: 0 recovery dtbo offset: 0x0000000000000000 boot header size: 1660 dtb size: 11792057 dtb address: 0x0000000011f00000
This is what the working boot.img built by the manufacturer yields: boot.img: Android bootimg, kernel (0x10008000), ramdisk (0x11000000), second stage (0x10f00000), page size: 2048, cmdline (console=ttyFIQ0 androidboot.baseband=N/A androidboot.wificountrycode=CN androidboot.veritymode=enforcing androidboot.hardware=r)
unpack_bootimg shows: boot magic: ANDROID! kernel_size: 31041552 kernel load address: 0x10008000 ramdisk size: 904155 ramdisk load address: 0x11000000 second bootloader size: 200192 second bootloader load address: 0x10f00000 kernel tags load address: 0x10000100 page size: 2048 os version: 11.0.0 os patch level: 2021-04 boot image header version: 2 product name: command line args: console=ttyFIQ0 androidboot.baseband=N/A androidboot.wificountrycode=CN androidboot.veritymode=enforcing androidboot.hardware=rk30board androidboot.console=ttyFIQ0 androidboot.verifiedbootstate=orange firmware_class.path=/vendor/etc/firmware init=/init rootwait ro loop.max_part=7 androidboot.selinux=permissive buildvariant=userdebug additional command line args: recovery dtbo size: 0 recovery dtbo offset: 0x0000000000000000 boot header size: 1660 dtb size: 111487 dtb address: 0x0000000011f00000
Given that the kernel image is actually smaller than the manufacturer one, there can't be an overlapping issue. The only real difference is the dtb size which is a lot bigger on the non-functional boot image (I don't know why). However, U-Boot is able to list all the DTBs.
The kernel source I'm using is the one provided in the BSP here (the 79G file): https://wiki.pine64.org/wiki/Quartz64_Software_Releases#Android_SDK
If I build the kernel with CONFIG_COMPRESS_GZ instead of CONFIG_COMPRESS_LZ4, I also get an error: ## Booting Android Image at 0x00c00800 ... Kernel load addr 0x00c01000 size 13176 KiB Kernel command line: console=ttyS2,1500000 earlycon=uart8250,mmio32,0xfe660000 androidboot.selinux=permissive service_locator.enable=1 kpti=0 loop.max_part=7 firmware_class.path=/vendor/etc/firmware buildvariant=userdebug RAM disk load addr 0x018df000 size 890 KiB ## Flattened Device Tree blob at 00600000 Booting using the fdt blob at 0x600000 Working FDT set to 600000 Uncompressing Kernel Image Error: inflate() returned -5 gzip compressed: uncompress error -1 Must RESET board to recover
file utility says this about the kernel inside the boot image: kernel: gzip compressed data, max compression, from Unix, original size modulo 2^32 30021648
I am able to decompress it using gzip -d < kernel > kernel_decompressed Now file utility says: kernel_decompressed: Linux kernel ARM64 boot executable Image, little-endian, 4K pages
The manufacturer kernel is not compressed, so it just tries to boot without decompressing:
## Booting Android Image at 0x00c00800 ... Kernel load addr 0x00c01000 size 30315 KiB Kernel command line: console=ttyFIQ0 androidboot.baseband=N/A androidboot.wificountrycode=CN androidboot.veritymode=enforcing androidboot.hardware=rk30board androidboot.console=ttyFIQ0 androidboot.verifiedbootstate=orange firmware_class.path=/vendor/etc/firmware init=/init rootwait ro loop.max_part=7 androidboot.selinux=permissive buildvariant=userdebug RAM disk load addr 0x0299c000 size 883 KiB ## Flattened Device Tree blob at 00600000 Booting using the fdt blob at 0x600000 Working FDT set to 600000 XIP Kernel Image Loading Ramdisk to ede39000, end edf15bdb ... OK ERROR: reserving fdt memory region failed (addr=0 size=0 flags=0) ERROR: reserving fdt memory region failed (addr=0 size=0 flags=0) Loading Device Tree to 00000000edd36000, end 00000000ede38fff ... OK Working FDT set to edd36000
Starting kernel ...
But the kernel does not start, the board just resets after a few seconds.
I'm guessing there's something wrong with my bootcommand.
I'd appreciate it if you could point me in the right direction. In case you need more information or want images of what I built, feel free to ask for it.
I have been trying to get this to work for weeks without any meaningful success and am running out of ideas.
Thanks a lot and best regards, Simao