CM5 without wifi hangs on reboot #6647
Comments
|
I did some further research and noticed that What could be the reason that It also seems that if the After I performed a firmware update to A downgrade to Handover to OS is about 8-9 seconds, so I don't think that the difference is resulted by something like this. So it seems to me that this might be a firmware related issue or at least it has some influence. Did also some testing on a CM4 without wifi and there |
|
Hi Nicolai, we'll look into disabling SDIO2 from the firmware for non-WiFi-enabled parts. |
|
Thanks Phil for the update |
|
pieeprom_cm5nowifi.zip |
|
Give me some minutes and I will test it, I have modules at hand ... |
|
I can confirm, Reboot is also working without hang / delay. |
|
Great. We'll get that merged, then into a release at some point. |
|
Thanks. In the meantime I will do some thinking and come up with some tooling for our end of line tests, so we can update the modules in place. |
|
Just a note for others which might need to work around the issue that the first reboot after firmware update still hangs (which is fine as we're still running the old firmware): |
|
It's odd that a non-WiFi CM5 is rebooting without issue for me. I've tried rebooting before the The power mode difference is just an indicator of whether or not the kernel has given up on there being something on that SDIO bus - it turns off the power when it loses hope. |
|
Yes, at some point the device is rebooting (after the driver gives up on |
|
The patch to disable sdio2 has been merged, so future EEPROM builds will include it. I do wonder though if the kernel retry mechanism can be adjusted to not take quite so long. |
That was also I was initially thinking when I raised this issue. Haven't had the time to dig deeper what the differences for bcm2711 and 2712 are here, but from a first look they share at least the same driver for |
* recovery: Walk partitions to delete recovery.bin Previously, recovery.bin would fail to delete itself if the bootrom loaded recovery.bin where there are multiple FAT partitions and the first partition does not contain recovery.bin Update the rename code to walk the partition table to find the recovery.bin file to delete. * pi5: Add config filter for simple boot variable expressions (experimental) Add support for a new bootloader/config.txt conditional filter which tests the partition, boot_count and boot_arg1 variables. Syntax (no spaces): ARG boot_arg1, boot_count or partition (EEPROM config stage only) [ARG=VALUE] selected if (ARG == VALUE) [ARG&MASK] selected if ((ARG & VALUE) != 0)) [ARG&MASK=VALUE] selected if ((ARG & MASK) == VALUE) [ARG<VALUE] selected if (ARG < VALUE) [ARG>VALUE] selected if (ARG > VALUE) where VALUE and MASK are unsigned integer constants and ARG corresponds to the value in the reset register before the config file is parsed. * pi5: Add a boot-count bootloader variable (experimental) Store the boot-count in a reset register and increment just before the boot-order state-machine. The boot-count variable is visible via device-tree /proc/device-tree/chosen/bootloader/count and can be read/set via vcmailbox GET: sudo vcmailbox 0x0003008d 4 4 0 SET to N: sudo vcmailbox 0x0003808d 4 4 N * pi5: Add user-defined reboot argument (boot_arg1) (experimental) Add support for a user-defined boot parameter stored in a reset-safe scratch register on BCM2712. This is visible via device-tree at /proc/device-tree/chosen/bootloader/arg1 and via vcmailboxes GET arg1: sudo vcmailbox 0x0003008c 8 8 1 0 SET arg1 to 42: sudo vcmailbox 0x0003808c 8 8 1 42 or via config.txt set_reboot_arg1=42 The variable is NOT cleared automatically and will persist until a power-on-reset. * Enable overriding of high partition numbers Previously, the PARTITION=N bootloader config setting would only be used at power on reset or if the partition number passed to reboot was zero. Change the behaviour so that the bootloader config PARTITION property can override the reboot partition number if the reboot parameter is > 31. * Disable WiFi PMIC output on CM5 modules without WiFi Disable the 3.7V WiFi power supply on CM5 modules which do not have a WiFi module fitted. This fixes some stability issues where a CM5 would shutdown due to a spurious over-voltage condition on the non-connected WiFi power supply. * Add memory barrier to the mbox handler Firmware issue 1944 reports receiving kernel warnings about firmware requests where the status return code is 0. This should not be possible, as handle_mbox_property always sets the top bit of the return code, with the bottom bit indicating success or failure. If the firmware had died, the firmware driver would report a timeout due to the lack of a mailbox interrupt, and that isn't happening. See: raspberrypi/firmware#1944 * support dts files with size-cells of 2 DTS files with a top-level #size-cells of 2 make a lot of sense for systems with a lot of RAM, but the firmware is currently inconsistent in its support for that. Fix up the other cases to honor #size-cells and #address-cells. * Disable SDIO2 for CM5s without WiFi It has been observed that CM5s without WiFi hang on reboot. To prevent that, disable the sdio2 node on those devices. See: raspberrypi/linux#6647 * arm_dt: Use dtoverlay_enable_node Convert the open-coded DT node status changes to use the new dtoverlay method dtoverlay_enable_node. * dtoverlay: Add dtoverlay_enable_node Add a helper function for setting the status of a node.
* recovery: Walk partitions to delete recovery.bin Previously, recovery.bin would fail to delete itself if the bootrom loaded recovery.bin where there are multiple FAT partitions and the first partition does not contain recovery.bin Update the rename code to walk the partition table to find the recovery.bin file to delete. * pi5: Add config filter for simple boot variable expressions (experimental) Add support for a new bootloader/config.txt conditional filter which tests the partition, boot_count and boot_arg1 variables. Syntax (no spaces): ARG boot_arg1, boot_count or partition (EEPROM config stage only) [ARG=VALUE] selected if (ARG == VALUE) [ARG&MASK] selected if ((ARG & VALUE) != 0)) [ARG&MASK=VALUE] selected if ((ARG & MASK) == VALUE) [ARG<VALUE] selected if (ARG < VALUE) [ARG>VALUE] selected if (ARG > VALUE) where VALUE and MASK are unsigned integer constants and ARG corresponds to the value in the reset register before the config file is parsed. * pi5: Add a boot-count bootloader variable (experimental) Store the boot-count in a reset register and increment just before the boot-order state-machine. The boot-count variable is visible via device-tree /proc/device-tree/chosen/bootloader/count and can be read/set via vcmailbox GET: sudo vcmailbox 0x0003008d 4 4 0 SET to N: sudo vcmailbox 0x0003808d 4 4 N * pi5: Add user-defined reboot argument (boot_arg1) (experimental) Add support for a user-defined boot parameter stored in a reset-safe scratch register on BCM2712. This is visible via device-tree at /proc/device-tree/chosen/bootloader/arg1 and via vcmailboxes GET arg1: sudo vcmailbox 0x0003008c 8 8 1 0 SET arg1 to 42: sudo vcmailbox 0x0003808c 8 8 1 42 or via config.txt set_reboot_arg1=42 The variable is NOT cleared automatically and will persist until a power-on-reset. * Enable overriding of high partition numbers Previously, the PARTITION=N bootloader config setting would only be used at power on reset or if the partition number passed to reboot was zero. Change the behaviour so that the bootloader config PARTITION property can override the reboot partition number if the reboot parameter is > 31. * Disable WiFi PMIC output on CM5 modules without WiFi Disable the 3.7V WiFi power supply on CM5 modules which do not have a WiFi module fitted. This fixes some stability issues where a CM5 would shutdown due to a spurious over-voltage condition on the non-connected WiFi power supply. * Add memory barrier to the mbox handler Firmware issue 1944 reports receiving kernel warnings about firmware requests where the status return code is 0. This should not be possible, as handle_mbox_property always sets the top bit of the return code, with the bottom bit indicating success or failure. If the firmware had died, the firmware driver would report a timeout due to the lack of a mailbox interrupt, and that isn't happening. See: raspberrypi/firmware#1944 * support dts files with size-cells of 2 DTS files with a top-level #size-cells of 2 make a lot of sense for systems with a lot of RAM, but the firmware is currently inconsistent in its support for that. Fix up the other cases to honor #size-cells and #address-cells. * Disable SDIO2 for CM5s without WiFi It has been observed that CM5s without WiFi hang on reboot. To prevent that, disable the sdio2 node on those devices. See: raspberrypi/linux#6647 * arm_dt: Use dtoverlay_enable_node Convert the open-coded DT node status changes to use the new dtoverlay method dtoverlay_enable_node. * dtoverlay: Add dtoverlay_enable_node Add a helper function for setting the status of a node.
|
The rescan code tries 3 different card types at 4 different clock frequencies. All of those tests involve timeouts of specific durations, so they shouldn't simply be shortened. The other approach would be to make the scanning interruptable at some granularity - at least between frequencies. There may be a way to mark that the interface is being shut down - perhaps using the |
Describe the bug
We stumbled over an issue where all CM5 without wifi seem to hang when rebooted. After some waiting the reboot is completed whereas all CM5 with wifi show no such error (same base boards, same software). As is some care cases the reboot even worked on CM5 without wifi I started to debug it further.
When reboot hangs:
When reboot works immediately:
The culprit seems to be (always present when the reboot works):
So it looks like there might be an issue with the unused sdio/
mmc1which is not used on the wifi less variant of CM5. In order to verify my suspicion I've created a simple overlay which deactivatessdio 2completely:With this the reboot works reliable in all tests so far. Even though it kinda works with a custom overlay it looks wrong. It also is not a reliable solution for production as during first boot only the cm5io dt loaded by the firmware is present and a subsequent reboot will fail very often.
Same works on CM4 with / without wifi (different overlay though, but should be irrelevant as it also happens with pure CM dt).
Any ideas / insights on this?
Steps to reproduce the behaviour
sudo rebootDevice (s)
Raspberry Pi CM5
System
EEPROM release: 1727096576
Kernel:
6.6.74+rpt-rpi-v8Logs
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: