Bug 2246428 - No display on monitor on Server images on some Raspberry Pi variants due to bug in firmware DTB
Summary: No display on monitor on Server images on some Raspberry Pi variants due to b...
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: bcm283x-firmware
Version: 39
Hardware: aarch64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Peter Robinson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker AcceptedFreezeExcepti...
Depends On:
Blocks: F39FinalFreezeException
TreeView+ depends on / blocked
 
Reported: 2023-10-26 18:58 UTC by Adam Williamson
Modified: 2023-12-06 18:05 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)
rpi4 vc4-hdmi patch to work around broken firmware DT (2.02 KB, patch)
2023-10-26 22:24 UTC, Janne Grunau
no flags Details | Diff

Description Adam Williamson 2023-10-26 18:58:49 UTC
This was discovered by Hristo Marinov and confirmed by Paul Whalen in testing of https://bugzilla.redhat.com/show_bug.cgi?id=2241252 and https://bugzilla.redhat.com/show_bug.cgi?id=2244305 . Per pwhalen, it seems there's a reproducible and understandable problem specific to the Server images even after the rollback to uboot-tools 2023.07.

pwhalen says "Its a bug in the FW provided dtb it seems. But it also seems to only affect some revisions"

As I understand it, the Server images (only) use the FW provided dtb because the kernel dtb cannot be read from xfs. Other images use the kernel dtb because they don't use xfs. That's why this bug is specific to Server images.

The bug can be worked around by booting with nomodeset.

Proposing as a Final blocker as a conditional violation of "Release-blocking ARM disk images must boot to the initial-setup utility" - when booting a Server image on an affected Pi variant and expecting to see i-s on an attached monitor.

Comment 1 Adam Williamson 2023-10-26 19:43:46 UTC
Discussed at 2023-10-26 go/no-go meeting acting as a blocker review meeting - https://meetbot-raw.fedoraproject.org/fedora-meeting/2023-10-26/ . Rejected as a blocker on the basis that physical monitor display is less important on Server than e.g. Workstation (many folks use it headless), this only affects some hardware and/or firmware versions, and there is a documentable workaround (using nomodeset).

Comment 2 Janne Grunau 2023-10-26 20:14:31 UTC
This issue reproduces here. HDMI device probing fails with "vc4_hdmi fef00700.hdmi: Could not register PCM component: -517" which is caused by the different device tree bindings used for the HDMI audio dma engine:
Kernel DTB

| dma@7e007000 {
|         compatible = "brcm,bcm2835-dma";
|         reg = <0x7e007000 0xb00>;
|         interrupts = <0x00 0x50 0x04 0x00 0x51 0x04 0x00 0x52 0x04 0x00 0x53 0x04 0x00 0x54 0x04 0x00 0x55 0x04 0x00 0x56 0x04 0x00 0x57 0x04 0x00 0x57 0x04 0x00 0x58 0x04 0x00 0x58 0x04>;
|         interrupt-names = "dma0\0dma1\0dma2\0dma3\0dma4\0dma5\0dma6\0dma7\0dma8\0dma9\0dma10";
|         #dma-cells = <0x01>;
|         brcm,dma-channel-mask = <0x7f5>;
|         phandle = <0x1b>;
| };

firmware DTB used by the server image:
| dma@7e007b00 {
|         compatible = "brcm,bcm2711-dma";;
|         reg = <0x00 0x7e007b00 0x00 0x400>;
|         interrupts = <0x00 0x59 0x04 0x00 0x5a 0x04 0x00 0x5b 0x04 0x00 0x5c 0x04>;
|         brcm,dma-channel-mask = <0x3000>;
|         interrupt-names = "dma11\0dma12\0dma13\0dma14"
|         #dma-cells = <0x01>;
|         phandle = <0x2e>;
| };

This could be worked around with a couple of line kernel patch adding a check to vc4_hdmi_audio_init() in drivers/gpu/drm/vc4/vc4_hdmi.c
This would "break" HDMI audio. Returning 0 after devm_snd_dmaengine_pcm_register() returns an error should work as well. As far as I can tell this would only break HDMI audio.

Comment 3 Adam Williamson 2023-10-26 20:23:04 UTC
Proposing as a Final FE, since we've slipped it would be good to fix this if we can.

Comment 4 Janne Grunau 2023-10-26 22:24:48 UTC
Created attachment 1995708 [details]
rpi4 vc4-hdmi patch to work around broken firmware DT

Attached patch allows vc40-hdmi to probe successfully with the broken firmware DT as seen on fedora-server. The trivial patch I had in mind unfortunately doesn't  work since the returned error is -EPROBE_DEFER. It's impossible to decide whether this is a temporary error (on a system with a DT matching the kernel's expectations and temporary unavailable module) or a permanent error like in the case with the broken firmware DT.
I can send the patch upstream tomorrow (Friday CEST) but I have doubts whether it will be accepted.

Comment 5 Adam Williamson 2023-10-30 23:39:32 UTC
+5 FE in https://pagure.io/fedora-qa/blocker-review/issue/1424 , marking accepted.

Comment 6 Brandon Nielsen 2023-11-01 14:16:05 UTC
Piling in here mostly so I get notifications for the right bug.

I am also seeing this, Raspberry Pi 4B Rev 1.1, BIOS 2023.07 07/01/2023.

Comment 7 Peter Robinson 2023-11-01 18:59:41 UTC
(In reply to Janne Grunau from comment #4)
> Created attachment 1995708 [details]
> rpi4 vc4-hdmi patch to work around broken firmware DT

Do we know what revs of upstream firmware this is broken for? Has it been fixed? Do you have details of an upstream issue/PR?

Comment 8 Janne Grunau 2023-11-01 21:24:55 UTC
(In reply to Peter Robinson from comment #7)
> (In reply to Janne Grunau from comment #4)
> > Created attachment 1995708 [details]
> > rpi4 vc4-hdmi patch to work around broken firmware DT
> 
> Do we know what revs of upstream firmware this is broken for? Has it been
> fixed? Do you have details of an upstream issue/PR?

This looks like a deliberate change in the Raspberry Pi kernel tree

https://github.com/raspberrypi/linux/commit/0491a0aecb999b1a013ad4a6ad3816c535ac6e73

and breaks compatibility with the upstream kernel.

Comment 9 Jonathan Watt 2023-11-05 19:01:31 UTC
(In reply to Adam Williamson from comment #0)
> The bug can be worked around by booting with nomodeset.

I've tried interupting the bootloader to edit the kernel parameters but there's no cursor and the screen isn't redrawing, so I can't see what I'm doing.

I also tried editing /root/anaconda-ks.cfg to add `--append=nomodeset` but that doesn't seem to work.

Any pointers on how else to apply the workaround would be welcome. I'm not currently familiar enough with the dracut, grub, etc. interactions to know how to do this for SBC images.

Comment 10 Lukas Brabec 2023-11-10 16:29:21 UTC
(In reply to Jonathan Watt from comment #9)
> (In reply to Adam Williamson from comment #0)
> > The bug can be worked around by booting with nomodeset.
> 
> I've tried interupting the bootloader to edit the kernel parameters but
> there's no cursor and the screen isn't redrawing, so I can't see what I'm
> doing.

Yes, that it is a huge annoyance. I just write single character, check where it appeared, delete it... and rinse and repeat until I'm on the right position.

> Any pointers on how else to apply the workaround would be welcome. I'm not
> currently familiar enough with the dracut, grub, etc. interactions to know
> how to do this for SBC images.

There is a common issue in Fedora Discussion: https://discussion.fedoraproject.org/t/some-raspberry-pi-boards-have-no-display-output-on-fedora-server/95064
You have three options:
- add nomodeset on every boot
- use arm-image-installer with argument --args "nomodeset" (note: it is possible that new kernel after update won't have nomodeset on the kernel line)
- you can add it permanently to grub config (see the link above)


Note You need to log in before you can comment on or make changes to this bug.