Bug 1573926 - the Kernel shipped with Fedora doesn't boot on chromebook snow (arm) [NEEDINFO]
Summary: the Kernel shipped with Fedora doesn't boot on chromebook snow (arm)
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 27
Hardware: arm
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: ARMTracker
TreeView+ depends on / blocked
 
Reported: 2018-05-02 14:20 UTC by Mauro Carvalho Chehab
Modified: 2018-08-29 15:09 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-29 15:09:06 UTC
Type: Bug
Embargoed:
jforbes: needinfo?


Attachments (Terms of Use)
Config file used to build the custom 4.16.6 kernel with boots with Fedora 27 (124.37 KB, text/plain)
2018-05-02 14:21 UTC, Mauro Carvalho Chehab
no flags Details
Kernel log from custom 4.16.6 (with boots fine with Fedora 27) (40.26 KB, text/plain)
2018-05-02 14:24 UTC, Mauro Carvalho Chehab
no flags Details

Description Mauro Carvalho Chehab 2018-05-02 14:20:04 UTC
Description of problem:

I have a chromebook with Fedora 28 installed on its MMC flash (just after dnf system-upgrade) and Fedora 27 on an USB stick.

Fedora standard Kernel doesn't boot on Samsung Snow neither with Fedora 27 nor with Fedora 28; the Kernel hangs just after cleaning the screen, without printing a single message on display.

This is not an upstream Kernel problem, as I'm able to build an upstream Kernel that boots properly with Fedora 27 (it partially boots on Fedora 28, but there is a systemd crash preventing it to finish booting - I'll report it as a separate bug).

How reproducible: Always.

Steps to Reproduce:

Ensure that boot/extlinux/extlinux.conf have the proper settings:

label Fedora (4.16.5-300.fc28.armv7hl) 28 (Twenty Eight)
	kernel /boot/vmlinuz-4.16.5-300.fc28.armv7hl
        fdt /boot/dtb-4.16.5-300.fc28.armv7hl/exynos5250-snow.dtb
        append ro rootwait console=tty0 console=ttyUSB0,115200n8 root=PARTUUID=1d4adb65-ea43-c641-9c2d-06ad8f38fb5c
        initrd /boot/initramfs-4.16.5-300.fc28.armv7hl.img
#        append ro rootwait console=tty0 console=ttyUSB0,115200n8 root=UUID=a8e93bda-b832-4ebc-a112-03616856b909 LANG=en_US.UTF-8

label zImage-4.16.6 - MMC
	kernel /boot/zImage-4.16.6
        fdt /boot/dtb-4.16.6/exynos5250-snow.dtb
        append ro rootwait console=tty0 console=ttyUSB0,115200n8 root=PARTUUID=1d4adb65-ea43-c641-9c2d-06ad8f38fb5c
#        append ro rootwait console=tty0 console=ttyUSB0,115200n8 root=UUID=a8e93bda-b832-4ebc-a112-03616856b909 LANG=en_US.UTF-8

label zImage-4.16.6 - USB
	kernel /boot/zImage-4.16.6
        fdt /boot/dtb-4.16.6/exynos5250-snow.dtb
        append ro rootwait console=tty0 console=ttyUSB0,115200n8 root=/dev/sda3

Results:

With Fedora Kernel, screen gets blank just after booting. Nothing else happens, even if waiting for *several* minutes. With custom-built kernel, from vanilla upstream, it boots and works fine with Fedora 27.

PS.: tried with both PARTUUID and UUID. With UUID, it waits forever for a root device.

Comment 1 Mauro Carvalho Chehab 2018-05-02 14:21:36 UTC
Created attachment 1430086 [details]
Config file used to build the custom 4.16.6 kernel with boots with Fedora 27

Comment 2 Mauro Carvalho Chehab 2018-05-02 14:24:48 UTC
Created attachment 1430087 [details]
Kernel log from custom 4.16.6 (with boots fine with Fedora 27)

Comment 3 Jeremy Cline 2018-05-02 14:29:22 UTC
Hi Mauro,

What kernel does the Fedora 27 on a USB stick have? If the Fedora 27 install has 4.16.4+, this sounds like it might be https://bugzilla.redhat.com/show_bug.cgi?id=1572944.

Comment 4 Mauro Carvalho Chehab 2018-05-02 14:36:14 UTC
(In reply to Jeremy Cline from comment #3)
> Hi Mauro,
> 
> What kernel does the Fedora 27 on a USB stick have? If the Fedora 27 install
> has 4.16.4+, this sounds like it might be
> https://bugzilla.redhat.com/show_bug.cgi?id=1572944.

Hi Jeremy,

I'm using the same Kernel (installed at MMC) to boot both. I also tested with Kernel 4.13.12 (both Fedora one and vanilla). Same thing: the vanilla Kernel from -stable works fine; the Fedora one cleans the screen and hangs.

Comment 5 Mauro Carvalho Chehab 2018-05-02 14:38:29 UTC
(In reply to Mauro Carvalho Chehab from comment #0)

> This is not an upstream Kernel problem, as I'm able to build an upstream
> Kernel that boots properly with Fedora 27 (it partially boots on Fedora 28,
> but there is a systemd crash preventing it to finish booting - I'll report
> it as a separate bug).

FYI, that's the systemd crash bug, with vanilla 4.16.6 Kernel, (built with this config file: https://bugzilla.redhat.com/attachment.cgi?id=1430086):

   https://bugzilla.redhat.com/show_bug.cgi?id=1573931

Comment 6 Jeremy Cline 2018-05-02 15:02:54 UTC
Ah, okay. I don't think any of the ARM patches we're carrying should be causing this problem, I'm guessing this is a configuration issue. Does a vanilla kernel built using the Fedora kernel configuration[0] work?

[0] https://src.fedoraproject.org/rpms/kernel/blob/f28/f/kernel-armv7hl.config

Comment 7 Mauro Carvalho Chehab 2018-05-02 16:23:20 UTC
(In reply to Jeremy Cline from comment #6)
> Ah, okay. I don't think any of the ARM patches we're carrying should be
> causing this problem, I'm guessing this is a configuration issue. Does a
> vanilla kernel built using the Fedora kernel configuration[0] work?
> 
> [0]
> https://src.fedoraproject.org/rpms/kernel/blob/f28/f/kernel-armv7hl.config

Yeah, I suspect the same. Just built it with the above config. I didn't generate any initrd file, though. Same thing: when the Kernel/dtb finishes loading, screen is cleaned and nothing else appears.

Comment 8 Mauro Carvalho Chehab 2018-05-02 16:25:03 UTC
(In reply to Mauro Carvalho Chehab from comment #7)
> (In reply to Jeremy Cline from comment #6)
> > Ah, okay. I don't think any of the ARM patches we're carrying should be
> > causing this problem, I'm guessing this is a configuration issue. Does a
> > vanilla kernel built using the Fedora kernel configuration[0] work?
> > 
> > [0]
> > https://src.fedoraproject.org/rpms/kernel/blob/f28/f/kernel-armv7hl.config
> 
> Yeah, I suspect the same. Just built it with the above config. I didn't
> generate any initrd file, though. Same thing: when the Kernel/dtb finishes
> loading, screen is cleaned and nothing else appears.

Ah, forgot to mention: with the above config, it asked several Kconfig questions. I just pressed enter to all of them, in order to get the default.

Comment 9 Jeremy Cline 2018-05-02 17:43:40 UTC
(In reply to Mauro Carvalho Chehab from comment #7)
> (In reply to Jeremy Cline from comment #6)
> > Ah, okay. I don't think any of the ARM patches we're carrying should be
> > causing this problem, I'm guessing this is a configuration issue. Does a
> > vanilla kernel built using the Fedora kernel configuration[0] work?
> > 
> > [0]
> > https://src.fedoraproject.org/rpms/kernel/blob/f28/f/kernel-armv7hl.config
> 
> Yeah, I suspect the same. Just built it with the above config. I didn't
> generate any initrd file, though. Same thing: when the Kernel/dtb finishes
> loading, screen is cleaned and nothing else appears.

Cool. Narrowing down what configuration is necessary may be a little trickier and I've not personally had to do that before so I don't have any great advice on that. Looking for things that are set in your configuration and not set in Fedora's might be a good start.

Seeing what's getting stuck might yield a clue, as well. Are you booting with "quiet" and "rhgb" on the command line? If so, try removing those to see what's going on. Adding "initcall_debug" might also be useful as it will log init calls as they're executed.

Comment 10 Mauro Carvalho Chehab 2018-05-02 19:03:26 UTC
(In reply to Jeremy Cline from comment #9)

> Cool. Narrowing down what configuration is necessary may be a little
> trickier and I've not personally had to do that before so I don't have any
> great advice on that. Looking for things that are set in your configuration
> and not set in Fedora's might be a good start.

Yeah, just wrote a small perl script that does config diffs. Yet, there are too many things there with are different.

> Seeing what's getting stuck might yield a clue, as well. Are you booting
> with "quiet" and "rhgb" on the command line? 

No. Those are usually the first thing I disable on every machine I install linux :-D

> If so, try removing those to
> see what's going on. Adding "initcall_debug" might also be useful as it will
> log init calls as they're executed.

Good tip. I'll try and see if it produces any results.

Comment 11 Peter Robinson 2018-05-02 22:31:59 UTC
So the Arndale which is the same Exynos 5250 SoC as the snow chromebook boots fine on Fedora 28 GA, not sure about display though ATM but I've asked the person that tested it to check when possible.

Did you happen to try whether there was output on HDMI?

Ultimately there's little testing on any of the Exynos devices and it's very much best effort, but with the Arndale and other reports [1] the general support for exynos devices do boot on Fedora-28/4.16 kernels. If would be helpful if you can pare down the differences between the Fedora kernel config and the custom config.

[1] https://blog.christophersmart.com/2018/04/28/fedora-on-odroid-hc1-mini-nas-armv7/

Comment 12 Mauro Carvalho Chehab 2018-05-03 21:41:28 UTC
(In reply to Peter Robinson from comment #11)
> So the Arndale which is the same Exynos 5250 SoC as the snow chromebook
> boots fine on Fedora 28 GA, not sure about display though ATM but I've asked
> the person that tested it to check when possible.
> 
> Did you happen to try whether there was output on HDMI?
> 
> Ultimately there's little testing on any of the Exynos devices and it's very
> much best effort, but with the Arndale and other reports [1] the general
> support for exynos devices do boot on Fedora-28/4.16 kernels. 

Connecting to HDMI doesn't make any difference.

> If would be
> helpful if you can pare down the differences between the Fedora kernel
> config and the custom config.

I was able to make it boot after enabling a lot of options (via a script that was just enabling/changing to 'y' everything that was there on my custom config). I'll try to do more tests later and see if I can identify what's missing.

> 
> [1]
> https://blog.christophersmart.com/2018/04/28/fedora-on-odroid-hc1-mini-nas-
> armv7/

Comment 13 Peter Robinson 2018-05-03 22:30:31 UTC
(In reply to Mauro Carvalho Chehab from comment #12)
> (In reply to Peter Robinson from comment #11)
> > So the Arndale which is the same Exynos 5250 SoC as the snow chromebook
> > boots fine on Fedora 28 GA, not sure about display though ATM but I've asked
> > the person that tested it to check when possible.
> > 
> > Did you happen to try whether there was output on HDMI?
> > 
> > Ultimately there's little testing on any of the Exynos devices and it's very
> > much best effort, but with the Arndale and other reports [1] the general
> > support for exynos devices do boot on Fedora-28/4.16 kernels. 
> 
> Connecting to HDMI doesn't make any difference.
> 
> > If would be
> > helpful if you can pare down the differences between the Fedora kernel
> > config and the custom config.
> 
> I was able to make it boot after enabling a lot of options (via a script
> that was just enabling/changing to 'y' everything that was there on my
> custom config). I'll try to do more tests later and see if I can identify
> what's missing.

So display works on a Odroid-XU4 but doesn't on the Arndale so I think it's a something minor. There's this error which I suspect is related:

[   21.095552] OF: graph: no port node found in /soc/hdmi@14530000

That maps to the DT for the HDMI but I've not had time to trace it further back. Overall the Arndale boots fine headless so if you've already got network configured on the device you might still be able to ssh in to get more info too.

Comment 14 Justin M. Forbes 2018-07-23 15:14:05 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 27 kernel bugs.

Fedora 27 has now been rebased to 4.17.7-100.fc27.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 28, and are still experiencing this issue, please change the version to Fedora 28.

If you experience different issues, please open a new bug report for those.

Comment 15 Justin M. Forbes 2018-08-29 15:09:06 UTC
*********** MASS BUG UPDATE **************
This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 5 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.


Note You need to log in before you can comment on or make changes to this bug.