Bug 1487305 - Raspberry Pi 3: run-initial-setup hangs
Summary: Raspberry Pi 3: run-initial-setup hangs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 27
Hardware: armhfp
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Peter Robinson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker
Depends On:
Blocks: F27BetaBlocker
TreeView+ depends on / blocked
 
Reported: 2017-08-31 15:17 UTC by Alessio
Modified: 2017-10-23 13:33 UTC (History)
30 users (show)

Fixed In Version: kernel-4.13.2-300.fc27
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-09-21 18:43:06 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
journalctl -u initial-setup -u anaconda (10.16 KB, text/plain)
2017-09-05 08:34 UTC, Alessio
no flags Details

Description Alessio 2017-08-31 15:17:59 UTC
Description of problem:

run-initial-setup hangs after a successful boot of F27.
It seems to hang only using a display: via serial console it works ok.

Version-Release number of selected component (if applicable):

Fedora-Server-armhfp-27-20170817.n.3-sda.raw.xz
Fedora-Server-armhfp-27-20170828.n.0-sda.raw.xz
Fedora-Minimal-armhfp-27-20170817.n.3-sda.raw.xz

How reproducible:

Always


Actual results:

Initial setup doesn't accept any option.
Ctrl-C or Crtl-Z doesn't have effect.


Additional info:

Even disabling SELinux doesn't make effects.
The initial setup hangs on every virtual console.

I added "exit 0" on top of /usr/libexec/initial-setup/run-initial-setup before inserting the SD card and booting the Raspberry. Doing that, the boot process completes successfully.

Comment 1 Adam Williamson 2017-09-04 15:36:25 UTC
Peter, have you been seeing / hearing about this?

Comment 2 Adam Williamson 2017-09-04 15:38:02 UTC
Proposing as a Beta blocker, as Pi 3 is a supported platform for F27, and initial-setup needs to run properly in order for the user to be able to log into the system (without i-s there are no accessible accounts): "A working mechanism to create a user account must be clearly presented during installation and/or first boot of the installed system.", from the Alpha criteria.

Comment 3 Martin Kolman 2017-09-04 16:53:28 UTC
So if I understand this correctly you can access the system & IS, but only over the serial console and there is nothing on the graphical consoles, right ?

In any case we need logs to debug this further - Initial Setup is logging to Journal, so this should give you it's log once you can get access to a console:

journalctl -u initial-setup

Thanks in advance! :)

Comment 4 Kamil Páral 2017-09-04 16:54:21 UTC
Discussed during blocker review [1]:

AcceptedBlocker (Beta) - this sounds like a violation of Alpha criterion " A working mechanism to create a user account must be clearly presented during installation and/or first boot of the installed system" for Raspberry Pi 3, which is a supported ARM platform for F27. If further testing indicates it's not so clear-cut, we will revisit

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2017-09-04/

Comment 5 Peter Robinson 2017-09-04 21:43:49 UTC
(In reply to Adam Williamson from comment #1)
> Peter, have you been seeing / hearing about this?

I was aware of something similar, pwhalen knows more details

Comment 6 Alessio 2017-09-05 08:18:32 UTC
(In reply to Martin Kolman from comment #3)
> So if I understand this correctly you can access the system & IS, but only
> over the serial console and there is nothing on the graphical consoles,
> right ?

Well, not exactly.
I'm using Minimal (or Server) image. As a side note, Xfce initial-setup GUI works as expected.
Using Minimal and an HDMI display, initial-setup TUI starts, but doesn't react to any input, and also CTRL-C doesn't have effects.
Sadly, if I enable the serial console (in extlinux.conf and config.txt), initial-setup TUI works, either via serial connection and also using the HDMI display and the keyboard.

I will post journalctl logs ASAP.

Thanks.

Comment 7 Alessio 2017-09-05 08:34:37 UTC
Created attachment 1322112 [details]
journalctl -u initial-setup -u anaconda

Comment 8 Paul Whalen 2017-09-06 13:48:34 UTC
This seems to only affect the Raspberry Pi 3, user input appears on the display but the system doesnt respond. Changing tty's does work but yields the same result. 

Using the Raspberry Pi 2 and other hardware, TUI works as expected.

Comment 9 Adam Williamson 2017-09-06 14:55:02 UTC
Have you checked if it's actually working but not updating the screen (i.e. you can do it blind if you know how it works) or just not working at all?

Comment 10 Alessio 2017-09-06 15:45:00 UTC
(In reply to Adam Williamson from comment #9)
> Have you checked if it's actually working but not updating the screen (i.e.
> you can do it blind if you know how it works) or just not working at all?

It seems to me that it just doesn't work at all.

I don't know if it could be useful. Here is a video: https://alciregi.fedorapeople.org/webm/rpi3.webm

Comment 11 Peter Robinson 2017-09-06 18:33:39 UTC
(In reply to Alessio from comment #10)
> (In reply to Adam Williamson from comment #9)
> > Have you checked if it's actually working but not updating the screen (i.e.
> > you can do it blind if you know how it works) or just not working at all?
> 
> It seems to me that it just doesn't work at all.
> 
> I don't know if it could be useful. Here is a video:
> https://alciregi.fedorapeople.org/webm/rpi3.webm

Could you try the latest u-boot, we fixed usb keyboards there to work recently and that should work with the latest rc4 build, i wonder if the console input is initially passed through from the bootloader before the proper input stack can take over.

https://koji.fedoraproject.org/koji/buildinfo?buildID=966566

Comment 12 Alessio 2017-09-06 19:46:37 UTC
(In reply to Peter Robinson from comment #11)
> Could you try the latest u-boot, we fixed usb keyboards there to work
> recently and that should work with the latest rc4 build, i wonder if the
> console input is initially passed through from the bootloader before the
> proper input stack can take over.

I will try ASAP.
However, as said before, if I put exit 0 on top of /usr/libexec/initial-setup/run-initial-setup, I'm able to login.
Once logged in, removing such line and restarting initial-setup systemd service lead to the same situation.

Comment 13 Alessio 2017-09-07 08:36:10 UTC
(In reply to Peter Robinson from comment #11)
> Could you try the latest u-boot, we fixed usb keyboards there to work
> recently and that should work with the latest rc4 build, i wonder if the
> 
> https://koji.fedoraproject.org/koji/buildinfo?buildID=966566

Is it enough to install these two packages?
uboot-images-elf-2017.09-0.5.rc4.fc27.armv7hl.rpm
uboot-tools-2017.09-0.5.rc4.fc27.armv7hl.rpm

If yes, nothing changes.

Comment 14 Martin Kolman 2017-09-07 09:05:29 UTC
AFAIK Initial Setup worked correctly on the RPI 3 in F26, right ?

Nothing really changed in the relevant IS/Anaconda code since then & the multi-TTY enabled UI seems to continue working correctly elsewhere (VMs, RPI 2) and I don't see anything wrong in the IS journal log in comment 3. 

So I kinda suspect some form of regressions elsewhere, most likely a quirk in the Linux console code.

BTW, this is the relevant part of the Initial Setup code:
https://github.com/rhinstaller/initial-setup/blob/master/initial_setup/tui/tui.py#L96

There is a select that sits on fds corresponding to all the open consoles and on the Anaconda stdout:

rlist, _wlist, _xlist = select.select(fds, [], [], 1.0)

Once the select is triggered by input (user inputs something and presses enter) the input handling is triggered:

https://github.com/rhinstaller/initial-setup/blob/master/initial_setup/tui/tui.py#L127

What likely happens here is that the select is not triggered when enter is pressed, so Initial Setup gets no input and nothing happens.

Comment 15 Paul Whalen 2017-09-07 12:56:50 UTC
(In reply to Peter Robinson from comment #11)
> Could you try the latest u-boot, we fixed usb keyboards there to work
> recently and that should work with the latest rc4 build, i wonder if the
> console input is initially passed through from the bootloader before the
> proper input stack can take over.
> 
> https://koji.fedoraproject.org/koji/buildinfo?buildID=966566

No change with uboot-images-armv7-2017.09-0.5.rc4.fc27.noarch

Comment 16 Paul Whalen 2017-09-07 15:26:58 UTC
(In reply to Martin Kolman from comment #14)
> AFAIK Initial Setup worked correctly on the RPI 3 in F26, right ?

Yes, it worked as expected in F26. 

> 
> Nothing really changed in the relevant IS/Anaconda code since then & the
> multi-TTY enabled UI seems to continue working correctly elsewhere (VMs, RPI
> 2) and I don't see anything wrong in the IS journal log in comment 3. 
> 
> So I kinda suspect some form of regressions elsewhere, most likely a quirk
> in the Linux console code.

We're also seeing some oddities with the console on aarch64 with the rpi3, when using the display it will crash during the boot(messages relating to tty_write_room and uart_write_room). When the serial console is enabled (in config.txt), it boots to initial-setup.

Comment 17 Martin Kolman 2017-09-08 12:32:10 UTC
(In reply to Paul Whalen from comment #16)
> (In reply to Martin Kolman from comment #14)
> > AFAIK Initial Setup worked correctly on the RPI 3 in F26, right ?
> 
> Yes, it worked as expected in F26. 
> 
> > 
> > Nothing really changed in the relevant IS/Anaconda code since then & the
> > multi-TTY enabled UI seems to continue working correctly elsewhere (VMs, RPI
> > 2) and I don't see anything wrong in the IS journal log in comment 3. 
> > 
> > So I kinda suspect some form of regressions elsewhere, most likely a quirk
> > in the Linux console code.
> 
> We're also seeing some oddities with the console on aarch64 with the rpi3,
> when using the display it will crash during the boot(messages relating to
> tty_write_room and uart_write_room). When the serial console is enabled (in
> config.txt), it boots to initial-setup.

This really looks more and more like the cause is somewhere else than in Initial Setup, likely somewhere down in the stack where console handling is done (kernel ?) and the bug should probably be reassigned accordingly to get proper triage.

Comment 18 Paul Whalen 2017-09-08 13:14:53 UTC
initial-setup on display is working with kernel-4.12, broken when using 4.13. Moving to the kernel.

Comment 19 Peter Robinson 2017-09-10 13:43:09 UTC
So looking at this some more it seems the 4.12 DT works with a 4.13 kernel. My guess is that there's been some fixes/improvements around bluetooth support (not there in 4.13 yet) but systemd/something is detecting the BT device on the end of the serial and this is some how screwing stuff up.

Comment 20 Peter Robinson 2017-09-11 00:42:05 UTC
scratch build to test https://koji.fedoraproject.org/koji/taskinfo?taskID=21773271

Comment 21 Alessio 2017-09-11 11:01:13 UTC
(In reply to Peter Robinson from comment #20)
> scratch build to test
> https://koji.fedoraproject.org/koji/taskinfo?taskID=21773271

Installed 
kernel-4.13.1-300.rpi1.fc27.armv7hl.rpm 
kernel-core-4.13.1-300.rpi1.fc27.armv7hl.rpm 
kernel-modules-4.13.1-300.rpi1.fc27.armv7hl.rpm

Now initial-setup works also using an hdmi monitor and a keyboard. :-)

Comment 22 Peter Robinson 2017-09-11 18:01:55 UTC
OK an official build kernel-4.13.1-301.fc27
 https://koji.fedoraproject.org/koji/taskinfo?taskID=21795161

Comment 23 Fedora Update System 2017-09-12 13:38:27 UTC
kernel-4.13.1-302.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-b52203af49

Comment 24 Fedora Update System 2017-09-12 19:57:43 UTC
kernel-4.13.1-302.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-b52203af49

Comment 25 Fedora Update System 2017-09-14 14:22:27 UTC
kernel-4.13.1-303.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-3c2574cd2a

Comment 26 Fedora Update System 2017-09-14 18:23:07 UTC
kernel-4.13.1-303.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-3c2574cd2a

Comment 27 Fedora Update System 2017-09-15 03:12:21 UTC
kernel-4.13.2-300.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0da1ebbe4f

Comment 28 Fedora Update System 2017-09-15 16:53:10 UTC
kernel-4.13.2-300.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-0da1ebbe4f

Comment 30 Kamil Páral 2017-09-19 15:44:05 UTC
sumantro, which kernel have you tested with?

Comment 32 Kamil Páral 2017-09-19 18:24:16 UTC
Ok, so Workstation works but Server and Minimal don't. Once kernel kernel-4.13.2-300.fc27 is included in the next compose, we need to test Server and Minimal.

Comment 33 Fedora Update System 2017-09-20 15:26:01 UTC
kernel-4.13.2-300.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 34 Peter Robinson 2017-09-20 15:30:10 UTC
Kmail: Why was this reopened without any comment as to why?

Comment 35 Kamil Páral 2017-09-20 15:31:25 UTC
This is an accepted blocker, we still need to verify it fixes the bug, even though it has been pushed.

Comment 36 Kamil Páral 2017-09-21 08:19:29 UTC
Paul, can you please verify with 20170921 compose once it's up? Thanks.

Comment 37 sumantro 2017-09-21 17:43:15 UTC
Fedora-27-20170921.n.0 Minimal works perfectly fine.

Comment 38 Kamil Páral 2017-09-21 18:43:06 UTC
Thanks, closing.

Comment 39 sumantro 2017-09-22 00:03:29 UTC
Fedora-27-20170921.n.0 Server works fine too

Comment 40 Paul Whalen 2017-10-23 13:33:35 UTC
Working with the latest images.


Note You need to log in before you can comment on or make changes to this bug.