Bug 1649531

Summary:	No characters visible on VTs with kbd-2.0.4-10
Product:	[Fedora] Fedora	Reporter:	Adam Williamson <awilliam>
Component:	kbd	Assignee:	Vitezslav Crhonek <vcrhonek>
Status:	CLOSED RAWHIDE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	jpokorny, robatino, vcrhonek
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	kbd-2.0.4-12.fc30	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-12-10 12:05:54 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1574713

Description Adam Williamson 2018-11-13 19:42:13 UTC

This is pretty weird, but seems to be true: with kbd-2.0.4-10 (which landed in Fedora-Rawhide-20181113.n.0), no characters are visible on TTYs. They're *working*, but you can only see the cursor. You can't see the login prompt, or the bash prompt, or anything you type, or any output from any commands, or...anything at all.

It causes all the openQA tests to fail, because they all try to do something or other at a console, at some point. Here's how it looks:

https://openqa.fedoraproject.org/tests/308276#step/_do_install_and_reboot/20

That's actually a working bash prompt, but...it doesn't look like one...

I confirmed this manually in a local VM with both a netinst and a live image from the compose. Then I built a couple of custom installer images with lorax.

The first was just a straightforward netinst build from the 20181113.n.0 tree with no package changes, as a control: as expected, it displays the same 'invisible tty' behaviour.

In the second, I added a side repo with this scratch build of kbd: https://koji.fedoraproject.org/koji/taskinfo?taskID=30866179

For that build, I reset git to the state of kbd-2.0.4-9 , then bumped the release to '10.1.really9', so it would be a higher version than 2.0.4-10. So it's basically a build with all the -10 changes dropped, but that's higher versioned than -10.

I verified that the scratch kbd actually got into the test image, then booted it...and the ttys are visible and work as normal. So this really seems to indicate that the bug is, somehow, in the kbd-2.0.4-10 changes.

Comment 1 Adam Williamson 2018-11-13 20:18:42 UTC

Huh...but...I tested an installed system, and it doesn't behave the same. The system installs with -10, and the bug is present...but downgrading to -9 doesn't help, nor does "upgrading" to -10.1.really9 . Haven't found a way to make the installed system *not* be buggy, yet.

So...I'm a bit confused now...but this seems clearly a Beta blocker, as it affects an installed system too. Violates "A system installed without a graphical package set must boot to a state where it is possible to log in through at least one of the default virtual consoles."...okay TECHNICALLY you *can* log in, but I think we can go with it. :)

Comment 2 Adam Williamson 2018-11-13 20:33:40 UTC

Hum - running 'setfont' from a logged-in console suddenly makes all the text appear...

Comment 3 Adam Williamson 2018-11-13 21:10:27 UTC

OK, curiouser and curiouser: it really *does* seem like kbd is the cause, but the issue is somehow tied to initial install (or image build, in the case of an image). If I run a netinstall such that -10 gets installed during initial install, the system has the bug, and _just_ upgrading to -10.1.really9 and rebooting does not fix it. If I run a netinstall such that -10.1.really9 gets installed during initial install, the system does *not* have the bug.

On systems that have the bug, running 'setfont' with -10 installed does *not* make the console text appear, but running 'setfont' with -10.1.really9 installed *does* make the console text appear.

At this point I think we have enough data to revert the -10 changes and do a -11, so I'll do that, just to get Rawhide fixed; then we can take as much time as necessary to figure out what the actual problem in the -10 changes is...

Comment 4 Adam Williamson 2018-11-13 23:28:52 UTC

OK, I triaged it down by commenting out one section of the patch at a time. The problematic changes are the changes to psffontop.c , the ones that add freeing of 'inputbuf' to various points in readpsffont(). I honestly can't see *why* those changes are bad, it must be something subtle, but...that's the problem.

Comment 5 Jan Pokorný [poki] 2018-11-14 08:33:45 UTC

Thanks for -11 build, just hit this.

Comment 6 Jan Pokorný [poki] 2018-11-14 08:35:02 UTC

Forgot to provide diagnostics:

Nov 14 09:24:11 systemd-vconsole-setup[533]: KD_FONT_OP_GET failed while trying to get the font metadata: Function not implemented
Nov 14 09:24:11 systemd-vconsole-setup[533]: Fonts will not be copied to remaining consoles

Comment 7 Vitezslav Crhonek 2018-11-14 10:06:56 UTC

(In reply to Adam Williamson from comment #4)
> OK, I triaged it down by commenting out one section of the patch at a time.
> The problematic changes are the changes to psffontop.c , the ones that add
> freeing of 'inputbuf' to various points in readpsffont(). I honestly can't
> see *why* those changes are bad, it must be something subtle, but...that's
> the problem.

Thanks for investigation and hotfix!

Comment 8 Jan Pokorný [poki] 2018-11-14 10:37:37 UTC

Btw. when the affected package happens to be present around or
directly participate in the package update transaction delivering
also dracut/kernel, the problem will spread into initramfs
(e.g. affecting LUKS password prompt).

Comment 9 Vitezslav Crhonek 2018-12-10 12:05:54 UTC

I've reapplied the patch without breaking changes in psffontop.c and it looks that everything works fine.