Bug 2017043

Proposed as a Blocker and Freeze Exception for 35-final by Fedora user sumantrom using the blocker tracking app because:

 https://fedoraproject.org/wiki/Fedora_35_Final_Release_Criteria#First_boot_experience

Are you connected through Bluetooth or a USB dongle, when you encounter this bug? Does the keyboard also support connecting using a USB cable? Does that work?

Also, since you can use the tty, please reproduce the bug and then provide `journalctl -b` output.

(In reply to Kamil Páral from comment #2)
> Are you connected through Bluetooth or a USB dongle, when you encounter this
> bug? Does the keyboard also support connecting using a USB cable? Does that
> work?

I am connected using a USB dongle. 
The keyboard when wired works pretty fine

What happens if you disconnect and reconnect the dongle while being in the g-i-s, does it fix the problem?

Also, can you please test Workstation install on x86_64, using the same keyboard, to figure out whether it is aarch64-specific?

(In reply to Kamil Páral from comment #5)
> What happens if you disconnect and reconnect the dongle while being in the
> g-i-s, does it fix the problem?

Nopes doesn't.

My second round of testing reveals, GIS is able to take in the wifi password but just wasn't able to take in Online accounts and any of the user creation steps.

(In reply to Kamil Páral from comment #6)
> Also, can you please test Workstation install on x86_64, using the same
> keyboard, to figure out whether it is aarch64-specific?

The said keyboard is my daily driver and I have been testing the x86_64 over VM since morning. Everything looks good there.
This is very specific to aarch64 and rpi4, unless someone else can reproduce it in some other board.

The journalctl output with thing stuck in between is hard to grab as it requires me to log in as user and GIS was specifically supposed to help create one.

Reproduced on the RPi4 with a Logitech wireless keyboard, same keyboard works OK on the Jetson Nano F35 Workstation.

I can confirm this on both RPi 3 and RPi4 with aarch64 workstation install. Tested with Logitech wireless keyboard/trackpad combo and am unable to enter Name and Username. Wifi password entry works fine. Tested on RPi400 (with internal keyboard) and it works OK.

Discussed during the 2021-10-25 blocker review meeting: [0]

The decision to classify this bug as a "RejectedBlocker (Final)" and an "AcceptedFreezeException (Final)" was made as there doesn't seem to a be blocking scenario (hardware plus input device plus environment) which is affected, but accepted as an FE issue as it cannot be fixed with an update and would be good to fix for the non-blocking RPi4 case if possible.

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2021-10-25/f35-blocker-review.2021-10-25-16.02.txt

(In reply to sumantro from comment #8)
> The journalctl output with thing stuck in between is hard to grab as it
> requires me to log in as user and GIS was specifically supposed to help
> create one.

If you can edit the kernel args in the bootloader (does it use standard grub?), you can add "systemd.debug-shell" there, and then you'll have a working root shell on TTY9.

(In reply to Kamil Páral from comment #12)
> (In reply to sumantro from comment #8)
> > The journalctl output with thing stuck in between is hard to grab as it
> > requires me to log in as user and GIS was specifically supposed to help
> > create one.
> 
> If you can edit the kernel args in the bootloader (does it use standard
> grub?), you can add "systemd.debug-shell" there, and then you'll have a
> working root shell on TTY9.

Thanks for the hint. I tried doing that last night itself, for some reason whenever I press e on GRUB menu
it just takes me to edit and then doesn't show me a cursor and in a second takes me to emergency shell right away
I gave up honestly on that

> This is very specific to aarch64 and rpi4, unless someone else can reproduce
> it in some other board.

The Raspberry Pi 4 isn't a supported device as yet, especially on Workstation, as there is no accelerated graphics, so I wouldn't be surprised if this was a fbdev/Wayland weird interaction issue.

If this is purely RPi4 I am -1 on a blocker

I have no clue what the correct component for this issue would be, but it's definitely not gnome-initial-setup. It's just a GTK application, it doesn't handle your input devices.

(In reply to Michael Catanzaro from comment #15)
> I have no clue what the correct component for this issue would be, but it's
> definitely not gnome-initial-setup. It's just a GTK application, it doesn't
> handle your input devices.

Maybe gtk rust bindings?

(In reply to Peter Robinson from comment #16)
> Maybe gtk rust bindings?

Maybe you are confusing gnome-initial-setup with gnome-tour? gnome-initial-setup is written in C. It is a normal GTK application, except it's running in a special desktop session.

Guess: try disabling selinux on the kernel command line?

*** Bug 1855522 has been marked as a duplicate of this bug. ***

So I think this is a Wayland or libinput bug, it's likely actually a bug on all architectures except for the fact on x86 there's always a "AT Translated Set 2 keyboard" even if there's not one physically attached which ultimately masks the problem there.

I can actually recreate this issue on Terminal too

(In reply to Peter Robinson from comment #19)
> So I think this is a Wayland or libinput bug

Or mutter/gnome-shell

(In reply to Peter Robinson from comment #19)

> all architectures except for the fact on x86 there's always a "AT Translated
> Set 2 keyboard" even if there's not one physically attached which ultimately
> masks the problem there.

In the case of the Jetson Nano success I reported in comment #9, Peter suggested this was due to the two gpio_keys (power and recovery) on the Nano which present as input devices. To test this I blacklisted 'gpio_keys' module and was able to reproduce the issue.

All input from hardware devices that passes through mutter/gnome-shell come from libinput, so moving there.

So digging into this some more via libinput util that comes with the library without any USB keyboard/mice attached:

My current theory is that a device that has multipurpose capabilities "Capabilities:     keyboard pointer" are for some reason excluded probably due to a bug. I don't think that's actually a bug in libinput but I suspect more likely something that sits on top of it (gtk3, mutter etc)

Jetson nano (keyboard via some gpio buttons):
$ libinput list-devices
Device:           gpio-keys
Kernel:           /dev/input/event0
Group:            1
Seat:             seat0, default
Capabilities:     keyboard
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      n/a
Nat.scrolling:    n/a
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   none
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   n/a
Rotation:         n/a

Raspberry Pi 4 (keyboard and pointer via HDMI CEC interfaces):
$ libinput list-devices
Device:           vc4
Kernel:           /dev/input/event0
Group:            1
Seat:             seat0, default
Capabilities:     keyboard pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   *button
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

Device:           vc4
Kernel:           /dev/input/event1
Group:            2
Seat:             seat0, default
Capabilities:     keyboard pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   *button
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

Raspberry Pi 3 (keyboard and pointer via HDMI CEC interface):
$ libinput list-devices
Device:           vc4
Kernel:           /dev/input/event0
Group:            1
Seat:             seat0, default
Capabilities:     keyboard pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   *button
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

The logitech wireless keyboard/trackpad:
Device:           Logitech K400 Plus
Kernel:           /dev/input/event0
Group:            1
Seat:             seat0, default
Capabilities:     keyboard pointer
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: disabled
Calibration:      n/a
Scroll methods:   button
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   flat *adaptive
Rotation:         n/a

Proposed as a Blocker for 36-beta by Fedora user pbrobinson using the blocker tracking app because:

 This bug affects a number of devices. We now know it's not limited to just GNOME initial setup on the RPi. It's reproducible with gnome-terminal and on other devices. It's possible it's reproducible on certain x86 devices as well.

best way to check if it's libinput's fault would be libinput debug-events (optionally with --verbose). If you see the expected key events, it's not in libinput but in the caller. libinput doesn't care too much about multiple capabilities, they're more a flag for the caller, so I don't think that's the reason (debug-events will show warnings though if we drop events because of capability mismatch). 

*maybe* it's an issue with mutter expecting at least one keyboard+pointer, as separate devices?

(In reply to Peter Hutterer from comment #26)
> best way to check if it's libinput's fault would be libinput debug-events
> (optionally with --verbose). If you see the expected key events, it's not in
> libinput but in the caller. libinput doesn't care too much about multiple

Adding the wireless usb dongle, moving the mouse and pressing keys with 'libinput debug-events --verbose':

event0  - Logitech M720 Triathlon: is tagged by udev as: Keyboard Mouse
event0  - Logitech M720 Triathlon: device is a pointer
event0  - Logitech M720 Triathlon: device is a keyboard
-event0   DEVICE_ADDED            Logitech M720 Triathlon           seat0 default group9  cap:kp left scroll-nat scroll-button
event1  - Logitech K850: is tagged by udev as: Keyboard
event1  - Logitech K850: device is a keyboard
-event1   DEVICE_ADDED            Logitech K850                     seat0 default group10 cap:kp scroll-nat
-event0   POINTER_MOTION          +482.889s	  0.96/  0.96 ( +3.00/ +3.00)
 event0   POINTER_MOTION          +482.897s	 25.86/ 39.65 (+15.00/+23.00)
 event0   POINTER_MOTION          +482.905s	  3.94/ 17.73 ( +2.00/ +9.00)
-event1   KEYBOARD_KEY            +488.684s	*** (-1) pressed
 event1   KEYBOARD_KEY            +488.764s	*** (-1) pressed
 event1   KEYBOARD_KEY            +488.884s	*** (-1) released
 event1   KEYBOARD_KEY            +488.906s	*** (-1) released
 event1   KEYBOARD_KEY            +488.924s	*** (-1) pressed
 event1   KEYBOARD_KEY            +489.066s	*** (-1) released
 event1   KEYBOARD_KEY            +489.368s	*** (-1) pressed
 event1   KEYBOARD_KEY            +489.548s	*** (-1) released

The expected key events were shown, but did not work in gnome-initial setup. 

Adding a wired keyboard:

event6  - HP USB Business Slim Keyboard: is tagged by udev as: Keyboard
event6  - HP USB Business Slim Keyboard: device is a keyboard
-event6   DEVICE_ADDED            HP USB Business Slim Keyboard     seat0 default group4  cap:k
event2  - HP USB Business Slim Keyboard: is tagged by udev as: Keyboard
event2  - HP USB Business Slim Keyboard: device is a keyboard
-event2   DEVICE_ADDED            HP USB Business Slim Keyboard     seat0 default group4  cap:k
event3  - HP USB Business Slim Keyboard System Control: is tagged by udev as: Keyboard
event3  - HP USB Business Slim Keyboard System Control: device is a keyboard
-event3   DEVICE_ADDED            HP USB Business Slim Keyboard System Control seat0 default group4  cap:k
event7  - HP USB Business Slim Keyboard Consumer Control: is tagged by udev as: Keyboard
event7  - HP USB Business Slim Keyboard Consumer Control: device is a keyboard
-event7   DEVICE_ADDED            HP USB Business Slim Keyboard Consumer Control seat0 default group4  cap:kp scroll-nat

After adding the wired keyboard, text entry worked with the wireless keyboard.

> *maybe* it's an issue with mutter expecting at least one keyboard+pointer, as separate devices?

Not sure how well sending key events via something that was determined a pointer device will work. Do you know if that'll work Carlos?

> -event0   POINTER_MOTION          +482.889s	  0.96/  0.96 ( +3.00/ +3.00)
> -event1   KEYBOARD_KEY            +488.684s	*** (-1) pressed

Seems like they are different devices however.

It's possible to check what mutter classified a device from libinput as (using gnome-shell):

 1. Open looking glass (Alt+F2 -> "lg")
 2. Paste the following: {for (device of Clutter.get_default_backend().get_default_seat().list_devices()) log("device " + device.device_node + " has type " + device.device_type);}

This will log each device node -> determined device type. 0 means pointer, 1 means keyboard, 4 means tablet, 5 means touchpad, 6 means touchscreen.

looks like the M720 is a pointer + keyboard device which is very common for mice these days (firmware allows to send keyboard events, so the HID bits are set). and the K860 is a separate device anyway. So really, there doesn't seem to be anything special about the devices here, and judging by the debug-events output at least the lower levels work fine. I have no idea why this would be an issue in g-i-s.

Out of interest, if you plug the HP keyboard in, ensure the other keyboard works, then unplug the HP keyboard again - does the other keyboard keep working?

> debug-events output at least the lower levels work fine. I have no idea why
> this would be an issue in g-i-s.

It's not just a g-i-s issue, as I mentioned above I recreated the same issue in gnome-terminal. In g-i-s the passphrase input works for the WiFi option (I suspect the password dialog is special here).
 
> Out of interest, if you plug the HP keyboard in, ensure the other keyboard
> works, then unplug the HP keyboard again - does the other keyboard keep
> working?

It's instantaneous if I plug in a wired USB keyboard, it starts working again, and as soon as I remove it the input from the wireless keyboard stops again.

This leads the issue back to mutter; maybe the device classification is problematic; we'll only advertise wl_keyboard support if there is a keyboard. For reference: https://gitlab.gnome.org/GNOME/mutter/-/blob/d5f2ec6f1e5589c7ce24e54be93b92010f55fdef/src/backends/native/meta-input-device-native.c#L1459-1479

(In reply to Jonas Ådahl from comment #31)
> This leads the issue back to mutter; maybe the device classification is
> problematic; we'll only advertise wl_keyboard support if there is a
> keyboard. For reference:
> https://gitlab.gnome.org/GNOME/mutter/-/blob/
> d5f2ec6f1e5589c7ce24e54be93b92010f55fdef/src/backends/native/meta-input-
> device-native.c#L1459-1479

I wonder if that, or a similar check, is only expecting one capability of a device and hence only reads only the first/last/a single option and hence misses there's a keyboard if the one returned is a pointer?

Yes, that definitely looks like it. The logic of the block is "if device has capability X, declare it's a device of type X and return, otherwise try the next capability in the list". And keyboard is the *last* option in the list. So for instance if we get here:

  else if (libinput_device_has_capability (ldev, LIBINPUT_DEVICE_CAP_POINTER))
    return CLUTTER_POINTER_DEVICE;

we're gonna declare that it's a "pointer" device and be done, we will never reach:

  else if (libinput_device_has_capability (ldev, LIBINPUT_DEVICE_CAP_KEYBOARD))
    return CLUTTER_KEYBOARD_DEVICE;

As we can see in pwhalen's output above, that looks a lot like what might be happening here:

event0  - Logitech M720 Triathlon: device is a pointer
event0  - Logitech M720 Triathlon: device is a keyboard

i.e. the device sure seems to have both pointer and keyboard capabilities.

Discussed during the 2022-02-07 blocker review meeting: [0]

The decision to classify this bug as an "AcceptedBlocker (Beta)" was made as it violates the following criterion:

"A system installed with a release-blocking desktop must boot to a log in screen where it is possible to log in to a working desktop using a user account created during installation or a 'first boot' utility" (can't log in or create user if you can't type) on affected systems (aarch64 with multi-capability input device).

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2022-02-07/f36-blocker-review.2022-02-07-17.00.txt

Actually, hmm, looking at it again it's not so clear. pwhalen's output shows two devices:

event0  - Logitech M720 Triathlon: is tagged by udev as: Keyboard Mouse
event0  - Logitech M720 Triathlon: device is a pointer
event0  - Logitech M720 Triathlon: device is a keyboard
-event0   DEVICE_ADDED            Logitech M720 Triathlon           seat0 default group9  cap:kp left scroll-nat scroll-button
event1  - Logitech K850: is tagged by udev as: Keyboard
event1  - Logitech K850: device is a keyboard
-event1   DEVICE_ADDED            Logitech K850                     seat0 default group10 cap:kp scroll-nat

event0 is a Logitech M720, which is indeed a mouse. It gets both pointer and keyboard capabilities probably for the reason whot said (its firmware is capable of sending keyboard events). Per the code above mutter will treat it as a pointer, which happens to probably be right.

But then event1 is a Logitech K850, which really is a keyboard. Just a keyboard. And libinput seems to treat it as just a keyboard, it doesn't show any other capability. So you'd think mutter should too.

Sounds like we need pwhalen (or someone else affected) to do what jadahl suggested in comment #28. I have a Logitech wireless mouse but unfortunately not a wireless keyboard :/

> 
> It's possible to check what mutter classified a device from libinput as
> (using gnome-shell):
> 
>  1. Open looking glass (Alt+F2 -> "lg")
>  2. Paste the following: {for (device of
> Clutter.get_default_backend().get_default_seat().list_devices()) log("device
> " + device.device_node + " has type " + device.device_type);}
> 
> This will log each device node -> determined device type. 0 means pointer, 1
> means keyboard, 4 means tablet, 5 means touchpad, 6 means touchscreen.

libinput debug-events --verbose

event1  - Logitech K850: is tagged by udev as: Keyboard
event1  - Logitech K850: device is a keyboard
-event1   DEVICE_ADDED            Logitech K850                     seat0 default group9  cap:kp scroll-nat
event0  - Logitech M720 Triathlon: is tagged by udev as: Keyboard Mouse
event0  - Logitech M720 Triathlon: device is a pointer
event0  - Logitech M720 Triathlon: device is a keyboard

From the logs:

...
Feb 08 12:33:19 rpi4-3w systemd-logind[723]: Watching system buttons on /dev/input/event0 (Logitech M720 Triathlon)
Feb 08 12:33:19 rpi4-3w systemd-logind[723]: Watching system buttons on /dev/input/event1 (Logitech K850)

Feb 08 12:33:25 rpi4-3w gnome-shell[1514]: device /dev/input/event1 has type 0
Feb 08 12:33:25 rpi4-3w gnome-shell[1514]: device /dev/input/event0 has type 0

The keyboard/mouse combo uses a single USB dongle, libinput shows the keyboard has both keyboard and pointer capabilities:

libinput list-devices

Device:           Logitech K850
Kernel:           /dev/input/event1
Group:            9
Seat:             seat0, default
Capabilities:     keyboard pointer 
Tap-to-click:     n/a
Tap-and-drag:     n/a
Tap drag lock:    n/a
Left-handed:      n/a
Nat.scrolling:    disabled
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   none
Click methods:    none
Disable-w-typing: n/a
Accel profiles:   n/a
Rotation:         n/a


Using the snippet in comment #28 , the keyboard is identified as a pointer device:

Feb 08 12:33:25 rpi4-3w gnome-shell[1514]: device /dev/input/event1 has type 0

Moving the bug to mutter based on that.

If that's the problem, then fixing it seems unfortunately difficult, at least to me. I did look into that a bit, and mutter is quite tied to the idea that a given 'device' will be of a single 'type'. It's built quite heavily into the design. Fixing this would likely require a decent amount of refactoring, it doesn't look like a simple tweak.

Thanks to Carlos, there's a proposed fix for this now (though it's quite big so could do with some review). I'll backport it for testing.

Scratch build for testing: https://koji.fedoraproject.org/koji/taskinfo?taskID=83854148

FEDORA-2022-21d81706cd has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-21d81706cd

The GNOME 42~rc megaupdate has the latest fix for this backported in mutter-42~rc-4.fc36; can folks please test and see if it works? Thanks!

FEDORA-2022-21d81706cd has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-21d81706cd`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-21d81706cd

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Why is this needinfo pwhalen?

FEDORA-2022-21d81706cd has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Peter: to confirm the fix, as he's one of the people who had a definite reproducer setup.

Confirmed fixed with Beta 1.1(mutter-42~rc-4.fc36).

Tested on RPi4 with wireless keyboard/trackpad combo. Was able to get through GIS without problems. mutter-42~rc-4.fc36

Great, thanks folks. We can close this then.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days