Bug 2229317

Summary: Lenovo Yoga Laptop Built-in Keyboard not Working on Kernel 6.4.7-200
Product: [Fedora] Fedora Reporter: moe_jo <rawashdeh.mohammad>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 38CC: acaringi, adscvr, airlied, alciregi, bskeggs, fedora, hdegoede, hpa, hugh, jarod, josef, kernel-maint, lgoncalv, linville, masami256, mchehab, niki.guldbrand, ptalbert, steved
Target Milestone: ---Keywords: Regression, Upgrades
Target Release: ---   
Hardware: x86_64   
OS: Linux   
URL: https://discussion.fedoraproject.org/t/86799
Whiteboard:
Fixed In Version: kernel-6.4.9-200.fc38 kernel-6.4.9-100.fc37 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-10 00:41:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Boot log of the kernel in question
none
Output of dmidecode
none
Output of acpidump
none
dmidecode
none
acpidump none

Description moe_jo 2023-08-04 22:05:16 UTC 
1. Please describe the problem:
The built-in keyboard stopped working after upgrading to latest kernel 6.4.7-200 on Fedora 38. It is a Lenovo Yoga Slim Pro Laptop.


2. What is the Version-Release number of the kernel:
kernel-6.4.7-200.fc38.x86_64


3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
Yes! It works with kernel 6.4.6-200. Kernels older than 5.20 also have the same issue according to the link below. It seems that the issue has resurfaced with the latest kernel on Fedora although it has been fixed in the past!
https://wiki.archlinux.org/title/Lenovo_Yoga_Slim_7_Pro_X_(14ARH7)#Keyboard



4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
Yes! If I boot my system using kernel 6.4.7-200 the built-in keyboard will not work at all.


5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
I do not use Rawhide kernel - unstable - on my main laptop.


6. Are you running any modules that not shipped with directly Fedora's kernel?:
None!


7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.
Please see the attached file!

Reproducible: Always
Comment 1 moe_jo 2023-08-04 22:12:11 UTC 
Created attachment 1981748 [details]
Boot log of the kernel in question

The form asks for a kernel dump/log from a previous boot. Attached is the output of the command below:

   journalctl --no-hostname -k > dmesg.txt

Thank you!
Comment 2 Niki Guldbrand 2023-08-04 22:46:32 UTC 
I have the same issue, that started when I upgraded from kernel 6.4.6 to 6.4.7.

I have found this issue: https://bugzilla.kernel.org/show_bug.cgi?id=217718
Which has this patch: https://lore.kernel.org/all/20230728191408.18141-1-mario.limonciello@amd.com/

But that patch doesn't work for my laptop, see comment #15 from the linked kernel issue.
Comment 3 Thorsten Leemhuis 2023-08-05 13:32:54 UTC 
Could anyone affected please attach the output from at least "sudo dmidecode" and ideally also "sudo acpidump"
Comment 4 Hans de Goede 2023-08-05 14:08:16 UTC 
Thank you for the bug report. 

As mentioned already this is likely caused by commit a9c4a912b7dc ("ACPI: resource: Remove "Zen" specific match and quirks"):

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a9c4a912b7dc

I have started a test-build of the Fedora 6.4.8 kernel with these changes reverted:

https://koji.fedoraproject.org/koji/taskinfo?taskID=104399221

Note this is still building atm, it should be finished in a couple of hours.

Please give this test build a try and let us know if it fixes things for you. See here for instructions how to install a kernel directly from koji (Fedora's buildsystem) :

https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt
Comment 5 moe_jo 2023-08-05 17:10:20 UTC 
Created attachment 1981838 [details]
Output of dmidecode
Comment 6 moe_jo 2023-08-05 17:12:07 UTC 
Created attachment 1981839 [details]
Output of acpidump
Comment 7 moe_jo 2023-08-05 18:25:21 UTC 
(In reply to Thorsten Leemhuis from comment #3)
> Could anyone affected please attach the output from at least "sudo
> dmidecode" and ideally also "sudo acpidump"

Please see the two attached files. Thank you
Comment 8 moe_jo 2023-08-05 19:10:29 UTC 
(In reply to Hans de Goede from comment #4)
> Thank you for the bug report. 
> 
> As mentioned already this is likely caused by commit a9c4a912b7dc ("ACPI:
> resource: Remove "Zen" specific match and quirks"):
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/
> ?id=a9c4a912b7dc
> 
> I have started a test-build of the Fedora 6.4.8 kernel with these changes
> reverted:
> 
> https://koji.fedoraproject.org/koji/taskinfo?taskID=104399221
> 
> Note this is still building atm, it should be finished in a couple of hours.
> 
> Please give this test build a try and let us know if it fixes things for
> you. See here for instructions how to install a kernel directly from koji
> (Fedora's buildsystem) :
> 
> https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

Thank you for your reply and sharing the links that give some interesting context! I installed the koji kernel, v6.4.8, and it has fixed the issue for me. Is there anything else I can test for you before I remove it?

I wonder if kernel 6.4.6 has the same commit applied? If not, why the commit was applied to 6.4.7 which caused the issue with the keyboard?! 

So I guess we should keep using an older kernel and the issue, hopefully, will not be shipped with the future kernels!

Thank you!
Comment 9 Niki Guldbrand 2023-08-05 20:39:54 UTC 
Created attachment 1981879 [details]
dmidecode
Comment 10 Niki Guldbrand 2023-08-05 20:40:28 UTC 
Created attachment 1981880 [details]
acpidump
Comment 11 Niki Guldbrand 2023-08-05 20:44:01 UTC 
I have also attached the requsted informtion.

I'm currently not sure how to test the kernel in Silverblue, but i'll see if I can figure something out.
Else I have to wait until the 6.4.8 fedora kernel gets release in ostree
Comment 12 Hans de Goede 2023-08-06 15:18:22 UTC 
Thank you for testing.

I have submitted a revert of a9c4a912b7dc ("ACPI: resource: Remove "Zen" specific match and quirks") upstream now, because I believe that that is the best way to fix the regressions caused by this (until we figure out a better way to deal with the kbd interrupt trigger-type issues):

https://lore.kernel.org/linux-acpi/20230806151453.10690-1-hdegoede@redhat.com/
Comment 13 Fedora Update System 2023-08-08 22:55:05 UTC 
FEDORA-2023-638681260a has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2023-638681260a
Comment 14 Fedora Update System 2023-08-08 22:55:23 UTC 
FEDORA-2023-ddfd3073b3 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-ddfd3073b3
Comment 15 moe_jo 2023-08-09 01:25:15 UTC 
(In reply to Hans de Goede from comment #12)
> Thank you for testing.
> 
> I have submitted a revert of a9c4a912b7dc ("ACPI: resource: Remove "Zen"
> specific match and quirks") upstream now, because I believe that that is the
> best way to fix the regressions caused by this (until we figure out a better
> way to deal with the kbd interrupt trigger-type issues):
> 
> https://lore.kernel.org/linux-acpi/20230806151453.10690-1-hdegoede@redhat.
> com/

Thank you for your quick response and taking care of it! Thank you also for everyone who participated in this report!
Comment 16 Fedora Update System 2023-08-09 02:01:09 UTC 
FEDORA-2023-ddfd3073b3 has been pushed to the Fedora 38 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-ddfd3073b3`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-ddfd3073b3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 17 Fedora Update System 2023-08-09 02:34:18 UTC 
FEDORA-2023-638681260a has been pushed to the Fedora 37 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-638681260a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-638681260a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 18 Fedora Update System 2023-08-10 00:41:50 UTC 
FEDORA-2023-ddfd3073b3 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 19 Fedora Update System 2023-08-10 00:42:44 UTC 
FEDORA-2023-638681260a has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 20 D. Hugh Redelmeier 2023-08-10 21:58:59 UTC 
I have a similar problem with my Asus Vivobook S M5402RA (one with an OLED screen).
The keyboard stopped working after 6.4.6 but works again with 6.4.9.
Great work fixing this!  Thanks!