Bug 2302253

Created attachment 2043228 [details]
acpidump LG Gram laptop

Iñaki, can you please reboot your laptop into a *not* working kernel and then immediately after boot run:

sudo dmesg > dmesg.txt

and attach the generated dmesg.txt file here ?

Having the kernel logs will help in debugging this.

Also as mentioned in the possibly related bug 2298938, please give this test kernel build which contains a possible fix for bug 2298938 a try:

https://koji.fedoraproject.org/koji/taskinfo?taskID=121344486

Here are instructions for installing a kernel directly from koji (Fedora's buildsystem): https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

I tried that kernel but it doesn't help. Same issues. :(
I saved the output from dmesg. Any part that I should supress from the output to post it here?

(In reply to Iñaki Ucar from comment #3)
> I tried that kernel but it doesn't help. Same issues. :(

Thank you for giving it a try.

> I saved the output from dmesg. Any part that I should supress from the
> output to post it here?

Not really, generally speaking there should not be any privacy sensitive info in dmesg output.

Created attachment 2043237 [details]
dmesg output LG Gram laptop

(In reply to Iñaki Ucar from comment #5)
> Created attachment 2043237 [details]
> dmesg output LG Gram laptop

Thank you. I'm seeing some possible related errors but I'm not sure. Can you also collect a dmesg with a working (e.g. 6.9.6) kernel so that I can see if that shows the same errors or not ?

Created attachment 2043238 [details]
dmesg output with working kernel

OK, now we are getting somewhere. Here is an email which I just send to the kernel list about this:

Ok, so the bugzilla now has 2 different dmesg outputs:

1. 6.9.6, this kernel works without problems

2. 6.9.12 with the following patch you suggested on top:

--- a/drivers/acpi/ec.c
+++ b/drivers/acpi/ec.c
@@ -1788,7 +1788,7 @@ void __init acpi_ec_dsdt_probe(void)
 	 * At this point, the GPE is not fully initialized, so do not to
 	 * handle the events.
 	 */
-	ret = acpi_ec_setup(ec, NULL, true);
+	ret = acpi_ec_setup(ec, NULL, false);
 	if (ret) {
 		acpi_ec_free(ec);
 		return;

Unfortunately this does not help. dmesg shows some EC _REG errors, which
are now (with the above diff applied) shown just before the
"Boot DSDT EC initialization complete" message, which shows that _REG now
runs from acpi_ec_add() rather then before:

[    1.007566] ACPI BIOS Error (bug): Could not resolve symbol [\_TZ.FN00._OFF], AE_NOT_FOUND (20230628/psargs-330)
[    1.007576] ACPI Error: Aborting method \_SB.PC00.LPCB.H_EC.EREG due to previous error (AE_NOT_FOUND) (20230628/psparse-52
[    1.007580] ACPI Error: Aborting method \_SB.PC00.LPCB.H_EC._REG due to previous error (AE_NOT_FOUND) (20230628/psparse-52
[    1.007639] ACPI: EC: interrupt unblocked
[    1.007640] ACPI: EC: event unblocked
[    1.007675] ACPI: EC: EC_CMD/EC_SC=0x66, EC_DATA=0x62
[    1.007676] ACPI: EC: GPE=0x6e
[    1.007677] ACPI: \_SB_.PC00.LPCB.LGEC: Boot DSDT EC initialization complete
[    1.007679] ACPI: \_SB_.PC00.LPCB.LGEC: EC: Used to handle transactions and events

Note that the errors are from calling _REG on \_SB.PC00.LPCB.H_EC, where as the actual
EC (and the only acpi_device on which _REG would get called for the EC Opregion before) is:
\_SB_.PC00.LPCB.LGEC.

Looking at the DSDT it seems that the H_EC is not used and is leftover from a copy/paste
from some reference design DSDT. Its _REG however does write to the EC before hitting the error
and I think that that write may be causing the issue...

The H_EC device does have an _STA method and looking closer the troublesome EREG method is
also called from _INI. So I guess that _STA is returning 0 causing _INI to not run and
that is the reason why we are not seeing the same EREG errors with kernel 6.9.6 where _REG is
only called for the EC opregion on \_SB_.PC00.LPCB.LGEC and not for the entire ACPI device
hierarchy as is done with >= 6.9.7 .

Maybe we should only call _REG for the EC opregion on present devices (and devices without
a _STA)?

Also note that both LGEC and H_EC use the same cmd + data ports.

I'll go and ask the reporter to retreive the status of both LGEC and H_EC and then see
from there.

As I mentioned in the email it would be useful to know what the _STA (status / present) method of the H_EC device returns.

Can you please from a terminal run:

cat /sys/bus/acpi/devices/PNP0C09\:00/path
cat /sys/bus/acpi/devices/PNP0C09\:00/status
cat /sys/bus/acpi/devices/PNP0C09\:01/path
cat /sys/bus/acpi/devices/PNP0C09\:01/status

and let me know the output of all 4 commands, please also copy and paste the actual commands from the terminal into your next comment here in bugzilla so that I can easily match up the output to each command.

Here it is, with kernel 6.9.6:

$ cat /sys/bus/acpi/devices/PNP0C09\:00/path
\_SB_.PC00.LPCB.H_EC
$ cat /sys/bus/acpi/devices/PNP0C09\:00/status
0
$ cat /sys/bus/acpi/devices/PNP0C09\:01/path
\_SB_.PC00.LPCB.LGEC
$ cat /sys/bus/acpi/devices/PNP0C09\:01/status
15

(In reply to Iñaki Ucar from comment #10)
> Here it is, with kernel 6.9.6:
> 
> $ cat /sys/bus/acpi/devices/PNP0C09\:00/path
> \_SB_.PC00.LPCB.H_EC
> $ cat /sys/bus/acpi/devices/PNP0C09\:00/status
> 0
> $ cat /sys/bus/acpi/devices/PNP0C09\:01/path
> \_SB_.PC00.LPCB.LGEC
> $ cat /sys/bus/acpi/devices/PNP0C09\:01/status
> 15

Great thank you, so this confirms that the _STA method for the extra \_SB_.PC00.LPCB.H_EC EC device which seems to be causing the issues returns 0 / not present.

So before the recent EC handling changes only _REG of \_SB_.PC00.LPCB.LGEC was run, but now the other _REG runs too and that seems to be causing the issue.

I'm not 100% sure how to best fix this and Rafael the upstream ACPI is taking a break this week. I'll get back to you when we have a possible fix to test. In the mean time you can stick with 6.9.6 as you are already doing.

It was expected, but for completeness, I just tried 6.10.3 (in updates-testing) and same issues.

Rafael, the upstream kernel ACPI maintainer has made a branch with a proposed fix this issue (1) available:

https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git/log/?h=acpi-ec-fixes

I have started a test Fedora 40 6.10.4 kernel build with the patches from the acpi-ec-fixes branch added:
https://koji.fedoraproject.org/koji/taskinfo?taskID=121834209

This kernel is still building at the moment (should be done in a couple of hours), please give this kernel a try once it is done building and let us know if this resolves the brightness key-press issue.

Here are instructions for installing a kernel directly from koji (Fedora's buildsystem): https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt



1) A fix for what we believe is the root cause of this issue, the calling of _REG on the non active second EC device in the ACPI table.

The latest test-kernel with the upstream fix for this is done building and is ready for testing now.

Created attachment 2044019 [details]
dmesg for working patch

There are still some ACPI errors in the dmesg output, but this solved the issues I had!

> There are still some ACPI errors in the dmesg output, but this solved the issues I had!

Thank you for testing. I've let Rafael know that his patches solve the problem so that he can move ahead with adding these to the upstream kernel.

I checked and those ACPI errors were already there with 6.9.6, these refer to a LG custom OpRegion which Linux does not implement and these are harmless. Another kernel contributor has actually been working on supporting this OpRegion, so at sometime in the future these errors will hopefully be gone too:

https://lore.kernel.org/platform-driver-x86/20240813022903.20567-1-W_Armin@gmx.de/

Do you happen to know when will this patch land in F40? Version 6.10.6 now is in testing, and it seems to solve the execheap denials that are so annoying.

(In reply to Iñaki Ucar from comment #18)
> Do you happen to know when will this patch land in F40? Version 6.10.6 now
> is in testing, and it seems to solve the execheap denials that are so
> annoying.

The patches fixing this are in the 6.10.y queue now:

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/log/?h=queue/6.10

(look for "ACPI: EC: Evaluate _REG outside the EC scope more carefully")

So they should show up in 6.10.7

So just a bit of patience and then Fedora will have them.

BTW, not sure if it is related, and please let me know if I should open a new issue for this... but I noticed that I see the IRQ storm (ACPI IRQ 9) again with the Thunderbolt hub connected. Not sure how to debug this either.

It definitely has something to do with this, because I see:

$ cat /sys/firmware/acpi/interrupts/gpe{_all,6E}
 2919465
 2918394  EN     enabled      unmasked

And if I disable gpe6E, the brightness buttons do not work anymore.

Never mind, it seems there's already a bug upstream (https://bugzilla.kernel.org/show_bug.cgi?id=217076). I'll continue the discussion there.