1. Please describe the problem: kernel-core is missing kmod ec_sys 2. What is the Version-Release number of the kernel: 5.11.? and later. Discovered on 5.11.7 5.11.2 appears to have this module present. 3. Did it work previously in Fedora? If so, what kernel version did the issue *first* appear? Old kernels are available for download at https://koji.fedoraproject.org/koji/packageinfo?packageID=8 : 5.11.7 4. Can you reproduce this issue? If so, please provide the steps to reproduce the issue below: Boot with affected kernel or later (5.12 affected too) Try to access /sys/kernel/debug/ec/ -- does not exist 5. Does this problem occur with the latest Rawhide kernel? To install the Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by ``sudo dnf update --enablerepo=rawhide kernel``: Probably. See https://koji.fedoraproject.org/koji/rpminfo?rpmID=25660624 , which does not list ec_sys among the included modules 6. Are you running any modules that not shipped with directly Fedora's kernel?: N/A 7. Please attach the kernel logs. You can get the complete kernel log for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the issue occurred on a previous boot, use the journalctl ``-b`` flag.
This message is a reminder that Fedora 33 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '33'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 33 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This is less a bug report, and more of a plea/request to return ec_sys to the non-debug kernel build. This module is used for accessing the EC for things like fan control and battery charge behavior. One example is for the ISW project hosted here: https://github.com/YoyPa/isw
This is also the mechanism used to manually control the LEDs on ThinkPads. I tried to run with a debug kernel for a few months, but it's just too much of a pain. Current recommendation for ThinkPad users is to switch away from Fedora which isn't really what I want to do.
Hi All, Fedora Contributor and upstream kernel maintainer for kernel drivers under drivers/platform/x86 here, Jatwill's remark about the LEDs on the ThinkPad made me take a look at this bug. In both the https://github.com/YoyPa/isw and the "manually control the LEDs on ThinkPads" cases, the right thing to do is really to add the desired functionality to the kernel, where it belongs, rather then poking the hw directly from userspace, which really is a bad idea. Note that even without setting the write_support parameter to the ec_sys module (which presumably you are doing) the module is still actually writing to the hardware, since every "read" transaction consists of first writing the desired address to the hw. We have had some pretty bad experiences (fried motherboards) with similar "safe" read transactions on I2C busses, so in general I consider allowing things like this a bad idea. For the https://github.com/YoyPa/isw case it would be good if you (fedorauser konojo) could reach out to the author of that code and ask them to write a kernel driver instead. I would be happy to help the author with getting their code into the kernel, feel free to share my email: hdegoede with them. For the ThinkPad LEDs case, the thinkpad_acpi driver already supports setting all LEDs through the /sys/class/leds interface. If the LED which you want to set is currently not there, can you try building a kernel with CONFIG_THINKPAD_ACPI_UNSAFE_LEDS=y set ? I guess it might be good to replace that Kconfig option with a module option to give users some more flexibility in this. Note though that this currently is disabled by default for good reasons: config THINKPAD_ACPI_UNSAFE_LEDS bool "Allow control of important LEDs (unsafe)" depends on THINKPAD_ACPI help Overriding LED state on ThinkPads can mask important firmware alerts (like critical battery condition), or misled the user into damaging the hardware (undocking or ejecting the bay while buses are still active), etc. LED control on the ThinkPad is write-only (with very few exceptions on very ancient models), which makes it impossible to know beforehand if important information will be lost when one changes LED state. Users that know what they are doing can enable this option and the driver will allow control of every LED, including the ones on the dock stations. Never enable this option on a distribution kernel. Say N here, unless you are building a kernel for your own use, and need to control the important firmware LEDs. Jatwill, may I ask what your use case for controlling the LEDs is and which LED you are trying to control directly ? ### FWIW I've tracked down the original Fedora kernel commit making this change, which says: commit 4f9c73be153e98717a5b7c64c4c07e53a81f59aa Author: Peter Robinson <pbrobinson> Date: Tue Dec 29 11:34:28 2020 +0000 All: Only build ACPI_EC_DEBUGFS for debug kernels As per the Kconfig help ACPI_EC_DEBUGFS is for poking at the system's Embedded Controller via a /sys/kernel/debug interface and we really shouldn't encourage this for standard use as it's explicitly designed for debugging drivers/hardware. Signed-off-by: Peter Robinson <pbrobinson> I've added Peter to the Cc so that he can comment on this if he wants to. Given that I don't really see a reason to change the CONFIG_ACPI_EC_DEBUGFS setting in Fedora, I'm going to close this but we can discussing alternative solutions for the use-cases you want this for here. Regards, Hans
Missing word in my last sentence: "but we can discussing alternative solutions for the use-cases you want this for here" should be: "but we can _continue_ discussing alternative solutions for the use-cases you want this for here"
Hi Hans, First, I really appreciate the time and research you've put into your response. I was leveraging https://www.reddit.com/r/thinkpad/comments/7n8eyu/thinkpad_led_control_under_gnulinux/ to either turn off or flash the red LED on the 'i' in 'ThinkPad' on the back of my laptop. I have it wired up in a custom build of Mixxx which flashes the LED when the current song has less than 30 seconds left. It's used as a warning that I can see across the dance floor that I need to get back to my laptop. I appreciate how niche it is, but it has improved my quality of life when DJing and I'm certain if there was a standard (and safe) way to fiddle with that light others would come up with use cases too. I have previously tried setting 'brightness' of the various tpacpi:: in /sys/class/leds, as well as /proc/acpi/ibm/ (specifically various <led>s in /proc/acpi/ibm/led), but only ec_sys has worked. I also tried tracking debug kernels, but would forget to boot into them until it was too late. I'll try building a kernel with THINKPAD_ACPI_UNSAFE_LEDS enabled and reply.
Hi Hans, I've built 5.15.4-200 with THINKPAD_ACPI_UNSAFE_LEDS=y and with that set, I can do: echo >/proc/acpi/ibm/leds "10 blink" to blink the red LED on the outside of my laptop (and "10 off" / "10 on" to turn it on / off). If this were somehow a module, it would solve my needs.
Ok, so you are interested in led number 10, and so is the author of: https://www.reddit.com/r/thinkpad/comments/7n8eyu/thinkpad_led_control_under_gnulinux/ Currently that has a name of "tpacpi::unknown_led2" in the upstream thinkpad_acpi.c driver, but I also just tested this on 2 ThinkPads which I have here and indeed this controls the lid-logo-led. I don't see any reason why allowing a user to control that LED would be unsafe, so I have submitted a patch upstream to change LED 10's name to "tpacpi::led_logo_led" and to add it to the mask of LEDs which are considered safe to allow user-control over: https://lore.kernel.org/platform-driver-x86/20211123210524.266705-2-hdegoede@redhat.com/ Unless there are objections from other developers about this change I plan to merge this as a fix for the 5.16 kernel, so that when Fedora moves over to the 5.16 kernel in a couple of months from now you should be able to do the echo to /proc/acpi/ibm/leds without needing to build your own kernel. Note you should also have a "/sys/class/leds/tpacpi::unknown_led2" dir now where you control the LED through the brightness attribute. You can also do as root: modprobe ledtrig-timer echo timer > /sys/class/leds/tpacpi::unknown_led2/trigger to make it blink. Then you will also get delay_on / delay_off attributes under: /sys/class/leds/tpacpi::unknown_led2/ which allow changing the blinking frequency / duty-cycle. Note changing these will fallback from using the embedded-controller's blink function to using timers on the CPU, since the EC's blink function is hardcoded to 1Hz 50% duty-cycle. Once my patch is merged the name of the dir will change to /sys/class/leds/tpacpi::lid_logo_led, I believe that this change is ok, since at the moment this dir is not present at all unless someone builds their own kernel setting the THINKPAD_ACPI_UNSAFE_LEDS option.
Hi Hans, That's excellent! Looking forward to tracking that patch until it comes to me via dnf one day. Thanks again!
Hans, I am happy to bring that patch into 5.15 for Fedora once it is queued for linus.
I'd not seen this bug before - but in the spirit of the title I wanted to put forth a reason why having access to the ec_sys is good. I've had a number of customer cases recently debugging why thermal firmware control isn't working as expected in their specific scenario and we've not been able to reproduce. It's really useful to the EC team to get the dump of the EC registers...which you can get using ec_sys. On Ubuntu systems it's great - I just tell them to modprobe and then hexdump ec/io. On Fedora we dance around getting a module/kernel built. I've been meaning to raise a bug requesting to add ec_sys back in by default and hadn't don it. Let me know if that would be best...or if I can just hijack this one and get an ack or nack as appropriate :) I have no idea if it would help other vendors...but at least for Lenovo systems it has real value. Thanks! Mark
I have no strong-objections against re-introducing the ec_sys module, with one caveat it really needs to get kernel-lockdown support first. Which would mean that users need to turn off secure boot before it can be used, but direct EC poking really goes against the lockdown principles which come with secure boot. I'm afraid that this would make it some what less useful for debugging. Although arguable we could still allow read-only mode when the system is locked-down. As for the "dangerous" aspect of this, the mentioned I2C problem is somewhat of a special case and the ec_sys module will only bind to an EC which has been declared as following the ACPI spec in the DSDT, so read-only mode really should be read-only mode, so I guess this is ok-ish.
(In reply to Mark Pearson from comment #12) > I'd not seen this bug before - but in the spirit of the title I wanted to > put forth a reason why having access to the ec_sys is good. > > I've had a number of customer cases recently debugging why thermal firmware > control isn't working as expected in their specific scenario and we've not > been able to reproduce. It's really useful to the EC team to get the dump of > the EC registers...which you can get using ec_sys. On Ubuntu systems it's > great - I just tell them to modprobe and then hexdump ec/io. On Fedora we > dance around getting a module/kernel built. > > I've been meaning to raise a bug requesting to add ec_sys back in by default > and hadn't don it. Let me know if that would be best...or if I can just > hijack this one and get an ack or nack as appropriate :) I have no idea if > it would help other vendors...but at least for Lenovo systems it has real > value. There's potential security implications of having it enabled, the reason I disabled it is because there's the potential of security compromise to do things to firmware that are not easily detectable and could potentially either brick the machine or worse, have something that remains even post reinstall of a machine and depending on the firmware that could even potentially enable further compromise. So frankly I'm not overly excited about it being re-enabled.
Peter, I completely understand your worries and I'm not overly enthusiastic about this myself either. Still I wonder what your opinion on this is when ec_sys is tied to the kernel-lockdown mechanism and thus to secureboot ? Without lockdown there are already many ways the root user can poke at the hw, so I think that allowing ec_sys when not locked down is somewhat ok. Regardless of this discussion, ec_sys really should get lockdown checks added in the upstream kernel. Mark would ec_sys still be useful for Lenovo if it requires turning secure-boot off? At least for write support, I believe that disallowing read-support should only be done when lockdown is set to also protect confidentiality and secureboot only sets it to integrity.
For reference this is enabled in the debug kernel, on the occasional case you need to get a user to debug a thermal issue why can't they install the debug kernel?
I was just taking a looks at the ec_sys code and it is using debugfs and even without the write-parameters it will still declare a file_operations.write callback, the parameter only controls the mode of the file, but root can override that. Since there is a debugfs write callback, the secure-boot lockdown currently already disallows using the ec_sys module when secure-boot is on. This is actually something which we can fix, so that the module will work in read-only mode with secure-boot on by dropping the write callback when the write_support module parameter is not passed. As for the discussion about enabling the Kconfig option, I forgot that we not only do debug kernel-builds in rawhide, but that we also do debug-builds in separate -debug subpackages fore regular builds too. Mark, you can just ask users to run: sudo dnf install kernel-debug And then boot into the debug kernel and then the will have the ec_sys module available; and once my fixes for that module land, they will even be able to use it with secureboot on. Mark, I believe that this should satisfy your needs, while keeping the option off in the default kernel builds?
Correction, I was reading the code wrong, ec_sys should already work with secureboot enabled.
> Mark, you can just ask users to run: > sudo dnf install kernel-debug Absolutely - I didn't realise it was in the debug build. Just tested and it works perfectly and that's a great solution I think. I withdraw the request :) Thanks for the pointers and help Mark
(In reply to Hans de Goede from comment #18) > Correction, I was reading the code wrong, ec_sys should already work with > secureboot enabled. Hans, Thank you for taking the time to respond so insightfully in this thread. I did have to disable secure boot to use ec_sys for fan control (obvsiously, for write access). If that's the primary concern for continuing exclusion of this module in the kernel, as I think you've already stated, the kernel lock-down is in place. Since filing this report, I had tried using the debug kernel, but that didn't work out so well with akmods. So, I've been building my own kernel with CONFIG_ACPI_EC_DEBUGFS=m, and it's been working great. I, personally, can continue with this ~solution, but I'll be interested to see if/when ec_sys is reintegrated to the release build. Thanks again, Joshua
Joshua, Ideally the author of: https://github.com/YoyPa/isw would use their knowledge of the firmware interfaces used on those devices to write a proper, standard kernel driver which exports the functionality using standard kernel interfaces. This is how this is done for every other laptop-vendor's firmware interfaces, which is why we have Asus, Dell, HP, Lenovo, Xiaomi, etc. laptop drivers in the kernel already. Since you are a user of this I was hoping that you can ask the author of this to write a proper kernel driver for this. I would be happy to help the author with getting their code into the kernel, feel free to share my email: hdegoede with them. Regards, Hans
Yes. Sorry I forgot to acknowledge that suggestion above. I'll be happy to contact the /YoyPa/ author and suggest a proper kernel driver. I appreciate your willingness to help with this!
The author of isw /YoyPa/ has been inactive for more than 2 year :( Only you can help us now Hans. Could you please try to integrate ec_sys with the release build. I badly miss fedora. I had to move to ubuntu because the laptop was overheating while using fedora. PLease integrate the ec_sys with the next kernel release... Many of us are facing this issue :( https://github.com/YoyPa/isw Thankyou!
Yeah, I did reach out to YoyPa ( https://github.com/YoyPa/isw/issues/226 ), but have yet to get a response. If the isw project has been abandoned or, possibly, put on hiatus, that would be unfortunate.
If your system is one of the Lenovo Linux certified models and is overheating please let me know (maybe off thread). My guidance generally would be that going in and directly modifying the EC manually is "not a good thing". I'm sure there are a bunch of caveats but if it's a Lenovo platform I'd love to know what the root cause is - and find better solutions so it's not needed. We use the ec_sys module to dump registers for debugging thermal problems - but ideally it shouldn't be a requirement for day-to-day operation.
(In reply to Justin M. Forbes from comment #11) > Hans, I am happy to bring that patch into 5.15 for Fedora once it is queued > for linus. FYI the patch for allowing control of the LID logo dot LED has landed in Torvald's master branch now.
(In reply to Shiva from comment #23) > The author of isw /YoyPa/ has been inactive for more than 2 year :( > Only you can help us now Hans. Could you please try to integrate ec_sys with > the release build. I'm afraid that re-enabling ec_sys really is not a good solution here. And I don't have the time nor the hardware to write a proper kernel driver myself. I notice that there are a bunch of pull-reqs on the upstream https://github.com/YoyPa/isw repo. Perhaps you can try reaching out to the authors of those pull-reqs and see if any of them is interested in writing a proper kernel driver for these MSI laptops ?
(In reply to Hans de Goede from comment #26) > (In reply to Justin M. Forbes from comment #11) > > Hans, I am happy to bring that patch into 5.15 for Fedora once it is queued > > for linus. > > FYI the patch for allowing control of the LID logo dot LED has landed in > Torvald's master branch now. Thanks, I pulled it into the fedora-5.15 branch, it should make the 5.15.9 builds today.