Bug 1253523

Summary: Boot Hang (Apparent systemd Problem) on Dell Inspiron 1012
Product: [Fedora] Fedora Reporter: Daniel Kian Mc Kiernan <Mc_Kiernan>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 29CC: ankit.rastogi, avezaatb, epatarroyo, gadde, gansalmon, huzaifashaikh0012, icj, itamar, ivanmatus683, jan.kratochvil, j.novak, johannbg, jonathan, jsynacek, kernel-maint, labbott, lnykryn, madhu.chinakonda, mchehab, Mc_Kiernan, msekleta, paul.mckimmy, raines, s, systemd-maint, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-09 17:46:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Daniel Kian Mc Kiernan 2015-08-14 01:45:12 UTC
Description of problem: 

On a Dell Inspiron Mini 1012, the boot process hangs during some systemd stuff.


Version-Release number of selected component (if applicable):

kernel-4.1.3-201.fc22.x86_64
kernel-4.1.4-200.fc22.x86_64


How reproducible: 

Always.


Steps to Reproduce:

1. Install Fedora Core 22 w/ Kernel 4.1.3-201 or 4.1.4-200 on Dell Inspiron 1012.
2. Boot.


Actual results:

System hangs during boot after printing three or all of these lines; 

[  OK  ] Created slice system-systemd\x2drfkill.slice.
         Starting Load/Save RF Kill Switch Status of rfkill0...
         Starting Load/Save RF Kill Switch Status of rfkill1...
         Starting Load/Save Screen Backlight Brightness of leds:dell::kbd_backlight...
[   28.865450] ath: phy0: Enable LNA combining
[   28.865450] ath: phy0: ASPM enabled: 0x42
[   28.942666] ieee80211 phy0: Atheros AR9285 Rev:2 mem=0xffffc90000400000, irq=17
[   29.075602] snd_hda_codec_realtek hdaudioC0D0: autoconfig for ALC272 line_outs=1 (0x14/0x0/0x0/0x0/0x0) type: speaker


Expected results:

Normal boot-up.


Additional info: 

No such problem discerned with kernel-4.0.8-300.fc22.x86_64 .

Comment 1 Daniel Kian Mc Kiernan 2015-08-25 05:31:55 UTC
Problem persists with 

kernel-4.1.5-200.fc22.x86_64

Comment 2 Daniel Kian Mc Kiernan 2015-08-27 11:58:35 UTC
Problem persists with 

kernel-4.1.6-200.fc22.x86_64

Comment 3 Lukáš Nykrýn 2015-09-08 11:01:13 UTC
Can you please set systemd to debug mode and add logs from the boot using debug shell?

https://wiki.freedesktop.org/www/Software/systemd/Debugging/

Comment 4 Daniel Kian Mc Kiernan 2015-09-08 12:13:14 UTC
Thank you for your attention. 

I am willing but unable to use the debug shell.  Before rebooting, I entered the command to enable it (“systemctl enable debug-shell.service”) but my system simply locked-up and the previously noted stage, and no shell is available. (In fact, I have to turn-off the machine to restart it.) 

Is there any other way for me to save logs of the process, such that they will not be over-written by a successful boot (with the old kernel)?

Comment 5 Daniel Kian Mc Kiernan 2015-09-12 08:19:50 UTC
Problem persists with 

kernel-4.1.6-201.fc22.x86_64 and the latest systemd stuff (219-23). 

The messages being sent to screen are a bit different, and especially in different order, but seem to correspond to about the same collection of events.

Comment 6 Daniel Kian Mc Kiernan 2015-09-21 21:49:12 UTC
Problem persists with the latest systemd stuff (219-24).

Comment 7 Daniel Kian Mc Kiernan 2015-09-24 20:44:46 UTC
Problem persists with 

kernel-4.1.7-200.fc22.x86_64

It appears that I need to begin looking for a different distro, as this problem is on track to be yet another WONTFIX.

Comment 8 Daniel Kian Mc Kiernan 2015-10-08 11:11:21 UTC
Problem persists with 

kernel-4.1.8-200.fc22.x86_64

Comment 9 Daniel Kian Mc Kiernan 2015-10-11 01:36:44 UTC
Problem persists with 

kernel-4.1.10-200.fc22.x86_64

Comment 10 Daniel Kian Mc Kiernan 2015-10-17 04:41:00 UTC
Problem persists with 

kernel-4.2.3-200.fc22.x86_64

Comment 11 Daniel Kian Mc Kiernan 2015-10-30 10:05:33 UTC
Problem persists with the latest systemd stuff (219-25).

Comment 12 Iván Jiménez 2015-11-03 04:28:31 UTC
This bug is caused by the file /var/lib/systemd/backlight/platform-dell-laptop:leds:dell::kbd_backlight created by (I guess) systemd-backlight@.service, because I can boot with kernel-4.2.3-200.fc22.x86_64 after I remove that file.

As that file is re-created at shutdown, I have to mask the service systemd-backlight@leds\:dell\:\:kbd_backlight.service with:

sudo systemctl mask systemd-backlight@leds\:dell\:\:kbd_backlight.service 

so kernel-4.2.3-200.fc22.x86_64 doesn't hang at boot.

As suggested by man systemd-backlight@.service, booting with the kernel command line parameter:

       systemd.restore_state=0 

also avoids the hang.

I think that file shouldn't be created in this laptop because it doesn't have a keyboard backlight or any LED in the keyboard.

Comment 13 Daniel Kian Mc Kiernan 2015-11-03 07:20:32 UTC
I confirm that the systemctl command present by Mr Jiménez ends the problem for me.  I thank him for this information.

Comment 14 Iván Jiménez 2015-11-04 15:55:45 UTC
This bug is still present in Fedora 23.

Comment 15 Daniel Kian Mc Kiernan 2015-11-06 06:56:35 UTC
I have filed an issue report at 

    https://github.com/systemd/systemd/issues/1792

Comment 16 Jan Synacek 2015-11-10 09:22:51 UTC
As suggested in the upstream bug report, I'm switching this bug to kernel.

Comment 17 Daniel Kian Mc Kiernan 2015-11-10 10:01:04 UTC
I have filed a bug report at 

    https://bugzilla.kernel.org/show_bug.cgi?id=107651

Comment 18 Iván Jiménez 2016-06-22 23:52:12 UTC
It still happens in Fedora 24 with kernel 4.5.5-300.fc24.x86_64.

Comment 19 Daniel Kian Mc Kiernan 2016-06-23 04:31:30 UTC
Thanks for the heads-up, Mr Jiménez. I have made a note of that datum at the bug report at kernel,org. 

But the developers at kernel.org are simply silent.

Comment 20 Syam Gadde 2016-06-24 01:36:42 UTC
Just wanted to report that I have been looking for a fix for this for nearly a year, and this fix works great on my system (also a Dell Mini 1012).  The latest kernel package I could rely upon was 4.0.8, and so I took care to remove intermediate versions as I tried out newer kernel versions so 4.0.4 wouldn't disappear.  I removed the file and added the option to the kernel command line, and it now boots into 4.4.10 on Fedora 22.

Very happy I happened on this!

Comment 21 Laura Abbott 2016-09-23 19:13:55 UTC
*********** MASS BUG UPDATE **************
 
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 24 kernel bugs.
 
Fedora 24 has now been rebased to 4.7.4-200.fc24.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you have moved on to Fedora 25, and are still experiencing this issue, please change the version to Fedora 25.
 
If you experience different issues, please open a new bug report for those.

Comment 22 Daniel Kian Mc Kiernan 2016-09-26 05:06:13 UTC
This bug persists in kernel-4.4.14-200.fc22.x86_64. 

No developer has been working on this bug, and with live in a world without magic. 

A need has no meaning except in relation to an objective; I suggest that the needinfo flag not again be set unless some developer adopts the objective of fixing this bug.

Comment 23 Daniel Kian Mc Kiernan 2016-09-26 05:24:33 UTC
This bug persists in kernel-4.7.4-200.fc24.x86_64. 

No developer has been working on this bug, and we live in a world without magic. 

A need has no meaning except in relation to an objective; I suggest that the needinfo flag not again be set unless some developer adopts the objective of fixing this bug.

Comment 24 Jóhann B. Guðmundsson 2016-09-26 08:21:04 UTC
The needinfo here was just to determine if this was still a problem on a more recent kernel. 

This [1] is probably what introduced all this broken dell backlight behaviour upstream and afaikt the only one in the distribution that might own the necessary hardware and possesses the familiarity to be able to poke this code would be Hans de Goede but in the end of the day the upstream report is being ignored which arguably means that this bug should just be closed cantfix/wontfix instead of keeping the report open thus the reporters expectation in the process that this will ever get fixed. 

1. https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=6cff8d60aa0aba5583ecda09984dbcb2f24cc28d

Comment 25 Daniel Kian Mc Kiernan 2016-09-26 21:10:06 UTC
The reporter doesn't have an expectation that the bug will be fixed.  The reporter concluded long ago that, whatever their original intention, bugzillas have become devices primarily for seeming to pursue bug-fixing without actually doing so.  The needinfo here was part of a ritual of pretense. 

Even in cases where I have asked the developers just to point me in the directions of relevant packages, so that I could investigate the bug without investing in a wider and redundant expertise, there has been no response.  In one case, someone not responsible for a package got the developer tio communicate by proposing himself to effect a solution, and the developer's response was that this was not the solution that he wanted and that he would not effect the solution that he wanted for some time to come and would _quit_ if the propose solution were even used as a stop-gap in the interim. 

The very existence of “wontfix” operationalizes as a bug, because it serves to facilitate the obscuring of bugs, to make them seem not to exists.  The best way to support an sincere intention to fix bugs would be to keep a spotlight on unfixed bugs, such that bugs that could no longer practicably be fixed would be like ugly scars, and efforts would be made to avoid accumulating still more.

Comment 26 Ankit Rastogi 2016-10-22 15:46:28 UTC
hi, I'll add that this problem also exists on the Dell Studio 14Z (2008) on 4.7.7-200.fc24.x86_64 and 4.7.4-200.fc24.x86_64, across multiple distributions including both F24 and Ubuntu 16.04-16.10. The workaround posted here by Ivan Jimenez worked without issue in both distributions.

Comment 27 Paul Raines 2016-11-14 20:04:54 UTC
yes, just installed F24 today on my Studio 14Z after first trying Ubuntu 16.  Both would install fine, boot the first time fine and then never boot again even in single user mode hanging in the boot up process. Finally figured out how to see the kernel log messages on boot (why is that so hard to find out how to do? 'rhgb quiet' should at least be removed from the default rescue grub entry) and searched the rfkill thing and found this.    Ivan Jimenez's solution works great for me.

Comment 28 Justin M. Forbes 2017-04-11 14:44:19 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 24 kernel bugs.

Fedora 25 has now been rebased to 4.10.9-100.fc24.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 26, and are still experiencing this issue, please change the version to Fedora 26.

If you experience different issues, please open a new bug report for those.

Comment 29 Daniel Kian Mc Kiernan 2017-04-12 03:39:21 UTC
On the hardware with which I originally reported the problem, now using kernel-4.10.8-200.fc25.x86_64 (the most recent version actually offered), I entered 

sudo systemctl unmask systemd-backlight@leds\:dell\:\:kbd_backlight.service 

and rebooted.  The computer did not hang during the rebooting. 

I hope to see reports from other users before this bug is declared vanished.

Comment 30 Iván Jiménez 2017-04-12 18:07:41 UTC
(In reply to Daniel Kian Mc Kiernan from comment #29)
> On the hardware with which I originally reported the problem, now using
> kernel-4.10.8-200.fc25.x86_64 (the most recent version actually offered), I
> entered 
> 
> sudo systemctl unmask systemd-backlight@leds\:dell\:\:kbd_backlight.service 
> 
> and rebooted.  The computer did not hang during the rebooting. 
> 
> I hope to see reports from other users before this bug is declared vanished.

On a Dell Inspiron Mini 1012 using 4.10.8-200.fc25.x86_64, I did the same and rebooted twice; in the second boot it still hangs.

Comment 31 Daniel Kian Mc Kiernan 2017-04-12 20:56:32 UTC
As with Mr Jiménez, I find that the bug is still manifest with the second and subsequent boot-loading.

Comment 32 huzaifashaikh0012 2017-06-01 21:26:29 UTC
Dell inspiron mini 1018
Linux kernel 4.10.0-21-generic
Xubuntu 17.04
Removing the file /var/lib/systemd/backlight/platform-dell-laptop:leds:dell::kbd_backlight 
Worked for me but on next reboot it again hangs
The systemctl unmask command did the same (just worked for one boot)

Comment 33 huzaifashaikh0012 2017-06-01 21:27:45 UTC
(In reply to huzaifashaikh0012 from comment #32)
> Dell inspiron mini 1018
> Linux kernel 4.10.0-21-generic
> Xubuntu 17.04
> Removing the file
> /var/lib/systemd/backlight/platform-dell-laptop:leds:dell::kbd_backlight 
> Worked for me but on next reboot it again hangs
> The systemctl unmask command did the same (just worked for one boot)

I also tried making a script to run at boot that removes the file on boot but i guess the boot process doesn't reach that point before the buggy part

Comment 34 huzaifashaikh0012 2017-06-02 06:46:23 UTC
(In reply to huzaifashaikh0012 from comment #33)
> (In reply to huzaifashaikh0012 from comment #32)
> > Dell inspiron mini 1018
> > Linux kernel 4.10.0-21-generic
> > Xubuntu 17.04
> > Removing the file
> > /var/lib/systemd/backlight/platform-dell-laptop:leds:dell::kbd_backlight 
> > Worked for me but on next reboot it again hangs
> > The systemctl unmask command did the same (just worked for one boot)
> 
> I also tried making a script to run at boot that removes the file on boot
> but i guess the boot process doesn't reach that point before the buggy part

Update:earlier i executed systemctl mask command in recovery mode...
This time i executed the command after booting up(by removing the backlight file)
And it's working now. 
No hang after multiple reboots and shutdown
So i guess the issue can be closed now...

Comment 35 Daniel Kian Mc Kiernan 2017-06-02 06:58:33 UTC
(In reply to huzaifashaikh0012 from comment #34) 

The issue here is the bug initially reported, not your failure to get some script to work.  Executing the systemctl command identified by  Iván Jiménez is a work-around for the bug; not a fix of the bug. 

Do not declare an issue closed that is still quite unresolved.

Comment 36 Bas 2017-06-20 09:54:35 UTC
(In reply to Iván Jiménez from comment #12)
> This bug is caused by the file
> /var/lib/systemd/backlight/platform-dell-laptop:leds:dell::kbd_backlight
> created by (I guess) systemd-backlight@.service, because I can boot with
> kernel-4.2.3-200.fc22.x86_64 after I remove that file.
> 
> As that file is re-created at shutdown, I have to mask the service
> systemd-backlight@leds\:dell\:\:kbd_backlight.service with:
> 
> sudo systemctl mask systemd-backlight@leds\:dell\:\:kbd_backlight.service 
> 
> so kernel-4.2.3-200.fc22.x86_64 doesn't hang at boot.
> 
> As suggested by man systemd-backlight@.service, booting with the kernel
> command line parameter:
> 
>        systemd.restore_state=0 
> 
> also avoids the hang.
> 
> I think that file shouldn't be created in this laptop because it doesn't
> have a keyboard backlight or any LED in the keyboard.

Thank you Mr. Jimenez, executing your command resolved my boot issues with the Dell XPS 15 9560 as well.

Comment 37 imb 2017-09-05 12:26:13 UTC
The bug exists in last release of UBUNTU 16.04 when installed in inspiron mini 1012. Solved by the suggestion of Ivan Jimenez.

Comment 38 Daniel Kian Mc Kiernan 2017-09-05 22:37:05 UTC
This forum is for bug reports concerning the Fedora distribution; the Ubuntu folk file their reports elsewhere. As this bug is in the kernel across distributions, you may also report this at 

    https://bugzilla.kernel.org/show_bug.cgi?id=107651 

Be sure to identify which kernel version you are using, and not simply which version of the Ubuntu distribution. 

Also, with administrators anxious to declare matters at an end, please do not declare anything to be solved when instead a work-around has been produced. Things are solved when there is no problem around which to work.

Comment 39 Fedora End Of Life 2017-11-16 18:41:58 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 40 Daniel Kian Mc Kiernan 2017-11-16 23:05:28 UTC
Please don't include perfectly insincere apologies.  It is not that developers were unable to fix this bug (which has existed since Core 22 and continues into Core 27); it is that developers are uninterested in fixing this bug.

Comment 41 Laura Abbott 2018-02-20 19:56:12 UTC
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  As kernel maintainers, we try to keep up with bugzilla but due the rate at which the upstream kernel project moves, bugs may be fixed without any indication to us. Due to this, we are doing a mass bug update across all of the Fedora 27 kernel bugs.
 
Fedora 27 has now been rebased to 4.15.3-300.f27.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you experience different issues, please open a new bug report for those.

Comment 42 Daniel Kian Mc Kiernan 2018-02-21 03:18:13 UTC
This bug is still biting, and still not being worked, years after it was first reported. 

The number of man-hours spent re-breaking our computers to check on that point, and then again implementing the work-around is greatly in excess of the number of man-hours that would be required to patch the code that introduced the bug. 

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=6cff8d60aa0aba5583ecda09984dbcb2f24cc28d

Obviously, at some point, we will just be worn-down, and stop conducting the test, and you can then pretend that the bug no longer bites.

Comment 43 Paul McKimmy 2018-06-18 21:44:04 UTC
Ivan's solution above worked for me as well:
Ubuntu 18.04 booted fine after initial installation on Dell Studio 1440.  After udpdates however, it froze after GRUB.Solution:

Removing file /var/lib/systemd/backlight/platform-dell-laptop:leds:dell::kbd_backlight 

and masking the service systemd-backlight@leds\:dell\:\:kbd_backlight.service with:

sudo systemctl mask systemd-backlight@leds\:dell\:\:kbd_backlight.service 

Posting so that others might search this up a bit easier.  Thanks

Comment 44 Daniel Kian Mc Kiernan 2018-06-18 23:05:26 UTC
(In reply to Paul McKimmy from comment #43)
> Ivan's solution above worked for me as well:
> Ubυntυ 18.04 booted fine after initial installation on Dell Studio 1440. 
<snip>
> Posting so that others might search this up a bit easier.  Thanks

Please, this is _not_ an Ubυntυ support group.  The Ubυntυ support groups are of so little help because their signal-to-noise ratios are so low.  If you attract Ubυntυ users to other communities, you will similarly reduce their signal-to-noise ratios, making them less usable for the legitimate participants.

Comment 45 Jirka Novak 2018-07-20 21:15:18 UTC
Hello,

I have same issue on Dell Precision 3530 with FC28 with kernels 4.17.3 and 4.17.6. There is small difference between 4.17.3 and 4.17.6. With .3 there is about 10% probability that system will boot up, with .6 there is no way how to boot system up.
When I masked systemd-backlight@leds\:dell\:\:kbd_backlight.service, issue is resolved.

I'm open to provide more troubleshooting if requested.

Comment 46 Daniel Kian Mc Kiernan 2018-07-21 00:33:16 UTC
Six cores later….

Comment 47 Daniel Kian Mc Kiernan 2018-07-21 00:39:30 UTC
(In reply to Jirka Novak from comment #45)
> 
> I'm open to provide more troubleshooting if requested.

Please also report the bug at 

  https://bugzilla.kernel.org/show_bug.cgi?id=107651

The Fedora developers will simply continue to point upstream when they don't ignore us altogether.  Upstream, the Dell driver team at kernel.org is more likely to act if there are more users reporting there.

Comment 48 Daniel Kian Mc Kiernan 2018-07-30 06:18:26 UTC
At the bug report at kernel.org Jirka Novak comments that he added messages to dell-laptop.c and to dell-smbios-base.c, to narrow-down the source of the problem. 

  https://bugzilla.kernel.org/show_bug.cgi?id=107651#c23 

He says that he learned that a specific SMBIOS call does not finish, and that a lock is therefore not released. (Mr Novak provided details, including a log.) 

Mr Novak requests help from a more skilled developer.

Comment 49 Justin M. Forbes 2019-01-29 16:24:37 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 28 kernel bugs.

Fedora 28 has now been rebased to 4.20.5-100.fc28.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 29, and are still experiencing this issue, please change the version to Fedora 29.

If you experience different issues, please open a new bug report for those.

Comment 50 Jirka Novak 2019-02-09 09:46:08 UTC
Hi,

  I tried it with 4.20.6-100.fc28.x86_64 and it looks OK now.

Best regards,

Jirka Novak