Bug 1357886

+++ This bug was initially created as a clone of Bug #1206912 +++

Description of problem:

After updating to latest F21 stable updates, hibernate stops working, because the information about the resume device is no longer processed correctly.

Version-Release number of selected component (if applicable):

systemd-216-21.fc21.x86_64
dracut-038-33.git20141216.fc21.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Put machine to hibernate
2. power up machine
3.

Actual results:
machine does not hibernate, it boots normally

Expected results:
machine should detect valid hibernation on swap device and resume from that

Additional info:

The latest dracut update takes a new feature of systemd into use:

/usr/lib/dracut/modules.d/95resume/module-setup.sh:

    if [[ $hostonly_cmdline == "yes" ]]; then
        cmdline  >> "${initdir}/etc/cmdline.d/95resume.conf"
        echo  >> "${initdir}/etc/cmdline.d/95resume.conf"
    fi

    # if systemd is included and has the hibernate-resume tool, use it and nothing else
    if dracut_module_included "systemd" && [[ -x $systemdutildir/systemd-hibernate-resume ]]; then
        inst_multiple -o \
                      $systemdutildir/system-generators/systemd-hibernate-resume-generator \
                      $systemdsystemunitdir/systemd-hibernate-resume@.service \
                      $systemdutildir/systemd-hibernate-resume
        return 0
    fi

On my system that generates this information into the initrd:

# lsinitrd | fgrep resume
resume
-rw-r--r--   1 root     root           40 Mar 29 16:17 etc/cmdline.d/95resume.conf
-rwxr-xr-x   1 root     root        28704 Mar 29 16:17 usr/lib/systemd/systemd-hibernate-resume
-rwxr-xr-x   1 root     root        37048 Mar 29 16:17 usr/lib/systemd/system-generators/systemd-hibernate-resume-generator
-rw-r--r--   1 root     root          635 Mar 13 10:44 usr/lib/systemd/system/systemd-hibernate-resume@.service

# lsinitrd -f etc/cmdline.d/95resume.conf
 resume=/dev/mapper/VolGroup00-LogVol02

But unfortunately systemd-hibernate-resume-generator(?) ignores the contents of etc/cmdline.d/95resume.conf and *ONLY* looks for "resume=" on the kernel command line. By adding "resume=/dev/mapper/VolGroup00-LogVol02" to /boot/grub2/grub.cfg I got hibernation working again.

Unfortunately I do not know if this needs to be fixed in dracut (add resume= to the kernel command line?) or systemd (check for etc/cmdline.d/ contents?), so I file this bug to dracut first.

--- Additional comment from Jonathan S on 2015-03-31 11:50:48 EDT ---

Yes, I have this some problem too.

Stefan's analysis of the problem seems to be correct and the adding of 'resume=....' to the kernel command line seems to be a valid workaround.

--- Additional comment from Roy Rankin on 2015-04-08 18:14:14 EDT ---

I also have this problem on the 3.19 kernels. By adding the output of lsinitrd -f etc/cmdline.d/95resume.conf which for me was " resume=UUID=88974181-39f3-44f6-af13-dab8e5b6d4fc" as above restored hibernate function with 3.19.3 kernel.
Note that without the workaround hibernate is still working with the 3.18.9 kernel.

--- Additional comment from Stefan Becker on 2015-04-08 22:08:02 EDT ---

(In reply to Roy Rankin from comment #2)
> Note that without the workaround hibernate is still working with the 3.18.9
> kernel.

I assume that you have not regenerated the initrd for the 3.18.9 kernel with latest dracut. Updating to a new kernel does not modify older initrds on /boot.

--- Additional comment from Adam Pribyl on 2015-04-09 14:39:43 EDT ---

After update of dracut from -32 to -33 I can not resume from hibernation anymore - system restarts ignoring the hibernation image. Downgrading the dracut and regenerating the initramfs solves the problem.

--- Additional comment from Harald Hoyer on 2015-04-23 08:52:45 EDT ---

(In reply to Adam Pribyl from comment #4)
> After update of dracut from -32 to -33 I can not resume from hibernation
> anymore - system restarts ignoring the hibernation image. Downgrading the
> dracut and regenerating the initramfs solves the problem.

even with "resume=..." on the kernel command line?

--- Additional comment from Adam Pribyl on 2015-04-24 03:50:33 EDT ---

I've added the resume to /etc/default/grub2 and upgraded the dracut and kernel to latest version. After reboot I was dropped to dracut rescue shell, the root file system had many corruptions (this could be just an accident), the newly generated entry in grub2.conf had no resume at kernel command line. After fixing the filesystem I regenerated grub2.conf manually, regenerated the initramfs and now I can again hibernate and resume again... but is this really way it _should_ work?!

--- Additional comment from Harald Hoyer on 2015-04-24 05:19:21 EDT ---

(In reply to Adam Pribyl from comment #6)
> I've added the resume to /etc/default/grub2 and upgraded the dracut and
> kernel to latest version. After reboot I was dropped to dracut rescue shell,
> the root file system had many corruptions (this could be just an accident),
> the newly generated entry in grub2.conf had no resume at kernel command
> line. After fixing the filesystem I regenerated grub2.conf manually,
> regenerated the initramfs and now I can again hibernate and resume again...
> but is this really way it _should_ work?!

"resume=<swapdev>" really should be added by anaconda on installation and corrected, if you change your swap device.

hibernation only works reliably, if you specify the device.

dracut can help you with the kernel command line:

On my machine:
$ dracut --print-cmdline 
 resume=UUID=03ea4b3f-94fd-40c8-81e8-802734082be8 root=UUID=d5e53483-30bc-4119-8c0c-841d47a2f76b rootfstype=btrfs rootflags=rw,noatime,compress=lzo,ssd,discard,space_cache,autodefrag,inode_cache,subvol=root:foo:hoyer.harald.OS:x86_64

--- Additional comment from Adam Pribyl on 2015-04-24 08:18:55 EDT ---

Up to -32 version of dracut I never used "resume=anything" intentionaly anywhere. It was "magicaly" present in initramfs:etc/cmdline.d/95resume.conf, thats all. I luckily did not used anaconda for many years... I do not know where the "resume" should be added by anaconda.


The dracut changelog says
* Fri Mar 06 2015 Till Maas <opensource> - 038-33.git20141216
- Do not fsck on resum from hibernate
Resolves: rhbz#1174945

* Tue Dec 16 2014 Harald Hoyer <harald> 038-32.git20141216
- fixed systemd journal in the initramfs
Resolves: rhbz#1174733

I did not checked exact changes in the source but I do not understand how the above change is related to this behavior change of dracut.

Now dracut prints for me:
dracut --print-cmdline 
 resume=UUID=2b46147e-18f0-4366-ac99-172e7de7e502 root=UUID=6789faaa-68e7-40cf-8d19-fa55c1b49531 rootflags=rw,relatime,data=ordered rootfstype=ext4
but this is probably irrelevant as it works now with "resume=" in grub2.conf.

--- Additional comment from Stefan Becker on 2015-04-24 08:33:38 EDT ---

(In reply to Adam Pribyl from comment #8)
> The dracut changelog says
> * Fri Mar 06 2015 Till Maas <opensource> - 038-33.git20141216
> - Do not fsck on resum from hibernate
> Resolves: rhbz#1174945

Corresponding git change:

 http://pkgs.fedoraproject.org/cgit/dracut.git/diff/0032-do-not-fsck-on-resume-from-hibernation.patch?h=f21&id=0716a8025f162faafd0b4b716a4f393c9786ed6c


> I did not checked exact changes in the source but I do not understand how
> the above change is related to this behavior change of dracut.

I do not see anything about "fsck" in the above change. It only takes "systemd-hibernate-resume" into use, so IMHO the changelog is mis-leading.

systemd-hibernate-resume & friends *IGNORE* the contents of etc/cmdline.d/95resume.conf and only use "resume=" from the kernel command line, i.e. what is in grub.cfg. *THAT* is the issue here.

If you roll back to -32 and run dracut, systemd-hibernate-resume will no longer be used, and hibernate will magically starts working again.

Can please someone who knows the dracut internals explain how the information in etc/cmdline.d/95resume.conf is supposed to be used?

(a) Do the dracut scripts have to read that information manually or
(b) does dracut automagically append this information to /proc/cmdline? (is that even possible????)

If it is (a) then the bug is in systemd-hibernate-resume.

If it is (b) then the bug is in dracut, i.e. it doesn't feed this information early enough into /proc/cmdline for systemd-hibernate-resume to pick up.

--- Additional comment from Lorenzo Pistone on 2015-05-02 06:37:42 EDT ---

Any update on this? I suppose it would affect Fedora 22 as well, and I would guess it's a blocker there.

--- Additional comment from Till Maas on 2015-05-04 05:55:32 EDT ---

IMHO it is a bug/missing feature of systemd unless there is a way to modify /proc/cmdline.

The patch should fix the fsck bug, because without it there seemed to be a race condition causing the root filesystem to be checked before a resume was attempted, the details are in the referenced bug 1174945.

--- Additional comment from A. Folger on 2015-05-05 06:07:37 EDT ---

I am suffering from the same problem since some recent updates. Is a fix in the works? (The upside is that I am now checking what dracut does, but really, we shouldn't expect every user to learn this ;-))

--- Additional comment from Adam Pribyl on 2015-05-05 15:39:22 EDT ---

I do not think so. The reason is, this IMHO happens only to the systems, that were upgraded, and most probably not via Anaconda. As I understood, Anaconda should place the required resume= option to the kernel command line. 

The other thing is why systemd ignores the etc/cmdline.d/95resume.conf in initramfs... to my knowledge, nobody posted a bug against systemd.

--- Additional comment from Lorenzo Pistone on 2015-05-05 15:52:03 EDT ---

No, my installation was fresh, not an upgrade.

--- Additional comment from A. Folger on 2015-07-28 07:24:27 EDT ---

Just want to report that after a fresh install of Fedora 22, the problem persists. Can we make headway? This is really annoying.

--- Additional comment from Adam Pribyl on 2015-07-28 07:33:27 EDT ---

Lets switch that bug to systemd...

In the meantime we have to use workaround.

--- Additional comment from Vít Ondruch on 2015-08-18 13:38:12 EDT ---

There are similar bugs:

bug 1199708
bug 1207425
bug 1213131
bug 1206936

It would be nice if this issue get more attention ...

--- Additional comment from Vít Ondruch on 2015-08-18 13:38:47 EDT ---



--- Additional comment from Lorenzo Pistone on 2015-10-04 10:03:18 EDT ---

With the recent kernels 4.1.7-200 and 4.1.8-200 the method of appending manually the "resume=" argument to the kernel command line doesn't work anymore. Or maybe it's a systemd upgrade that broke it? I don't know, it is time already to fix this bug.

--- Additional comment from Lorenzo Pistone on 2015-10-04 10:12:37 EDT ---

I noticed that using the partition name (/dev/sdxy) instead of the partition UUID works. Needless to say, I double-checked that the partition UUID that I use is correct and so it is.

--- Additional comment from Chris Murphy on 2015-10-04 12:11:24 EDT ---

(In reply to Lorenzo Pistone from comment #20)
> I noticed that using the partition name (/dev/sdxy) instead of the partition
> UUID works. Needless to say, I double-checked that the partition UUID that I
> use is correct and so it is.

I don't expect it to work with partition UUID, which is a GPT concept. It needs to be set to the swap fs UUID as reported by blkid.

--- Additional comment from Lorenzo Pistone on 2015-10-04 12:13:26 EDT ---

Sorry, I meant the fs UUID, not PARTUUID.

--- Additional comment from Adam Pribyl on 2015-10-06 13:55:26 EDT ---

I do not understand how this bug could be ignored for such a long time, when it has to hit anybody using hibernation. I can confirm that with kernel-4.1.8-100.fc21.x86_64 the resume does not work.


dracut --print-cmdline 
resume=UUID=2b42147e-18f0-4366-ac99-172e7de7e602 root=UUID=4389fa2a-68e7-40cf-8d19-fa55c1b49531 rootflags=rw,relatime,data=ordered rootfstype=ext4

it is on the dracut commandline, but the resume terribly fails with various letters from systemd startup reamining on the screen. 

Does anybody know the workaround if the systemd-maint does not care?

--- Additional comment from Chris Murphy on 2015-10-06 14:45:22 EDT ---

As far as I'm aware, kernel maintainers have given up on hibernation (suspend to disk). It appears to all be very hardware specific. And while systemd has a hook in for the support still in the kernel, it ultimately depends on the kernel.

These days on both Macs and PCs hibernation is managed by firmware. I think about the best case is if we can have some hybrid sleep where the battery gets low enough and it causes the system to wake and poweroff instead of just dying in sleep or trying to hibernate, either of which can result in data loss or corruption.

--- Additional comment from Adam Pribyl on 2015-10-06 15:56:56 EDT ---

Chris, this is the first time I read this... would welcome any reference. 

I do use the hibernation on normal desktop PCs without batteries because it saves me whole desktop work and waits for me even when power is lost in meantime. Of course, this also saves more power and stops various LED blinking "effects".

--- Additional comment from Chris Murphy on 2015-10-06 16:25:59 EDT ---

https://bugzilla.redhat.com/show_bug.cgi?id=781749
https://bugzilla.redhat.com/show_bug.cgi?id=781749#c12

The basic gist there is that it's fundamentally dangerous and inherently broken, is unlikely to get fixed, and is being removed as much as possible from user space.

--- Additional comment from Till Maas on 2015-10-06 16:33:49 EDT ---

(In reply to Chris Murphy from comment #26)

> The basic gist there is that it's fundamentally dangerous and inherently
> broken, is unlikely to get fixed, and is being removed as much as possible
> from user space.

afaics, the only thing that is broken here (reported in this bug) is that systemd/dracut does not properly check whether there is a hibernate image on swap before continuing to boot regularly. The support in the kernel is fine.

--- Additional comment from Lorenzo Pistone on 2015-10-07 08:18:43 EDT ---

Imo that comment doesn't mean that it is actively being killed off, it's just stating the fact that there's not enough manpower to maintain it at consistent quality. Also it's a discussion from March 2012 so most probably irrelevant today.

I agree with Till Maas, the problem here is just poor cooperation between dracut/anaconda/systemd in adding a "resume=..." argument to the kernel command line, *and* (I believe) a recent regression in passing a UUID in such argument, rather than a partition name. If someone can direct me in debugging this second bug on my system I'll be happy to do it.

--- Additional comment from Fedora End Of Life on 2015-11-04 05:17:11 EST ---

This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

--- Additional comment from Stefan Becker on 2015-11-04 06:32:04 EST ---

Moving to F22 as the problem still exists

--- Additional comment from Lorenzo Pistone on 2015-11-04 06:34:05 EST ---

On a fresh installation?

--- Additional comment from Stefan Becker on 2015-11-04 06:36:08 EST ---

A fresh installation adds "resume=" to grub configuration in F22? I doubt that.

--- Additional comment from Lorenzo Pistone on 2015-11-04 07:15:37 EST ---

My bad. I meant in rawhide/23. It's the only explanation left to understand why this appears to be ignored.

--- Additional comment from  on 2016-05-12 18:34:16 EDT ---

I can go ahead and confirm resume= is still missing in a fresh install of F23.

--- Additional comment from Benjamin Xiao on 2016-06-01 12:40:42 EDT ---

(In reply to Lorenzo Pistone from comment #20)
> I noticed that using the partition name (/dev/sdxy) instead of the partition
> UUID works. Needless to say, I double-checked that the partition UUID that I
> use is correct and so it is.

I can confirm this. Using FS UUID makes the hibernation fail randomly but specifying the swap partition by /dev works every time.

Not sure why this is taking such a long time to fix. We really just need the installer to specify a grub parameter for the swap partition. The lack of a resume parameter is really dangerous, especially for situations where laptops run out of batteries. Gnome attempts to hibernate when laptops run out of power, but if the resume parameter isn't there, you might as well just hard power off...

--- Additional comment from Chris Murphy on 2016-06-01 13:23:44 EDT ---

What's taking a long time is there's a disagreement whether resume= should be handled in dracut or by the installer. An additional problem, in my opinion, is there is a rather significant lack of interest in power management in Fedora, from the Workstation WG, the kernel and installer teams. So until that changes, it's unlikely to get significant investment in making this work out of the box and maintaining it.

In the meantime, bug 1332266 is accepted as freeze exception, that change will cause systemd to understand resume= is missing, and therefore it won't mislead upower into thinking suspend to disk is supported, and therefore you shouldn't get a message in the DE indicating the system will hibernate. Instead, in critical power situations, you should get a notification the system will shut down.

--- Additional comment from Chris Murphy on 2016-06-01 13:31:33 EDT ---

As described, this bug is probably not a bug, but a duplicate of bug 1206936.

Also, it's unreliable to support this with resume=/dev/ designation because on any system with two or more drives, that designation can flip between boots and there is no way around that. So it needs to work reliably with UUID or with a label, and if a label that means anaconda needs to give swap a label at creation time.

An additional problem is that by default Anaconda puts swap on LVM. The dracut maintainer says the hibernation image being located on LVM isn't supportable.
http://systemd-devel.freedesktop.narkive.com/yDJDcA9o/systemd-hibernator-generator-does-not-function-on-default-fedora-install#post9

--- Additional comment from Stefan Becker on 2016-06-01 13:55:20 EDT ---

(In reply to Chris Murphy from comment #37)
> An additional problem is that by default Anaconda puts swap on LVM. The
> dracut maintainer says the hibernation image being located on LVM isn't
> supportable.

Funny, swap on LVM and hibernation has worked flawlessly in Fedora for well over 10 years now. Swap on encrypted LVM is also the easiest way for whole disk encryption from usability PoV.

I can only urge dracut and/or systemd maintainers that they are not going to break this.

--- Additional comment from Stefan Becker on 2016-06-01 14:08:18 EDT ---

(In reply to Chris Murphy from comment #37)
> Also, it's unreliable to support this with resume=/dev/ designation because
> on any system with two or more drives, that designation can flip between
> boots and there is no way around that.

That statement doesn't apply to LVM: resume=/dev/mapper/<VG name>-<LV name> is a name that is always the same at each boot.

--- Additional comment from Fedora End Of Life on 2016-07-19 09:15:48 EDT ---

Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.