Bug 1394220

Summary: Stream of (udev?) errors at shutdown (luks?)
Product: [Fedora] Fedora Reporter: Dr. David Alan Gilbert <dgilbert>
Component: systemdAssignee: systemd-maint
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 27CC: accounts+fedora, amurdaca, fedora, johannbg, lnykryn, ludovic, msekleta, muadda, palazzotti, sbonazzo, ssahani, s, systemd-maint, zbyszek
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-11-30 18:34:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
journalctl -b-1 trimmed to start just before I start shutdown none

Description Dr. David Alan Gilbert 2016-11-11 12:37:48 UTC
Description of problem:
This is new for me in f25; I hadn't noticed it in f20..f24.
When I shutdown my laptop I see a screen full of messages rapidly scroll past; I think they're udev related but haven't managed to catch them yet.  I do however see in journalctl -b-1 the following:

Nov 10 20:35:33 dgilbert-t530 systemd[1]: Deactivated swap /dev/mapper/fedworkssd-swap.
Nov 10 20:35:33 dgilbert-t530 systemd[1]: Stopped File System Check on /dev/disk/by-uuid/e5a3c1ec-b6b4-4acf-882e-364db461696a.
Nov 10 20:35:33 dgilbert-t530 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-disk-by\x2duuid-e5a3c1ec\x2db6b4\x2d4acf\x2d882e\x2d364db461696a comm="systemd" ex
Nov 10 20:35:33 dgilbert-t530 kernel: audit: type=1131 audit(1478810133.385:854): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-disk-by\x2duuid-e5a3c1ec\x2db6b4\x2d4acf\x2d882e\x2d3
Nov 10 20:35:33 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:33 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:33 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:34 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:34 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:34 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:34 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:34 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Unmounted /home.
Nov 10 20:35:35 dgilbert-t530 kernel: audit: type=1131 audit(1478810135.054:855): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-mapper-fedworkssd\x2dhome comm="systemd" exe="/usr/li
Nov 10 20:35:35 dgilbert-t530 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-mapper-fedworkssd\x2dhome comm="systemd" exe="/usr/lib/systemd/systemd" hostname=?
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Stopped File System Check on /dev/mapper/fedworkssd-home.
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Removed slice system-systemd\x2dfsck.slice.
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Stopped target Local File Systems (Pre).
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Stopped Create Static Device Nodes in /dev.
Nov 10 20:35:35 dgilbert-t530 kernel: audit: type=1131 audit(1478810135.058:856): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-setup-dev comm="systemd" exe="/usr/lib/systemd/system
Nov 10 20:35:35 dgilbert-t530 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-setup-dev comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Stopped Remount Root and Kernel File Systems.
Nov 10 20:35:35 dgilbert-t530 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-remount-fs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=s
Nov 10 20:35:35 dgilbert-t530 kernel: audit: type=1131 audit(1478810135.062:857): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-remount-fs comm="systemd" exe="/usr/lib/systemd/systemd" hostn
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Stopping Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Nov 10 20:35:35 dgilbert-t530 lvm[2282]:   WARNING: Failed to connect to lvmetad. Falling back to device scanning.
Nov 10 20:35:35 dgilbert-t530 kernel: device-mapper: ioctl: unable to remove open device luks-ebbae151-fe41-46ae-97bc-f6016b75d241
Nov 10 20:35:35 dgilbert-t530 lvm[2282]:   3 logical volume(s) in volume group "fedworkssd" unmonitored
Nov 10 20:35:35 dgilbert-t530 systemd[1]: Stopped Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Nov 10 20:35:35 dgilbert-t530 audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-monitor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success
Nov 10 20:35:35 dgilbert-t530 kernel: audit: type=1131 audit(1478810135.123:858): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=lvm2-monitor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? 
Nov 10 20:35:36 dgilbert-t530 systemd[1]: Received SIGRTMIN+20 from PID 2158 (plymouthd).
Nov 10 20:35:38 dgilbert-t530 systemd-cryptsetup[2257]: Failed to deactivate: Device or resource busy
Nov 10 20:35:38 dgilbert-t530 systemd[1]: systemd-cryptsetup@luks\x2debbae151\x2dfe41\x2d46ae\x2d97bc\x2df6016b75d241.service: Control process exited, code=exited status=1
Nov 10 20:35:38 dgilbert-t530 systemd[1]: Stopped Cryptography Setup for luks-ebbae151-fe41-46ae-97bc-f6016b75d241.
Nov 10 20:35:38 dgilbert-t530 systemd[1]: systemd-cryptsetup@luks\x2debbae151\x2dfe41\x2d46ae\x2d97bc\x2df6016b75d241.service: Unit entered failed state.
Nov 10 20:35:38 dgilbert-t530 audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-cryptsetup@luks\x2debbae151\x2dfe41\x2d46ae\x2d97bc\x2df6016b75d241 comm="systemd" exe="/u
Nov 10 20:35:38 dgilbert-t530 systemd[1]: systemd-cryptsetup@luks\x2debbae151\x2dfe41\x2d46ae\x2d97bc\x2df6016b75d241.service: Failed with result 'exit-code'.
Nov 10 20:35:38 dgilbert-t530 systemd[1]: Reached target Unmount All Filesystems.

The laptop has a pretty simple luks setup luks->lvm->{/,/home,swap)
so only one luks partition (and the UUID it's moaning about corresponds to the /dev/mapper/luks-... device that's the pv for the lvm set.

Maybe an lvm vs luks shutdown/ordering thing???

Version-Release number of selected component (if applicable):
systemd-231-10.fc25.x86_64
cryptsetup-1.7.2-3.fc25.x86_64
lvm2-2.02.161-3.fc25.x86_64

How reproducible:
most but not every night since I upgraded to f25

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Zbigniew Jędrzejewski-Szmek 2016-11-11 13:52:13 UTC
Can you attach the full log from shutdown? It seems that the root device is being unmounted too early, this shouldn't happen until after the switch back to initramfs.

Comment 2 Dr. David Alan Gilbert 2016-11-11 15:20:12 UTC
Created attachment 1219819 [details]
journalctl -b-1 trimmed to start just before I start shutdown

As requested here's the log.
For reference; I shutdown by issuing the 'poweroff' command as my normal user (I have a keybinding for it in xfce)

Comment 3 Sandro Bonazzola 2016-12-11 07:24:50 UTC
I've the same issue on Fedora 24

dic 11 07:16:49 arilinn.home systemd[1]: Shutting down.
dic 11 07:16:49 arilinn.home systemd[1]: Starting Power-Off...
dic 11 07:16:49 arilinn.home systemd[1]: Reached target Final Step.
dic 11 07:16:49 arilinn.home systemd[1]: Reached target Shutdown.
dic 11 07:16:49 arilinn.home systemd[1]: Removed slice system-systemd\x2dcryptsetup.slice.
dic 11 07:16:49 arilinn.home systemd[1]: Reached target Unmount All Filesystems.
dic 11 07:16:49 arilinn.home systemd[1]: systemd-cryptsetup@luks\x2d68d9a7fe\x2d2200\x2d456b\x2db5d4\x2d025d13723c2a.service: Failed with result 'exit-code'.
dic 11 07:16:49 arilinn.home systemd[1]: systemd-cryptsetup@luks\x2d68d9a7fe\x2d2200\x2d456b\x2db5d4\x2d025d13723c2a.service: Unit entered failed state.
dic 11 07:16:49 arilinn.home systemd[1]: Stopped Cryptography Setup for luks-68d9a7fe-2200-456b-b5d4-025d13723c2a.
dic 11 07:16:49 arilinn.home audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-cryptsetup@luks\x2d68d9a7fe\x2d2200\x2d456b\x2
dic 11 07:16:49 arilinn.home systemd[1]: systemd-cryptsetup@luks\x2d68d9a7fe\x2d2200\x2d456b\x2db5d4\x2d025d13723c2a.service: Control process exited, code=exited status=1
dic 11 07:16:49 arilinn.home systemd-cryptsetup[7222]: Failed to deactivate: Device or resource busy
dic 11 07:16:46 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:45 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:45 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:45 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:45 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:45 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:44 arilinn.home kernel: device-mapper: ioctl: unable to remove open device luks-68d9a7fe-2200-456b-b5d4-025d13723c2a
dic 11 07:16:44 arilinn.home systemd[1]: Stopped LVM2 metadata daemon.
dic 11 07:16:44 arilinn.home systemd[1]: Stopping LVM2 metadata daemon...
dic 11 07:16:44 arilinn.home systemd[1]: Stopped Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
dic 11 07:16:44 arilinn.home systemd[1]: Stopped Create Static Device Nodes in /dev.
dic 11 07:16:44 arilinn.home lvm[7245]:   3 logical volume(s) in volume group "fedora_arilinn" unmonitored


SystemD is: systemd-229-16.fc24.x86_64

Not sure if this is related, but found https://lists.freedesktop.org/archives/systemd-devel/2016-June/036967.html which seems pretty similar to what's happening.

Comment 4 Antonio Murdaca 2016-12-16 10:55:25 UTC
Having the same issue in Fedora 25:

Sep 08 19:31:40 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:40 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:40 localhost.localdomain systemd[1]: Unmounted /home.

and:

Sep 08 19:31:40 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:41 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:41 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:41 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:41 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:41 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:42 localhost.localdomain kernel: device-mapper: ioctl: unable to remove open device luks-8e30cd08-09f4-45ac-b426-76b1322116bf
Sep 08 19:31:45 localhost.localdomain systemd-cryptsetup[32166]: Failed to deactivate: Device or resource busy
Sep 08 19:31:45 localhost.localdomain systemd[1]: systemd-cryptsetup@luks\x2d8e30cd08\x2d09f4\x2d45ac\x2db426\x2d76b1322116bf.service: Control process exited, code=exited status=1
Sep 08 19:31:45 localhost.localdomain systemd[1]: Stopped Cryptography Setup for luks-8e30cd08-09f4-45ac-b426-76b1322116bf.
Sep 08 19:31:45 localhost.localdomain audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-cryptsetup@luks\x2d8e30cd08\x2d09f4
Sep 08 19:31:45 localhost.localdomain systemd[1]: systemd-cryptsetup@luks\x2d8e30cd08\x2d09f4\x2d45ac\x2db426\x2d76b1322116bf.service: Unit entered failed state.
Sep 08 19:31:45 localhost.localdomain systemd[1]: systemd-cryptsetup@luks\x2d8e30cd08\x2d09f4\x2d45ac\x2db426\x2d76b1322116bf.service: Failed with result 'exit-code'.
Sep 08 19:31:45 localhost.localdomain systemd[1]: Reached target Unmount All Filesystems.
Sep 08 19:31:45 localhost.localdomain systemd[1]: Removed slice system-systemd\x2dcryptsetup.slice.
Sep 08 19:31:45 localhost.localdomain systemd[1]: Reached target Shutdown.
Sep 08 19:31:45 localhost.localdomain systemd[1]: Reached target Final Step.
Sep 08 19:31:45 localhost.localdomain systemd[1]: Starting Power-Off...
Sep 08 19:31:45 localhost.localdomain systemd[1]: Shutting down.

Comment 5 Christian Stadelmann 2017-04-10 16:56:45 UTC
I think this is a duplicate (or result) of https://bugzilla.redhat.com/show_bug.cgi?id=1026119.

Comment 6 Fedora End Of Life 2017-11-16 18:57:47 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 7 Dr. David Alan Gilbert 2017-11-17 20:24:00 UTC
still seeing it in f27

Comment 8 Christian Stadelmann 2017-11-18 13:58:11 UTC
(In reply to Christian Stadelmann from comment #5)
> I think this is a duplicate (or result) of
> https://bugzilla.redhat.com/show_bug.cgi?id=1026119.

Have you checked that?

Comment 9 Ben Cotton 2018-11-27 17:57:12 UTC
This message is a reminder that Fedora 27 is nearing its end of life.
On 2018-Nov-30  Fedora will stop maintaining and issuing updates for
Fedora 27. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora  'version' of '27'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 27 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 10 Ben Cotton 2018-11-30 18:34:46 UTC
Fedora 27 changed to end-of-life (EOL) status on 2018-11-30. Fedora 27 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.