Bug 1746405

Summary:	"Failed to start udev Coldplug all Devices"
Product:	[Fedora] Fedora	Reporter:	Menanteau Guy <menantea>
Component:	systemd	Assignee:	systemd-maint
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	31	CC:	dan, extras-orphan, hannsj_uhl, jonathan, lnykryn, msekleta, ssahani, s, systemd-maint, watanabe.yu, zbyszek
Target Milestone:	---
Target Release:	---
Hardware:	ppc64le
OS:	Linux
Whiteboard:
Fixed In Version:	systemd-243-3.gitef67743.fc31	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-10-26 17:24:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1071880

Description Menanteau Guy 2019-08-28 11:53:46 UTC

After a fedora fresh installation with fedora 31 on architecture ppc64le, we can see a systemd-udev-trigger.service failled with "Failed to start udev Coldplug all Devices.1133C"

This always reproducible. I can say that problem was not present on 2019/07/17 but was there on 2019/08/02.

● systemd-udev-trigger.service - udev Coldplug all Devices
   Loaded: loaded (/usr/lib/systemd/system/systemd-udev-trigger.service; static; vendor preset: disabled)
  Drop-In: /usr/lib/systemd/system/systemd-udev-trigger.service.d
           └─systemd-udev-trigger-no-reload.conf
   Active: failed (Result: exit-code) since Wed 2019-08-28 10:56:25 BST; 39s ago
     Docs: man:udev(7)
           man:systemd-udevd.service(8)
  Process: 562 ExecStart=/usr/bin/udevadm trigger --type=subsystems --action=add (code=exited, status=0/SUCCESS)
  Process: 575 ExecStart=/usr/bin/udevadm trigger --type=devices --action=add (code=exited, status=1/FAILURE)
 Main PID: 575 (code=exited, status=1/FAILURE)
      CPU: 148ms

Aug 28 10:56:25 localhost.localdomain udevadm[575]: Failed to write 'add' to '/sys/devices/vio/uevent': No such device


in messages log:
Aug 28 10:56:23 localhost systemd[1]: Starting udev Coldplug all Devices...
Aug 28 10:56:23 localhost kernel: audit: type=1130 audit(1566986183.010:4): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udevd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 28 10:56:23 localhost systemd[1]: Mounting Kernel Configuration File System...
Aug 28 10:56:23 localhost systemd[1]: Mounted Kernel Configuration File System.
Aug 28 10:56:23 localhost udevadm[321]: Failed to write 'add' to '/sys/devices/vio/uevent': No such device
Aug 28 10:56:23 localhost systemd[1]: systemd-udev-trigger.service: Main process exited, code=exited, status=1/FAILURE result 'exit-code'.
Aug 28 10:56:23 localhost systemd[1]: Failed to start udev Coldplug all Devices.
Aug 28 10:56:23 localhost audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udev-trigger comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Aug 28 10:56:23 localhost kernel: synth uevent: /devices/vio: failed to send uevent
Aug 28 10:56:23 localhost kernel: vio vio: uevent: failed to send synthetic uevent
Aug 28 10:56:23 localhost kernel: audit: type=1130 audit(1566986183.130:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udev-trigger comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'


[root@vm54 log]# ls /sys/bus/vio/drivers/
hvc_console  ibmvscsi  tpm_ibmvtpm

[root@vm54 log]# ls -ltra /sys/bus/vio/drivers/ibmvscsi/
total 0
drwxr-xr-x. 5 root root     0 Aug 28 10:56 ..
--w-------. 1 root root 65536 Aug 28 10:56 uevent
drwxr-xr-x. 2 root root     0 Aug 28 10:56 .
--w-------. 1 root root 65536 Aug 28 12:16 unbind
lrwxrwxrwx. 1 root root     0 Aug 28 12:16 module -> ../../../../module/ibmvscsi
--w-------. 1 root root 65536 Aug 28 12:16 bind
lrwxrwxrwx. 1 root root     0 Aug 28 12:16 71000002 -> ../../../../devices/vio/71000002

[root@vm54 log]# ls -ltra /sys/bus/vio/drivers/hvc_console/
total 0
--w-------. 1 root root 65536 Aug 28 10:56 uevent
drwxr-xr-x. 5 root root     0 Aug 28 10:56 ..
drwxr-xr-x. 2 root root     0 Aug 28 10:56 .
lrwxrwxrwx. 1 root root     0 Aug 28 12:17 71000000 -> ../../../../devices/vio/71000000

[root@vm54 log]# ls -ltra /sys/bus/vio/drivers/tpm_ibmvtpm/
total 0
--w-------. 1 root root 65536 Aug 28 10:56 uevent
drwxr-xr-x. 5 root root     0 Aug 28 10:56 ..
drwxr-xr-x. 2 root root     0 Aug 28 10:56 .
--w-------. 1 root root 65536 Aug 28 12:17 unbind
lrwxrwxrwx. 1 root root     0 Aug 28 12:17 module -> ../../../../module/tpm_ibmvtpm
--w-------. 1 root root 65536 Aug 28 12:17 bind

devices are
[root@vm54 log]# ls -ltra /sys/bus/vio/devices/
total 0
drwxr-xr-x. 4 root root 0 Aug 28 10:56 ..
lrwxrwxrwx. 1 root root 0 Aug 28 10:56 vio -> ../../../devices/vio
lrwxrwxrwx. 1 root root 0 Aug 28 10:56 71000002 -> ../../../devices/vio/71000002
lrwxrwxrwx. 1 root root 0 Aug 28 10:56 71000001 -> ../../../devices/vio/71000001
lrwxrwxrwx. 1 root root 0 Aug 28 10:56 71000000 -> ../../../devices/vio/71000000

[root@vm54 log]# ls -ltra /sys/bus/vio/devices/../../../devices/vio/71000000
total 0
-rw-r--r--. 1 root root 65536 Aug 28 10:56 uevent
lrwxrwxrwx. 1 root root     0 Aug 28 10:56 subsystem -> ../../../bus/vio
drwxr-xr-x. 6 root root     0 Aug 28 10:56 ..
drwxr-xr-x. 3 root root     0 Aug 28 10:56 .
drwxr-xr-x. 2 root root     0 Aug 28 12:11 power
lrwxrwxrwx. 1 root root     0 Aug 28 12:11 of_node -> ../../../firmware/devicetree/base/vdevice/vty@71000000
-r--r--r--. 1 root root 65536 Aug 28 12:11 name
-r--r--r--. 1 root root 65536 Aug 28 12:11 modalias
lrwxrwxrwx. 1 root root     0 Aug 28 12:11 driver -> ../../../bus/vio/drivers/hvc_console
-r--r--r--. 1 root root 65536 Aug 28 12:11 devspec

[root@vm54 log]# ls -ltra /sys/bus/vio/devices/../../../devices/vio/71000001
total 0
-rw-r--r--. 1 root root 65536 Aug 28 10:56 uevent
lrwxrwxrwx. 1 root root     0 Aug 28 10:56 subsystem -> ../../../bus/vio
drwxr-xr-x. 6 root root     0 Aug 28 10:56 ..
drwxr-xr-x. 3 root root     0 Aug 28 10:56 .
drwxr-xr-x. 2 root root     0 Aug 28 12:11 power
lrwxrwxrwx. 1 root root     0 Aug 28 12:11 of_node -> ../../../firmware/devicetree/base/vdevice/nvram@71000001
-r--r--r--. 1 root root 65536 Aug 28 12:11 name
-r--r--r--. 1 root root 65536 Aug 28 12:11 modalias
-r--r--r--. 1 root root 65536 Aug 28 12:11 devspec

[root@vm54 log]# ls -ltra /sys/bus/vio/devices/../../../devices/vio/71000002
total 0
-rw-r--r--. 1 root root 65536 Aug 28 10:56 uevent
lrwxrwxrwx. 1 root root     0 Aug 28 10:56 subsystem -> ../../../bus/vio
drwxr-xr-x. 6 root root     0 Aug 28 10:56 ..
drwxr-xr-x. 4 root root     0 Aug 28 10:56 .
drwxr-xr-x. 9 root root     0 Aug 28 10:56 host0
lrwxrwxrwx. 1 root root     0 Aug 28 10:56 driver -> ../../../bus/vio/drivers/ibmvscsi
drwxr-xr-x. 2 root root     0 Aug 28 12:11 power
lrwxrwxrwx. 1 root root     0 Aug 28 12:11 of_node -> ../../../firmware/devicetree/base/vdevice/v-scsi@71000002
-r--r--r--. 1 root root 65536 Aug 28 12:11 name
-r--r--r--. 1 root root 65536 Aug 28 12:11 modalias
-r--r--r--. 1 root root 65536 Aug 28 12:11 devspec

Comment 1 Dan Horák 2019-09-20 09:02:31 UTC

switching to systemd as udev is part of it

Comment 2 Yu Watanabe 2019-10-14 16:35:01 UTC

This should be fixed by https://github.com/systemd/systemd/pull/13656.

Comment 3 Zbigniew Jędrzejewski-Szmek 2019-10-14 20:35:31 UTC

That patch was part of the latest update that will go stable when F31 final freeze ends.

Comment 4 Fedora Update System 2019-10-14 20:36:02 UTC

FEDORA-2019-5cb48e5111 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-5cb48e5111

Comment 5 Fedora Update System 2019-10-26 17:24:45 UTC

systemd-243-3.gitef67743.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.