Bug 1741140

Summary: The vm cannot be started anymore after reboot host when libvirtd stopped
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: yisun
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: yisun
Severity: high Docs Contact:
Priority: high    
Version: 8.1CC: chhu, fjin, jdenemar, lmen, xuzhang, yafu, yisun
Target Milestone: rcKeywords: Regression, Upstream
Target Release: 8.0Flags: knoel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-5.6.0-4.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-06 07:18:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1652078    

Description yisun 2019-08-14 11:10:09 UTC
Description:
The vm cannot be started anymore after reboot host when libvirtd stopped

Versions:
kernel-4.18.0-107.el8.x86_64
qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3.x86_64
libvirt-5.6.0-1.module+el8.1.0+3890+4d3d259c.x86_64

How reproducible:
100%


Steps:
1. have a running vm
$ virsh domblklist avocado-vt-vm1
 Target   Source
----------------------------------------------------------------
 vda      /var/lib/libvirt/images/RHEL-8.1-x86_64-latest.qcow2

$ virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started

2. check the libvirtd existing
$ ps -ef | grep libvirtd
root      1976     1  0 06:41 ?        00:00:00 /usr/sbin/libvirtd --timeout 120

3. stop libvirtd
$ systemctl stop libvirtd
Warning: Stopping libvirtd.service, but it can still be activated by:
  libvirtd-admin.socket
  libvirtd.socket
  libvirtd-ro.socket

$ ps -ef | grep libvirtd | grep -v grep | wc -l
0

4. reboot host
$reboot


reconnecting to host ...

5. start the vm again, failure happened
$ virsh start avocado-vt-vm1 
error: Failed to start domain avocado-vt-vm1
error: internal error: child reported (status=125): Requested operation is not valid: Setting different SELinux label on /var/lib/libvirt/images/RHEL-8.1-x86_64-latest.qcow2 which is already in use


Additional info:
Since libvirtd now have a "--timeout 120" keepalive setting by default. So sometimes we don't need to manually turn off libvirtd as step 3 does.

Comment 1 Michal Privoznik 2019-08-14 14:41:32 UTC
Oh, I've ran into this ussue the other day, wrote the patches but never send them. Well, now I just did:

https://www.redhat.com/archives/libvir-list/2019-August/msg00520.html

Comment 2 Michal Privoznik 2019-08-22 12:32:19 UTC
v2 posted upstream:

https://www.redhat.com/archives/libvir-list/2019-August/msg01030.html

Comment 3 Michal Privoznik 2019-08-22 15:21:38 UTC
Fixed upstream as:

7cfb7aab57 security_util: Remove stale XATTRs
8b802f13cb util: Introduce virhostuptime

v5.6.0-300-g7cfb7aab57

Comment 7 yisun 2019-09-05 11:50:08 UTC
Verified on: libvirt-5.6.0-4.virtcov.el8.x86_64

[root@dell-per730-58 ~]# virsh domblklist avocado-vt-vm1
 Target   Source
----------------------------------------------------------------
 vda      /var/lib/libvirt/images/RHEL-8.1-x86_64-latest.qcow2

[root@dell-per730-58 ~]# virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started

[root@dell-per730-58 ~]# ps -ef | grep libvirtd | grep -v grep | wc -l
1

[root@dell-per730-58 ~]# systemctl stop libvirtd
Warning: Stopping libvirtd.service, but it can still be activated by:
  libvirtd.socket
  libvirtd-ro.socket
  libvirtd-admin.socket

[root@dell-per730-58 ~]# ps -ef | grep libvirtd | grep -v grep | wc -l
0

[root@dell-per730-58 ~]# reboot
Connection to 10.73.196.87 closed by remote host.
Connection to 10.73.196.87 closed.

ovpn-12-152:~ lento$ ssh root.196.87
root.196.87's password:
Last login: Thu Sep  5 07:41:35 2019 from 10.72.12.55

[root@dell-per730-58 ~]# virsh start avocado-vt-vm1
Domain avocado-vt-vm1 started

Comment 9 errata-xmlrpc 2019-11-06 07:18:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3723