Bug 1033626

Summary: libvirt-client update causes suspend on all guests
Product: Red Hat Enterprise Linux 7 Reporter: Jiri Denemark <jdenemar>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: acathrow, dallan, dyuan, eblake, jdenemar, libvirt-maint, mjenner, mzhan, ydu, zhwang, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.1.1-15.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 962225 Environment:
Last Closed: 2014-06-13 13:18:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 962225    
Bug Blocks:    

Description Jiri Denemark 2013-11-22 14:18:22 UTC
+++ This bug was initially created as a clone of Bug #962225 +++

When updating libvirt-client, the scriptlets run a '/usr/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || :' and this causes all my guests to be suspended. 

When there's a bunch of guests this takes a while and disrupts service to them. 

Is there any way it could not do this on upgrade?

--- Additional comment from Cole Robinson on 2013-11-21 20:16:27 UTC ---

I've posted a patch upstream:

http://www.redhat.com/archives/libvir-list/2013-November/msg00860.html

Comment 1 Jiri Denemark 2013-11-22 14:47:40 UTC
Fixed upstream by v1.1.4-139-gd65e0e1:

commit d65e0e1466b09964b85c79b8cc78be454df53583
Author: Cole Robinson <crobinso>
Date:   Thu Nov 21 15:04:35 2013 -0500

    spec: Don't save/restore running VMs on libvirt-client update
    
    Restarting an active libvirt-guests.service is the equivalent of
    doing:
    
    /usr/libexec/libvirt-guests.sh stop
    /usr/libexec/libvirt-guests.sh start
    
    Which in a default configuration will managedsave every running VM,
    and then restore them. Certainly not something we should do every
    time the libvirt-client RPM is updated.
    
    Just drop the try-restart attempt, I don't know what purpose it
    serves anyways.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=962225

Comment 4 zhenfeng wang 2013-11-25 12:24:14 UTC
Hi jiri
I try to reproduce this bug with the following steps, however, still can't hit the issue , can you help me have a look, maybe i miss some steps, thank you.

steps

My current libvirt-client version was libvirt-client-1.1.1-10.el7.x86_64.rpm

1. start 5 guests and keep libvirt-guests service in running status 
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 122   rhel6                          running
 123   win7                           running
 124   win7raw                        running
 125   rhel7qcow2                     running


# systemctl status libvirt-guests
libvirt-guests.service - Suspend Active Libvirt Guests
   Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; enabled)
Active: active (exited) since Mon 2013-11-25 19:42:23 CST; 5min ago

2.Reinstall the libvirt-client with libvirt-10, after updating, The guest was not to be suspended, however was still in running status
# rpm -ivh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]
]# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 122   rhel6                          running
 123   win7                           running
 124   win7raw                        running
 125   rhel7qcow2                     running


# rpm -Uvh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]
[root@zhwang71 libvirtnew]# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 122   rhel6                          running
 123   win7                           running
 124   win7raw                        running
 125   rhel7qcow2                     running




3.restart the libvirt-guests service during updating the libvirt-client.

In 1 terminal update the libvirt-client
# rpm -ivh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]

In another terminal restart the libvirt-guests service with the following command
# /usr/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || :

check the guest status after the libvirt-guests service was finishing restart, the guests was still in running status
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 104   rhel6                          running
 105   win7                           running
 106   win7raw                        running
 107   rhel7qcow2                     running

In 1 terminal update the libvirt-client
# rpm -Uvh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]

In another terminal restart the libvirt-guests service with the following command
# /usr/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || :

check the guest status after the libvirt-guests service was finishing restart, the guests was still in running status
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 139   rhel6                          running
 140   win7                           running
 141   win7raw                        running
 142   rhel7qcow2                     running


4.Update the libvirt-client to libvirt-12, The guest was not to be suspended, however was still in running status
#rpm -Uvh libvirt-* 
#rpm -qa|grep libvirt-client
libvirt-client-1.1.1-12.el7.x86_64
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 135   rhel6                          running
 136   win7                           running
 137   win7raw                        running
 138   rhel7qcow2                     running

Comment 5 Jiri Denemark 2013-11-25 15:14:32 UTC
So it seems I gave you a bit misleading information. Reinstalling the same version does not trigger the post-uninstall script in the package. Thus, you need to actually upgrade (or downgrade) the package. When you have -10 libvirt, you can try upgrading to -11. The domains that were running before should still be running but you should see their IDs changed. This is a sign that the domains were actually saved and started again. If the bug is fixed, domain IDs won't change.

Comment 6 Jiri Denemark 2013-11-25 15:22:13 UTC
In any case, the patch removes one save/restore iteration on libvirt-client update but unfortunately, there were two iterations so domains are still saved/restored during update (although "just" once).

Comment 7 Jiri Denemark 2013-12-09 12:59:02 UTC
Fixed upstream by

commit 604f79b3603d6f7d922c7155d67ff9617411e79f
Author: Jiri Denemark <jdenemar>
Date:   Mon Nov 25 16:37:32 2013 +0100

    spec: Don't save/restore running VMs on libvirt-client update
    
    The previous attempt (commit d65e0e1) removed just one of two
    libvirt-guests restarts that happened on libvirt-client update. Let's
    remove the last one too :-)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=962225
    
    Signed-off-by: Jiri Denemark <jdenemar>

Comment 10 zhenfeng wang 2013-12-16 06:10:50 UTC
Hi jiri
I verify this bug right now, i found the issue was still existing while i update the libvirt from libvirt-1.1-14 -> libvirt-1.1.1-15. I guest that maybe we need update the libvirt from the libvirt-1.1.1-15 to a newer libvirt version to verify this bug, right? If my presumptions are true then we may have to wait for another libvirt packet which >libivrt-1.1.1-15 to verify this bug. please give me some advise, thanks

Comment 11 Jiri Denemark 2013-12-16 09:37:27 UTC
(In reply to zhenfeng wang from comment #10)
> I verify this bug right now, i found the issue was still existing while i
> update the libvirt from libvirt-1.1-14 -> libvirt-1.1.1-15. I guest that
> maybe we need update the libvirt from the libvirt-1.1.1-15

Right, libvirt-1.1.1-15 is the one that must be replaced by another version to check the post-uninstall script in -15 works correctly.

> we may have to wait for another libvirt packet which >libivrt-1.1.1-15 to
> verify this bug. please give me some advise, thanks

That's one option. Another one is you can downgrade libvirt back from -15 to -14.

Comment 12 zhenfeng wang 2013-12-17 03:22:03 UTC
Thanks for jiri's response, verify this bug with libvirt-1.1.1-15, the verify steps as following

pkg info
libvirt-1.1.1-15.el7.x86_64
kernel-3.10.0-60.el7.x86_64
qemu-kvm-rhev-1.5.3-21.el7.x86_64

steps
1.Check the current libvirt version, then start two guests

# rpm -qa|grep libvirt
libvirt-1.1.1-15.el7.x86_64

# virsh list
 Id    Name                           State
----------------------------------------------------
 6     rhel7                          running
 7     rhel7qcow2                     running

2.Downgrade the libvirt from libvirt-15 -> libvirt14
#yum downgrade libvirt

3.After finished downlgrade, The guest's id didn't changed
# rpm -qa|grep libvirt
libvirt-1.1.1-14.el7.x86_64

# virsh list
 Id    Name                           State
----------------------------------------------------
 6     rhel7                          running
 7     rhel7qcow2                     running

restart libvirtd service, the guest's id didn't changed
# service libvirtd restart
Redirecting to /bin/systemctl restart  libvirtd.service

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 6     rhel7                          running
 7     rhel7qcow2                     running

Also i can reproduce this bug while downgrade the libvirt from libvirt-14 ->libvirt 13, so mark this bug verifed

Comment 13 Ludek Smid 2014-06-13 13:18:30 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.