1033626 – libvirt-client update causes suspend on all guests

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1033626 - libvirt-client update causes suspend on all guests

Summary: libvirt-client update causes suspend on all guests

Keywords:
Status:	CLOSED CURRENTRELEASE
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	7.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Jiri Denemark
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:	962225
Blocks:
TreeView+	depends on / blocked

Reported:	2013-11-22 14:18 UTC by Jiri Denemark
Modified:	2014-06-18 00:59 UTC (History)
CC List:	11 users (show)
Fixed In Version:	libvirt-1.1.1-15.el7
Doc Type:	Bug Fix
Doc Text:
Clone Of:	962225
Environment:
Last Closed:	2014-06-13 13:18:30 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Jiri Denemark 2013-11-22 14:18:22 UTC

+++ This bug was initially created as a clone of Bug #962225 +++

When updating libvirt-client, the scriptlets run a '/usr/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || :' and this causes all my guests to be suspended. 

When there's a bunch of guests this takes a while and disrupts service to them. 

Is there any way it could not do this on upgrade?

--- Additional comment from Cole Robinson on 2013-11-21 20:16:27 UTC ---

I've posted a patch upstream:

http://www.redhat.com/archives/libvir-list/2013-November/msg00860.html

Comment 1 Jiri Denemark 2013-11-22 14:47:40 UTC

Fixed upstream by v1.1.4-139-gd65e0e1:

commit d65e0e1466b09964b85c79b8cc78be454df53583
Author: Cole Robinson <crobinso>
Date:   Thu Nov 21 15:04:35 2013 -0500

    spec: Don't save/restore running VMs on libvirt-client update
    
    Restarting an active libvirt-guests.service is the equivalent of
    doing:
    
    /usr/libexec/libvirt-guests.sh stop
    /usr/libexec/libvirt-guests.sh start
    
    Which in a default configuration will managedsave every running VM,
    and then restore them. Certainly not something we should do every
    time the libvirt-client RPM is updated.
    
    Just drop the try-restart attempt, I don't know what purpose it
    serves anyways.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=962225

Comment 4 zhenfeng wang 2013-11-25 12:24:14 UTC

Hi jiri
I try to reproduce this bug with the following steps, however, still can't hit the issue , can you help me have a look, maybe i miss some steps, thank you.

steps

My current libvirt-client version was libvirt-client-1.1.1-10.el7.x86_64.rpm

1. start 5 guests and keep libvirt-guests service in running status 
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 122   rhel6                          running
 123   win7                           running
 124   win7raw                        running
 125   rhel7qcow2                     running


# systemctl status libvirt-guests
libvirt-guests.service - Suspend Active Libvirt Guests
   Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; enabled)
Active: active (exited) since Mon 2013-11-25 19:42:23 CST; 5min ago

2.Reinstall the libvirt-client with libvirt-10, after updating, The guest was not to be suspended, however was still in running status
# rpm -ivh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]
]# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 122   rhel6                          running
 123   win7                           running
 124   win7raw                        running
 125   rhel7qcow2                     running


# rpm -Uvh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]
[root@zhwang71 libvirtnew]# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 122   rhel6                          running
 123   win7                           running
 124   win7raw                        running
 125   rhel7qcow2                     running




3.restart the libvirt-guests service during updating the libvirt-client.

In 1 terminal update the libvirt-client
# rpm -ivh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]

In another terminal restart the libvirt-guests service with the following command
# /usr/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || :

check the guest status after the libvirt-guests service was finishing restart, the guests was still in running status
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 104   rhel6                          running
 105   win7                           running
 106   win7raw                        running
 107   rhel7qcow2                     running

In 1 terminal update the libvirt-client
# rpm -Uvh libvirt-client-1.1.1-10.el7.x86_64.rpm  --force
Preparing...                          ################################# [100%]
Updating / installing...
   1:libvirt-client-1.1.1-10.el7      ################################# [100%]

In another terminal restart the libvirt-guests service with the following command
# /usr/bin/systemctl try-restart libvirt-guests.service >/dev/null 2>&1 || :

check the guest status after the libvirt-guests service was finishing restart, the guests was still in running status
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 139   rhel6                          running
 140   win7                           running
 141   win7raw                        running
 142   rhel7qcow2                     running


4.Update the libvirt-client to libvirt-12, The guest was not to be suspended, however was still in running status
#rpm -Uvh libvirt-* 
#rpm -qa|grep libvirt-client
libvirt-client-1.1.1-12.el7.x86_64
# virsh list
 Id    Name                           State
----------------------------------------------------
 2     rhel7                          running
 135   rhel6                          running
 136   win7                           running
 137   win7raw                        running
 138   rhel7qcow2                     running

Comment 5 Jiri Denemark 2013-11-25 15:14:32 UTC

So it seems I gave you a bit misleading information. Reinstalling the same version does not trigger the post-uninstall script in the package. Thus, you need to actually upgrade (or downgrade) the package. When you have -10 libvirt, you can try upgrading to -11. The domains that were running before should still be running but you should see their IDs changed. This is a sign that the domains were actually saved and started again. If the bug is fixed, domain IDs won't change.

Comment 6 Jiri Denemark 2013-11-25 15:22:13 UTC

In any case, the patch removes one save/restore iteration on libvirt-client update but unfortunately, there were two iterations so domains are still saved/restored during update (although "just" once).

Comment 7 Jiri Denemark 2013-12-09 12:59:02 UTC

Fixed upstream by

commit 604f79b3603d6f7d922c7155d67ff9617411e79f
Author: Jiri Denemark <jdenemar>
Date:   Mon Nov 25 16:37:32 2013 +0100

    spec: Don't save/restore running VMs on libvirt-client update
    
    The previous attempt (commit d65e0e1) removed just one of two
    libvirt-guests restarts that happened on libvirt-client update. Let's
    remove the last one too :-)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=962225
    
    Signed-off-by: Jiri Denemark <jdenemar>

Comment 10 zhenfeng wang 2013-12-16 06:10:50 UTC

Hi jiri
I verify this bug right now, i found the issue was still existing while i update the libvirt from libvirt-1.1-14 -> libvirt-1.1.1-15. I guest that maybe we need update the libvirt from the libvirt-1.1.1-15 to a newer libvirt version to verify this bug, right? If my presumptions are true then we may have to wait for another libvirt packet which >libivrt-1.1.1-15 to verify this bug. please give me some advise, thanks

Comment 11 Jiri Denemark 2013-12-16 09:37:27 UTC

(In reply to zhenfeng wang from comment #10)
> I verify this bug right now, i found the issue was still existing while i
> update the libvirt from libvirt-1.1-14 -> libvirt-1.1.1-15. I guest that
> maybe we need update the libvirt from the libvirt-1.1.1-15

Right, libvirt-1.1.1-15 is the one that must be replaced by another version to check the post-uninstall script in -15 works correctly.

> we may have to wait for another libvirt packet which >libivrt-1.1.1-15 to
> verify this bug. please give me some advise, thanks

That's one option. Another one is you can downgrade libvirt back from -15 to -14.

Comment 12 zhenfeng wang 2013-12-17 03:22:03 UTC

Thanks for jiri's response, verify this bug with libvirt-1.1.1-15, the verify steps as following

pkg info
libvirt-1.1.1-15.el7.x86_64
kernel-3.10.0-60.el7.x86_64
qemu-kvm-rhev-1.5.3-21.el7.x86_64

steps
1.Check the current libvirt version, then start two guests

# rpm -qa|grep libvirt
libvirt-1.1.1-15.el7.x86_64

# virsh list
 Id    Name                           State
----------------------------------------------------
 6     rhel7                          running
 7     rhel7qcow2                     running

2.Downgrade the libvirt from libvirt-15 -> libvirt14
#yum downgrade libvirt

3.After finished downlgrade, The guest's id didn't changed
# rpm -qa|grep libvirt
libvirt-1.1.1-14.el7.x86_64

# virsh list
 Id    Name                           State
----------------------------------------------------
 6     rhel7                          running
 7     rhel7qcow2                     running

restart libvirtd service, the guest's id didn't changed
# service libvirtd restart
Redirecting to /bin/systemctl restart  libvirtd.service

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 6     rhel7                          running
 7     rhel7qcow2                     running

Also i can reproduce this bug while downgrade the libvirt from libvirt-14 ->libvirt 13, so mark this bug verifed

Comment 13 Ludek Smid 2014-06-13 13:18:30 UTC

This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.

Note You need to log in before you can comment on or make changes to this bug.