Bug 607735

Summary: Operations on remote (high latency) system stall interactivity with other VMs
Product: Red Hat Enterprise Linux 7 Reporter: Alex Williamson <alex.williamson>
Component: virt-managerAssignee: Giuseppe Scrivano <gscrivan>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 7.0CC: codong, crobinso, cwei, duffy, dyuan, gscrivan, juzhou, jwu, mzhan, xen-maint, zpeng
Target Milestone: rcKeywords: Upstream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: virt-manager-1.1.0-3.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 10:02:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alex Williamson 2010-06-24 16:58:00 UTC
Description of problem:
Working outside the Red Hat firewall, I have a running KVM VM on my local system managed with virt-manager, console up and active.  When I have virt-manager connect to a system behind the firewall (openvpn, 150-200ms ping times), interactivity with the console on the local VM stutters until the connection completes.

Then it gets weird.  If I right click the remote system from the main virt-manager window and select new, interactivity with the running VM is shot.  The mouse barely registers with the new VM dialog sitting at the first screen.  If instead, I right click on localhost and select new, then change the connection to the remote system from the first screen, the local VM interactivity stutters while switching over, but then behaves ok.

Version-Release number of selected component (if applicable):
virt-manager-0.8.4-5.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1. see description
2.
3.
  
Actual results:
Don't try to multitask, operating on remote VM kills interactivity with local VMs.

Expected results:
Operations on remote, high latency systems don't affect interactivity with other VMs.

Additional info:

Comment 2 RHEL Program Management 2010-06-24 17:12:58 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 3 Alex Williamson 2010-06-24 17:17:11 UTC
Once I get done creating the VM on the remote system, the dialog hangs long
enough for the window manager to prompt me that it's not responding.  If I
choose wait rather than force close, it eventually continues :(

Comment 4 RHEL Program Management 2010-07-15 14:55:20 UTC
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 6 Cole Robinson 2010-11-23 20:22:08 UTC
*** Bug 617388 has been marked as a duplicate of this bug. ***

Comment 7 Suzanne Logcher 2011-03-28 21:09:57 UTC
Since RHEL 6.1 External Beta has begun, and this bug remains 
unresolved, it has been rejected as it is not proposed as an 
exception or blocker.

Red Hat invites you to ask your support representative to 
propose this request, if appropriate and relevant, in the 
next release of Red Hat Enterprise Linux.

Comment 9 Cole Robinson 2011-09-27 15:16:50 UTC
The 6.2 situation is better than 6.1, but there are still some areas where we can greatly improve here, so deferring to 6.3.

Comment 11 Cole Robinson 2011-12-09 23:14:02 UTC
We currently have reduced capacity for virt-manager/virtinst fixes in RHEL. This issue is something that would be good to fix for RHEL, but the 'fix' covers a lot of areas and would require a rebase. Setting conditional nack for now, might have a better idea about likeliness of fixing for 6.3 come january

Comment 12 Dave Allan 2012-01-13 22:05:39 UTC
If there's any low hanging fruit here, it'd certainly be nice to have virt-manager perform better over WAN links.  We should do what we can incrementally.

Comment 13 Cole Robinson 2012-02-03 20:12:53 UTC
Okay, facing the reality that virt-manager probably won't be rebased again in RHEL6, i'm moving this bug to RHEL7. Solving this (in as much as we can solve it) will require some arch changes in a few places that just aren't backportable.

Comment 14 Cole Robinson 2014-02-11 20:48:00 UTC
FWIW the next upstream virt-manager release will be many orders of magnitude better here, but still the changes aren't backportable. If we rebase in RHEL7.1, I'd say this bug is 'fixed' but obviously QE should confirm.

Comment 17 CongDong 2014-09-11 07:31:06 UTC
Test with :
virt-manager-1.1.0-1.el7.noarch

Steps:

1. Prepare two hosts A and B, on host A, set the latency to a high value
# tc qdisc add dev eno1 root netem delay 175ms 15ms
so the latency will be a value 160ms~190ms

2. open virt-manager on host A
# virt-manager

3. double click a guest and open a guest console

4. Add remote connection to host B with virt-manager
Click File -> Add a new connection -> File ip of host B on "Hostname" -> Connect
Then input password, make sure can see the guest list of host B under the connection

5. Right click on connection with host B -> New

Result:
As the latency is high, the dialog will come out after a little while, before it comes out, try to do some drag operation(or move the icon on desktop with mouse) on guest console which is opend in step 3, but nothing happens, the guest console hang there, after the dialog comes out, it works.


As the result, the problem is not fixed.
So set ASSIGNED

Comment 18 Cole Robinson 2014-09-15 14:16:12 UTC
Yeah there is a similar report in Fedora:

https://bugzilla.redhat.com/show_bug.cgi?id=1123266

Compared to how virt-manager was in previous versions, things are massively improved, particularly for post-connection operations.

However I added a bunch of patches upstream that greatly improve the connection speed and app interactivity when opening a connection (starts at commit f0ab05410968e63daed1b2b7e7328d5816a3e1d6) However the new patches are also invasive and haven't seen a lot of testing yet so I don't recommend just pulling them into RHEL

My suggestion would be that we still mark this bug as fixed since the original report of the app being unusable for remote connections isn't true anymore. If users/customers report bugs about the further remaining issues then we can consider addressing those, but getting to a point where remote connections cause zero interactivity issues will probably never happen, so there will always be more issues to find.

Comment 19 CongDong 2014-09-18 09:14:18 UTC
(In reply to Cole Robinson from comment #18)
> Yeah there is a similar report in Fedora:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1123266
> 
> Compared to how virt-manager was in previous versions, things are massively
> improved, particularly for post-connection operations.

Which version should I compare with virt-manager-1.1.0-1.el7 can show the increase of connection operation.

I also test with virt-manager-1.1.0-2.el7, with same steps in comment 17, but get the same result.

Comment 20 Cole Robinson 2014-09-18 12:41:02 UTC
I meant compared to RHEL7.0 virt-manager version 0.10 something or other. Compare remote connection speed post connection, when clicking around in the UI, launching the new VM wizard, etc.

But thinking about this some more, there's some low hanging fruit patches that will make connections not hang as much, and they should be fairly straightforward and safe backports:

commit f0ab05410968e63daed1b2b7e7328d5816a3e1d6
Author: Cole Robinson <crobinso>
Date:   Thu Sep 11 17:44:59 2014 -0400

    connection: Do mediadev setup in a thread

commit f36d2ed9607b2530864ba331379f3167ea5ab86c
Author: Cole Robinson <crobinso>
Date:   Fri Sep 12 08:57:13 2014 -0400

    domain: Remove the description/title hotplug hacks

commit b398a46e9bfcd192d9286d6edc81deade77bb1a8
Author: Cole Robinson <crobinso>
Date:   Thu Sep 11 18:01:41 2014 -0400

    domain: Cache has_managed_save value

commit 1cf4671f851ba054bbb527859c8890b2af250f2b
Author: Cole Robinson <crobinso>
Date:   Fri Sep 12 09:12:10 2014 -0400

    domain: Prime the XML cache when object is created

Comment 23 zhoujunqin 2014-09-29 08:34:06 UTC
Seen above comments, i try to verify this issue in the way: comparing two virt-manager versions showing:
old version: virt-manager-0.10.0-20.el7.noarch
new version: virt-manager-1.1.0-3.el7.noarch

steps:
1. Prepare two hosts A and B
on host A:
set the latency to a high value
# tc qdisc add dev eno1 root netem delay 175ms 15ms
so the latency will be a value 160ms~190ms
# ping www.baidu.com
PING www.a.shifen.com (61.135.169.121) 56(84) bytes of data.
64 bytes from 61.135.169.121: icmp_seq=1 ttl=53 time=175 ms
64 bytes from 61.135.169.121: icmp_seq=2 ttl=53 time=190 ms
64 bytes from 61.135.169.121: icmp_seq=3 ttl=53 time=203 ms
64 bytes from 61.135.169.121: icmp_seq=4 ttl=53 time=192 ms
64 bytes from 61.135.169.121: icmp_seq=5 ttl=53 time=194 ms

2. open virt-manager,double click a guest and open a guest console
# virt-manager

3. Add remote connection to host B with virt-manager
Click File -> Add a new connection -> File ip of host B on "Hostname" -> Connect
Then input password, make sure can see the guest list of host B under the connection

4. Right click on connection with host B -> New

Result:
As the latency is high, first compare the dialog coming's time:
time(virt-manager-0.10.0-20.el7.noarch) is much larger than time(virt-manager-1.1.0-3.el7.noarch).

secondly, compare operation( before ‘New WM’ dialog comes out, try to do some drag operation(move the icon on desktop with mouse)) on the the opening guest taking effect time on hostA:
operation-taking-effact-time(virt-manager-0.10.0-20.el7.noarch) is much larger than operation-taking-effact-time(virt-manager-1.1.0-3.el7.noarch).

As a result, move this bug from ON_QA to VERIFIED.

Comment 25 errata-xmlrpc 2015-03-05 10:02:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0427.html