Bug 757608

Summary: virt-manager tries to connect to qemu and does not get connected at all times.
Product: [Fedora] Fedora Reporter: Balaji G <balajig81>
Component: virt-managerAssignee: Cole Robinson <crobinso>
Status: CLOSED WORKSFORME QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 16CC: berrange, crobinso, dougsland, dpierce, hbrock, jforbes, matt, nandanlbhat, ronderode, Shurik, tcamuso, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-01-27 18:09:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Logging output from libvirt
none
systemctl status libvirtd.service output
none
debug output of virsh
none
virt-manager --debug output
none
ps axwww output none

Description Balaji G 2011-11-28 03:16:54 UTC
Description of problem:
On starting virt-manager, the dialog box appears and it prompts for the password and once the password is entered, it goes into connecting state and stays there. The virt-manager after few seconds becomes unresponsive and i got to close the application. This happens in the latest fedora 16 stock kernel as well in the vanilla kernel 3.1.2. virt-manager worked the first time when i upgraded to the latest kernel and then once i closed it and reopened the problem started happening. I tried manually starting the libvirtd daemon too and it starts but the problem still exists. On addition to all these when i want to shutdown my system, the system logs out of my DE which is XFCE and it goes to Plymoth screen and it stays there it does not shutdown completely.

Starting virt-manager from terminal does not throw any verbose output either so not sure what's happening.

Version-Release number of selected component (if applicable):
virt-manager-common-0.9.0-7.fc16.noarch
virt-manager-0.9.0-7.fc16.noarch
Linux localhost.localdomain 3.1.2-1.fc16.x86_64 #1 SMP Tue Nov 22 09:00:57 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux


How reproducible:


Steps to Reproduce:
1.start virt-manager
2.Enter Password and create a new VM and once the required options are given to create the VM, close the options window and virt-manager before the VM installation starts.
3.start virt-manager again and enter the password, it goes into connecting state
  
Actual results:


Expected results:


Additional info:

Comment 1 Nandan Bhat 2011-12-12 07:25:54 UTC
I am seeing somewhat similar behaviour on my system.

Virt-Manager remains frozen in perpetuity with a message: localhost(QEMU) - Connecting...

With debugging messages enabled, I get the messages as in virt-manager-debug.txt (attached)

With virsh, I get the messages as in virsh-debug.txt (attached).

I have enabled logging in /etc/libvirt/libvirt.conf with the line
log_outputs="1:file:/var/log/libvirt/libvirtd.log"

libvirtd.log-201112121232 is attached.

There is a line in /var/log/messages 
Dec 12 12:27:37 nandan libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: No such file or directory

I have also observed the delayed shutdown behaviour. I have been shutting down using the hardware power button whenever that happens (intermittently).

Comment 2 Nandan Bhat 2011-12-12 07:30:07 UTC
Created attachment 545605 [details]
Logging output from libvirt

Comment 3 Nandan Bhat 2011-12-12 07:31:02 UTC
Created attachment 545606 [details]
systemctl status libvirtd.service output

Comment 4 Nandan Bhat 2011-12-12 07:31:56 UTC
Created attachment 545607 [details]
debug output of virsh

Comment 5 Nandan Bhat 2011-12-12 07:32:38 UTC
Created attachment 545608 [details]
virt-manager --debug output

Comment 6 Alexander Dyadyun 2011-12-16 05:22:02 UTC
Similar problem. 

After some update and reboot I can not connect to libvirt neither virt-manager nor virsh. Restart of libvirt service or restart computer does not help.

And, yes, computer can't correctly shutdown.

My workaround.
1. Backup contents of /etc/libvirt
2. Total erase of virt-manager/qemu*/libvirt* (its overkill, but I am angry :) )
3. reboot
4. Install qemu/virt-manager/libvirt
5. Start virt-manager
6. ???
7. PROFIT!!! (I can manage my virtual machines from virt-manager/virsh and can correctly shutdown host machine.)

Comment 7 Alexander Dyadyun 2011-12-16 05:29:12 UTC
Sorry... my soft config.

Linux Host.bfg-10k.ru 3.1.5-2.fc16.x86_64 #1 SMP Mon Dec 12 21:25:51 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

virt-manager.noarch 0.9.0-7.fc16
virt-manager-common.noarch 0.9.0-7.fc16

libvirt.x86_64 0.9.6-2.fc16
libvirt-client.x86_64 0.9.6-2.fc16
libvirt-python.x86_64 0.9.6-2.fc16

Comment 8 Cole Robinson 2012-01-18 16:02:25 UTC
Hmm, strange. Anyone still seeing this with latest f16 packages? Can everyone confirm that as root, virsh --connect qemu:///system is hanging as well?

Comment 9 Tony Camuso 2012-01-19 17:42:04 UTC
Yes, I am seeing virt-manager hang with all the latest f16 updates.

The following command ...

# virsh --connect qemu:///system

... gives the following warning.

# virsh -vv --connect qemu:///system

Comment 10 Ron 2012-01-23 22:48:55 UTC
I have similar problems on Red Hat Enterprise Linux 6.2 which is running under Vmware Workstation 8.0. On version 6.1 everything works fine.

Linux maui.example.com 2.6.32-220.2.1.el6.x86_64 #1 SMP Tue Dec 13 16:21:34 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

qemu-kvm-0.12.1.2-2.209.el6_2.1.x86_64
libvirt-python-0.9.4-23.el6_2.4.x86_64
libvirt-0.9.4-23.el6_2.4.x86_64
libvirt-client-0.9.4-23.el6_2.4.x86_64
virt-manager-0.9.0-7.el6.x86_64

Virtual Machine Manager and virsh are unable to open a connection to the libvirt management daemon. So "virsh --connect qemu:///system" is hanging as well.

When initiation a connection the Virtual Machine Manager and virsh stay in a state of "connection" which doesn't finish. ("localhost (QEMU) - connecting...")

Another sympton is that after the previous has occurred, the server won't shutdown properly after initiating a shutdown and hangs at the following state: "Restarting...Running guests on default URI:"

When restarting the libvirtd service before shutting down, the server shuts down properly and doesn't hang on "Restarting...Running guests on default URI:"

This problem occurs as well as with the default RHEL 6.2 installation as with currently updated qemu, virt-manager and libvirt versions

The following reinstall method didn't resolve the problem:
"yum groupremove virtualization "virtualization client" "virtualization tools" "virtualization platform"
"yum groupinstall virtualization "virtualization client" "virtualization tools" "virtualization platform"

Comment 11 Cole Robinson 2012-01-27 15:31:15 UTC
Can anyone provide the following info:

- Desktop environment you are using (gnome, kde, lxde, etc)
- While virsh --connect qemu:///system is hanging, please provide output of 'ps axwww' run from another terminal (note to self: looking for polkit running, or udevadm running)
- Does running system-config-printer, and selecting the menu entry 'Server->Settings' similarly hang? That is also using polkit

Comment 12 Tony Camuso 2012-01-27 15:41:07 UTC
I'm running Gnome in 2D mode. 

I've been keeping up with updates, and it seems that a recent yum update sequence has somewhat mitigated the problem. 

I am now able to connect to qemu:///system from virt-manager, but it occasionally drops off and I have to reconnect. At least it's not hanging anymore.

Comment 13 Cole Robinson 2012-01-27 18:09:36 UTC
Thanks Tony, the dropped connection issue is another known issue that I'm working on.

I'll close this bug as WORKSFORME, but if anyone else is still seeing this problem, please reopen this bug, providing the info in comment #11.

Comment 14 Matt 2012-02-21 16:00:20 UTC
Created attachment 564733 [details]
ps axwww output

Requested in Comment 11

Comment 15 Matt 2012-02-21 16:02:24 UTC
Running FC16 x86_64 as guest in VMware Workstation 8 (with virtualised Intel VT-x/EPT)

Desktop env is fallback gnome

Attached output of ps axwww above

system-config-printer polkit works as expected.

I have tried setting the libvirtd auth settings to none - it still hangs on connect.

Comment 16 Cole Robinson 2012-02-22 16:14:28 UTC
Matt, what's the output of virsh --connect qemu:///system from a terminal?

Comment 17 Matt 2012-02-22 19:36:50 UTC
Hi Cole.

There is no output when running that command (even with --debug).  I am running the command as root.

Matt

Comment 18 Matt 2012-02-22 20:12:03 UTC
Here's a backtrace of virsh as it tries to connect.  This means nothing to me, but will hopefully be of some use to someone!

#0  0x00007f7b902c6ab8 in __GI___poll (fds=0x7fff7fd4ee20, nfds=2, timeout=<optimized out>)
    at ../sysdeps/unix/sysv/linux/poll.c:83
#1  0x00007f7b92a35ab5 in virNetClientIOEventLoop (client=0x7f7b92f71010, thiscall=0x15ea470) at rpc/virnetclient.c:899
#2  0x00007f7b92a36f04 in virNetClientIO (thiscall=0x15ea470, client=0x7f7b92f71010) at rpc/virnetclient.c:1136
#3  virNetClientSend (client=0x7f7b92f71010, msg=<optimized out>, expectReply=true) at rpc/virnetclient.c:1218
#4  0x00007f7b92a37678 in virNetClientProgramCall (prog=0x15edc10, client=0x7f7b92f71010, serial=0, proc=66, 
    args_filter=0x7f7b90302940 <__GI_xdr_void>, args=0x0, ret_filter=0x7f7b92a324a0 <xdr_remote_auth_list_ret>, 
    ret=0x7fff7fd4f180) at rpc/virnetclientprogram.c:291
#5  0x00007f7b92a1bb23 in call (conn=<optimized out>, priv=0x15eb5f0, flags=<optimized out>, proc_nr=66, 
    args_filter=<optimized out>, args=<optimized out>, ret_filter=0x7f7b92a324a0 <xdr_remote_auth_list_ret>, 
    ret=0x7fff7fd4f180 "") at remote/remote_driver.c:4111
#6  0x00007f7b92a29a30 in remoteAuthenticate (authtype=0x0, auth=0x7f7b92db64c0, priv=0x15eb5f0, conn=0x15edaf0)
    at remote/remote_driver.c:2405
#7  doRemoteOpen (conn=0x15edaf0, priv=0x15eb5f0, auth=0x7f7b92db64c0, flags=0) at remote/remote_driver.c:649
#8  0x00007f7b92a2de5b in remoteOpen (conn=0x15edaf0, auth=0x7f7b92db64c0, flags=<optimized out>)
    at remote/remote_driver.c:828
#9  0x00007f7b929ee1b1 in do_open (name=0x15e86a0 "qemu:///system", auth=0x7f7b92db64c0, flags=0) at libvirt.c:1073
#10 0x00007f7b929f02a6 in virConnectOpenAuth (name=0x15e86a0 "qemu:///system", auth=0x7f7b92db64c0, flags=0)
    at libvirt.c:1299
#11 0x0000000000409ca1 in vshInit (ctl=0x7fff7fd4f610) at virsh.c:15508
#12 main (argc=<optimized out>, argv=0x7fff7fd4f7b8) at virsh.c:16205

Comment 19 Cole Robinson 2012-02-22 21:19:37 UTC
No output as in it just exits? Doesn't connect at all? Any different if trying that command as a regular user?


If virsh isn't working, please open a new libvirt bug to track the issue there.

Comment 20 Matt 2012-02-22 21:41:59 UTC
Sorry - It hangs, until such time as you ^C - Same with normal user.

Will open a new bug as directed, thanks.

Comment 21 Matt 2012-02-22 21:59:49 UTC
Libvirt bug raised: Bug 796451