Bug 1035368

Summary: virt-manager does not work without dbus-daemon
Product: Red Hat Enterprise Linux 6 Reporter: Albert Flügel <af>
Component: virt-managerAssignee: Martin Kletzander <mkletzan>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.5CC: acathrow, af, agajania, codong, dyuan, hyao, lcui, mkletzan, mzhan, tzheng, zsong
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-03-06 16:11:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Albert Flügel 2013-11-27 15:58:24 UTC
Description of problem:
Additionally to Bugs 872166 and 986365 reporting that the virsh / virt-manager
infrastructure does not work without an authentication agent now (in earlier
versions this was working) it does not work without dbus-daemon either.

The only way to successfully start virt-manager is to start it locally
on the machine, where libvirtd is running, in a GNOME or other session,
that starts dbus-daemon. No way to make all needed things work from remote.
It's more and more broken failing in different ways showing more and more
dependencies.

If it needs an authentication agent, why doesn't it start one.
If it needs dbus-daemon, it should start one. Maybe this works, but
within the first hours after a host system boot it doesn't.

Scenarios and how they fail:
1) ssh -l root remote-system (DISPLAY tunneling, or setting DISPLAY to
the remote system address does not matter) and start virt-manager:
ERROR popup, no password prompted:
--------------------------------------------------------------------------
Error starting Virtual Machine Manager: Failed to contact configuration server; some possible causes are that you need to enable TCP/IP networking for ORBit, or you have stale NFS locks due to a system crash. See http://projects.gnome.org/gconf/ for information. (Details -  1: Failed to get connection to session: Failed to connect to socket /tmp/dbus-92vjx9L6ca: Connection refused)

Traceback (most recent call last):
  File "/usr/share/virt-manager/virt-manager.py", line 383, in <module>
    main()
  File "/usr/share/virt-manager/virt-manager.py", line 315, in main
    config = virtManager.config.vmmConfig(appname, appversion, glade_dir)
  File "/usr/share/virt-manager/virtManager/config.py", line 98, in __init__
    self.conf.add_dir(self.conf_dir, gconf.CLIENT_PRELOAD_NONE)
GError: Failed to contact configuration server; some possible causes are that you need to enable TCP/IP networking for ORBit, or you have stale NFS locks due to a system crash. See http://projects.gnome.org/gconf/ for information. (Details -  1: Failed to get connection to session: Failed to connect to socket /tmp/dbus-92vjx9L6ca: Connection refused)
--------------------------------------------------------------------------
Notes: Where does it have this /tmp/dbus-92vjx9L6ca ? There is no dbus-daemon running on the remote system. The libvirtd is of course running.

2) Going as normal user to the remote system with ssh and starting virt-manager the same happens immediately.

After a while of runtime (a day or so) the scenario 2) changes (no idea why) and the following is seen.

3) Going to the remote system via ssh without RSA authentication as non-root and starting virt-manager:
A popup appears to ask for a password. Whatever i enter here (my password or the root password), it does not work.
I get several popups:
--------------------------------------------------------------------------
SSH password dialog could not grab the keyboard input.
This might be caused by application such as screensaver, however it could also mean that someone may be eavesdropping on your session.
Either close the application which grabs the keyboard or log out and log in again to prevent this from happening.
Unable to open a connection to the libvirt management daemon.
--------------------------------------------------------------------------

In the syslog i see:
sshd[27382]: error: PAM: Authentication failure for root from myhost.domain.org

This applies for connections of type ssh. For connections of type qemu:///system
the known authentication daemon thing is barfed:
--------------------------------------------------------------------------
Libvirt URI is: qemu+ssh://root@virthost/system

Verify that:
 - The 'libvirtd' daemon has been started

authentication failed: Authorization requires authentication but no agent is available.

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/connection.py", line 1154, in _open_thread
    self.vmm = self._try_open()
  File "/usr/share/virt-manager/virtManager/connection.py", line 1138, in _try_open
    flags)
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
    if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirtError: authentication failed: Authorization requires authentication but no agent is available.
--------------------------------------------------------------------------

4) going to the host via ssh and using password-free RSA authentication (ssh-agent), then start virt-manager

I really get the virt-manager up and connecting virtual hosts with ssh-type
connections.
But as said above, i have to wait for this to work for a day or longer after
a host system boot.
Now virt-manager seems to start a dbus-daemon itself. Connections of type
qemu:///system still do not work at all.

5) Running virt-manager locally and using a connection of type ssh to the machine, where the virtual systems are running, with password-less (ssh-agent) RSA

Works (also only if password-less, entering passwords fails), but the console
cannot be connected without permitting the VNC connections from remote via
TCP, what i do not want and what is not the default.

Version-Release number of selected component (if applicable):
0.9.0-18.el6.x86_64

How reproducible:
see above

Actual results:
virt-manager works the way i (and quite likely not only me) want
it to be used only after a day or more and does never work, when
a password must be given, especially not with ssh type connections

Expected results:
virt-manager also works when started as root, immediately, when
the hosting system has booted (an not having to wait a day or more)
and also, when a password must be given.

Additional info:
Scenarios 3) and 4) succeeded and 2) did work supplying the root password in
the dialog until the RPM update to the current versions of kernel and
KVM/QEMU components, it also worked immediately after host system startup.
Scenario 1) succeeded after a day or so of host system uptime
until the RPM update.
Did not try 5) with the old RPM update status, but this is not relevant.
With the old RPM versions connections of tyep qemu:///system in general
worked as root, now don't seem to work anymore at all.
The previous RPM update status dated to the versions current around
May 2013

Comment 2 RHEL Program Management 2013-11-30 19:46:16 UTC
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 3 Martin Kletzander 2013-12-11 07:06:33 UTC
I tried booting cleanly into runlevel 3 and stopping messagebus service.  I used 'ssh -Y' to connect there as root ('ssh -X' works too).  No dbus daemon was running at the time and starting virt-maganer succeeded, starting/stopping a machine worked as well.  Running 'virt-manager --debug -c qemu+ssh://root@myhost/system' worked as well, only asking for the password appeared on the command line.  Removing --debug caused virt-manager to ask with a dialog and again, everything worked as expected.

Logging in there as a user, I could do everything except connecting to qemu:///system locally due to this error:

authentication failed: 
** (process:5256): WARNING **: Error connecting to bus: org.freedesktop.DBus.Error.NoServer: Failed to connect to socket /var/run/dbus/system_bus_socket: Connection refused

It's pretty obvious that I get this message when the system dbus is stopped.


Can you isolate one issue that you are facing and try to reproduce that also without virt-manager, in order to see where the problem lies?  Thank you.

Comment 4 tingting zheng 2014-01-06 08:50:56 UTC
(In reply to Martin Kletzander from comment #3)
> I tried booting cleanly into runlevel 3 and stopping messagebus service.  I
> used 'ssh -Y' to connect there as root ('ssh -X' works too).  No dbus daemon
> was running at the time and starting virt-maganer succeeded,
> starting/stopping a machine worked as well.  Running 'virt-manager --debug
> -c qemu+ssh://root@myhost/system' worked as well, only asking for the
> password appeared on the command line.  Removing --debug caused virt-manager
> to ask with a dialog and again, everything worked as expected.
> 

I tried to kill dbus dameon on remote machine manually,then ssh to that machine and try to launch virt-manager,I will get the same error as description:

Error starting Virtual Machine Manager: Failed to contact configuration server; some possible causes are that you need to enable TCP/IP networking for ORBit, or you have stale NFS locks due to a system crash. See http://projects.gnome.org/gconf/ for information. (Details -  1: Failed to get connection to session: Failed to connect to socket /tmp/dbus-LZlim7hIEG: Connection refused)

Comment 5 Albert Flügel 2014-01-10 12:42:50 UTC
The bad news is, that i don't have access anymore to the environment, where i experienced the described phenomena (due to a change of the employer). Probably i'll have the chance to create a similar setup elsewhere, but this may take some time.
It seems really weird to me, that the problem cannot be reproduced. It hit me notoriously throughout for many months on 2 machines and tracing and experimenting with some configuration parameters of ssh, authentication, desktop environment, virt-manager connection type etc did not change anything or give any clue.

Comment 6 Martin Kletzander 2014-01-10 14:50:54 UTC
Would you be OK with us closing this as INSUFFICIENT DATA for now as you'll be able to reopen it any time you have new info?

Comment 7 Albert Flügel 2014-01-12 16:03:05 UTC
Frankly, too me it feels a bit unpleasant, because it shows, that the issue is still not even understood at all.
However. If this is your preferred way to handle it, feel free.

Comment 8 Martin Kletzander 2014-01-13 08:08:44 UTC
(In reply to Albert Flügel from comment #7)
The problem is that we are unable to reproduce this issue and thus we don't know where the problem might be (it looks like a libvirt bug, I guess and it might be fixed already as well).  There just isn't enough information for us to search for it.  We'd be more than happy to help with the issue and fix it (if applicable), but for now we can only guess where the problem might be.

I'm closing this now as INSUFFICIENT_DATA, don't hesitate to reopen the bug anytime you (or anyone else) have anything related to this issue.  Thank you for understanding.

Comment 9 Martin Kletzander 2014-01-13 08:37:43 UTC
Ignore previous comment, I missed comment #4, sorry.

Comment 10 Martin Kletzander 2014-03-06 16:11:20 UTC
Inspecting this BZ again I'm closing it as INSUFFICIENT_DATA (again) with the following explanation:

This BZ was filed in order to address multiple issues and there was not one issue isolated we could focus on.  From the description it looks like dbus-daemon died (it was started earlier, hence the "/tmp/dbus-92vjx9L6ca" setting, probably left in some environment variable like DBUS_SESSION_BUS_ADDRESS or similar).  We couldn't isolate neither identify the issue and since the environment in which this happened doesn't exist (or is unacceesable) anymore, there is no way how to retrieve additional data.

In the case of initiating dbus-daemon; that should be done on a system level in order to properly share that daemon for the same session, but it is not clear that missing daemon was the reason (see previous paragraph).

So (similarly to previous comment #8) I'm closing this BZ.  It can be opened in case there is the possibility to request additional information from environment experiencing this (these) issues.  Thank you for understanding.

Comment 11 Aram Agajanian 2015-08-26 21:09:15 UTC
I found this bug because I was seeing problems in virt-manager similar to 

https://www.redhat.com/archives/virt-tools-list/2014-May/msg00031.html

It turns out that the problem was that I had multiple ifcfg files for DEVICEs em1, em2, and bond0.  This seemed to be confusing libvirtd.  The following lines were found in the logs

Aug 24 19:02:18 hostname libvirtd[1768]: (interface_definition):14: Extra content at the end of the document
                                                          </interface><interface type="ethernet" name="em1">
                                                          ------------^

It was surprising to me that issues with ifcfg files would cause the virt-manager GUI to become unstable.

Comment 12 Aram Agajanian 2015-09-04 20:36:24 UTC
I just tried unsuccessfully to re-create an installation where I can successfully run virt-manager after connecting with ssh -X .  (See Comment #11 for when I thought I had it working.)  Even after installing the "Server with GUI" environment group, I still get an unstable virt-manager GUI when logging in with ssh -X.