Red Hat Bugzilla – Bug 1324016
"Failed to Establish Libvirt Connection" on RHEV-H 7.2 for RHEV 3.6.4
Last modified: 2016-07-27 00:28:44 EDT
Description of problem: "Failed to Establish Libvirt Connection" on RHEV-H 7.2 for RHEV 3.6.4, it is hard to reproduced. Version-Release number of selected component (if applicable): Red Hat Enterprise Virtualization Hypervisor release 7.2 (20160328.0.el7ev) ovirt-node-3.6.1-8.0.el7ev.noarch libvirt-1.2.17-13.el7_2.3.x86_64 vdsm-4.17.23.2-1.el7ev.noarch How reproducible: 5%, it is hard to reproduced. Steps to Reproduce: 1. TUI clean installed rhevh successful 2. Login TUI 3. Setup host name 4. Setup network via dhcp 5. Please see attachment for more detail. Actual results: "Failed to Establish Libvirt Connection" on RHEV-H 7.2 for RHEV 3.6.4, Expected results: without such error Additional info: # systemctl status libvirtd vdsmd ● libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/libvirtd.service.d └─unlimited-core.conf Active: failed (Result: start-limit) since Tue 2016-04-05 07:16:49 UTC; 8min ago Docs: man:libvirtd(8) http://libvirt.org Process: 18450 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=6) Main PID: 18450 (code=exited, status=6) Apr 05 07:16:49 localhost systemd[1]: Failed to start Virtualization daemon. Apr 05 07:16:49 localhost systemd[1]: Unit libvirtd.service entered failed state. Apr 05 07:16:49 localhost systemd[1]: libvirtd.service failed. Apr 05 07:16:49 localhost systemd[1]: libvirtd.service holdoff time over, scheduling restart. Apr 05 07:16:49 localhost systemd[1]: start request repeated too quickly for libvirtd.service Apr 05 07:16:49 localhost systemd[1]: Failed to start Virtualization daemon. Apr 05 07:16:49 localhost systemd[1]: Unit libvirtd.service entered failed state. Apr 05 07:16:49 localhost systemd[1]: libvirtd.service failed. ● vdsmd.service - Virtual Desktop Server Manager Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled) Active: inactive (dead) Apr 05 07:16:48 localhost systemd[1]: Dependency failed for Virtual Desktop Server Manager. Apr 05 07:16:48 localhost systemd[1]: Job vdsmd.service/start failed with result 'dependency'. Apr 05 07:16:48 localhost systemd[1]: Dependency failed for Virtual Desktop Server Manager. Apr 05 07:16:48 localhost systemd[1]: Job vdsmd.service/start failed with result 'dependency'. Apr 05 07:16:49 localhost systemd[1]: Dependency failed for Virtual Desktop Server Manager. Apr 05 07:16:49 localhost systemd[1]: Job vdsmd.service/start failed with result 'dependency'. Apr 05 07:16:49 localhost systemd[1]: Dependency failed for Virtual Desktop Server Manager. Apr 05 07:16:49 localhost systemd[1]: Job vdsmd.service/start failed with result 'dependency'. Apr 05 07:16:49 localhost systemd[1]: Dependency failed for Virtual Desktop Server Manager. Apr 05 07:16:49 localhost systemd[1]: Job vdsmd.service/start failed with result 'dependency'. [root@localhost admin]# # systemctl restart libvirtd Job for libvirtd.service failed because the control process exited with error code. See "systemctl status libvirtd.service" and "journalctl -xe" for details. [root@localhost admin]# # systemctl status supervdsmd ● supervdsmd.service - Auxiliary vdsm service for running helper functions as root Loaded: loaded (/usr/lib/systemd/system/supervdsmd.service; static; vendor preset: enabled) Active: failed (Result: start-limit) since Tue 2016-04-05 07:25:57 UTC; 22s ago Process: 18679 ExecStart=/usr/share/vdsm/daemonAdapter /usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock (code=killed, signal=TERM) Main PID: 18679 (code=killed, signal=TERM) Apr 05 07:25:57 localhost systemd[1]: Started Auxiliary vdsm service for running helper functions as root. Apr 05 07:25:57 localhost systemd[1]: Starting Auxiliary vdsm service for running helper functions as root... Apr 05 07:25:57 localhost systemd[1]: Stopping Auxiliary vdsm service for running helper functions as root... Apr 05 07:25:57 localhost systemd[1]: start request repeated too quickly for supervdsmd.service Apr 05 07:25:57 localhost systemd[1]: Failed to start Auxiliary vdsm service for running helper functions as root. Apr 05 07:25:57 localhost systemd[1]: Unit supervdsmd.service entered failed state. Apr 05 07:25:57 localhost systemd[1]: supervdsmd.service failed. [root@localhost admin]#
Created attachment 1143772 [details] failed to Establish libvirt connection
Created attachment 1143774 [details] /var/log/*.*
Can you please retry this with RHEV 3.6.5? And can VMs be spawned?
(In reply to Fabian Deutsch from comment #3) > Can you please retry this with RHEV 3.6.5? > The bug is hard to reproduce, and I can't reproduce this issue on rhev-hypervisor7-7.2-20160406.0 at present. > And can VMs be spawned? I will give result ASAP once reproduced.
Can you please also post the contents of /etc/pki? It looks like the clock on the system was out of sync when the vdsm server certificate was generated, and libvirt is refusing it (since the date is probably in the future), so libvirt doesn't start, and vdsm can't communicate with it.
(In reply to Ryan Barry from comment #5) > Can you please also post the contents of /etc/pki? > > It looks like the clock on the system was out of sync when the vdsm server > certificate was generated, and libvirt is refusing it (since the date is > probably in the future), so libvirt doesn't start, and vdsm can't > communicate with it. The original env has been destroyed due to new build testing, but I still will tries to provide the contents of /etc/pki if I can reproduce this issue.
Created attachment 1165049 [details] all log info Reproduce this issue on RHEV-H 7.2 for RHEV 3.6.7 (rhev-hypervisor7-7.2-20160602.0.iso ) + rhev-hypervisor7-7.2-20160602.0.iso. Attach all log info for reference. /var/log/*.* /tmp/ovirt.log /etc/pki/* sosreport
Created attachment 1165050 [details] 0602
Chen, can you please retry this on one of the more recent 3.6.z builds?
(In reply to Fabian Deutsch from comment #9) > Chen, can you please retry this on one of the more recent 3.6.z builds? ok, the bug is hard to reproduce, but I will try and let you know the result ASAP.
(In reply to Fabian Deutsch from comment #9) > Chen, can you please retry this on one of the more recent 3.6.z builds? Test version: rhev-hypervisor7-7.2-20160711.0 ovirt-node-3.6.1-13.0.el7ev.noarch After several times testing on different types of machine, can't reproduce this bug.
(In reply to shaochen from comment #11) > (In reply to Fabian Deutsch from comment #9) > > Chen, can you please retry this on one of the more recent 3.6.z builds? > > Test version: > rhev-hypervisor7-7.2-20160711.0 > ovirt-node-3.6.1-13.0.el7ev.noarch > > After several times testing on different types of machine, can't reproduce > this bug. Thanks shaochen, closing this bug for now, fell free to re-open anytime.