Bug 1019360 - vdsm not starts properly after clean installation.
Summary: vdsm not starts properly after clean installation.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: unspecified
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.3.0
Assignee: Yaniv Bronhaim
QA Contact: Leonid Natapov
URL:
Whiteboard: infra
Depends On:
Blocks: 3.3snap2
TreeView+ depends on / blocked
 
Reported: 2013-10-15 14:48 UTC by Leonid Natapov
Modified: 2016-02-10 19:17 UTC (History)
9 users (show)

Fixed In Version: is22
Doc Type: Bug Fix
Doc Text:
VDSM did not start properly upon clean installation, which could lead to failure when deploying the hosted engine. This update changes the return method from the vdsmd start function, so VDSM starts properly.
Clone Of:
Environment:
Last Closed: 2014-01-21 16:18:35 UTC
oVirt Team: Infra
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
vdsm log file (32.93 KB, text/plain)
2013-10-15 14:49 UTC, Leonid Natapov
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:0040 0 normal SHIPPED_LIVE vdsm bug fix and enhancement update 2014-01-21 20:26:21 UTC
oVirt gerrit 20578 0 None MERGED Changing return method from vdsmd start function 2021-02-05 09:32:00 UTC

Description Leonid Natapov 2013-10-15 14:48:10 UTC
Description of problem:
vdsm not starts properly after clean installation.
when starting vdsm daemon for the first time getting following exception:
------------

[root@nott-vds1 ~]# service vdsmd start
Starting libvirtd daemon:                                  [  OK  ]
supervdsm start                                            [  OK  ]
vdsm: Running run_init_hooks                                      
vdsm: Running gencerts                                            
vdsm: Running reconfigure_sanlock                                  
vdsm: Running reconfigure_libvirt                                  
Stopping libvirtd daemon: [  OK  ]                                
Reconfiguration of libvirt is done.                                
 
To start working with the new configuration, execute:
'vdsm-tool libvirt-configure-services-restart'      
This will manage restarting of the following services:
libvirtd, supervdsmd                                  
 
diff: /etc/init/libvirtd.conf: No such file or directory
vdsm: Running syslog_available                          
vdsm: Running nwfilter                                  
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
raceback (most recent call last):                                                                            
  File "/usr/bin/vdsm-tool", line 143, in <module>                                                            
    sys.exit(main())                                                                                          
  File "/usr/bin/vdsm-tool", line 140, in main                                                                
    return tool_command[cmd]["command"](*args[1:])                                                            
  File "/usr/lib64/python2.6/site-packages/vdsm/tool/nwfilter.py", line 35, in main                          
    conn = libvirtconnection.get(None, False)
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 132, in get
    conn = utils.retry(libvirtOpenAuth, timeout=10, sleep=0.2)
  File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 903, in retry
    return func()
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
    if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
vdsm: failed to execute nwfilter, error code 1
Starting up vdsm daemon:
vdsm start                                                 [  OK  ]

---------

As a result /var/run/vdsm/pools directory not created and host can't connect to pool. Restart vdsm solves the problem.

Version-Release number of selected component (if applicable):
vdsm-4.13.0-0.2.beta1.el6ev.x86_64

How reproducible:
100%

Steps to Reproduce:
1.remove vdsm from the host
2.install vdsm
3.start vdsm daemon



Additional info:

Comment 1 Leonid Natapov 2013-10-15 14:48:43 UTC
vdsm.log attached

Comment 2 Leonid Natapov 2013-10-15 14:49:05 UTC
Created attachment 812567 [details]
vdsm log file

Comment 4 Barak 2013-10-16 12:40:25 UTC
If you install & start vdsm manually it explains simply what needs to be done in order to start it.

This applies only for manual work - through host-deploy it is done automatically.

Hence moving to CLOSE NOTABUG

Comment 6 Dan Kenigsberg 2013-10-16 13:23:00 UTC
Frankly, this is a dup of bug 1013371. But for ovirt-3.3.1 we should not attempt to backport the relevant patches from master. We'd better just re-introduce libvirtd restart after an actual configuration thereof.

Comment 8 Yaniv Bronhaim 2013-10-23 15:35:29 UTC
This is the same issue as reported in Bug 1013371. This was fixed and the exception won't appear with http://gerrit.ovirt.org/#/c/19884. 

This fix was not backported to ovirt-3.3, for that please ack rhevm-3.3 flag

Comment 9 Yaniv Bronhaim 2013-10-23 15:37:49 UTC
btw, Dan, I prefer not to re-introduce the libvirt restart. It's simple patch to merge in without any expected regressions that I can think of..

Comment 11 Yaniv Bronhaim 2013-10-27 11:51:47 UTC
Still needs appropriate acks to proceed.

Comment 12 Leonid Natapov 2013-11-17 14:37:33 UTC
is23. same error on vdsmd start.

vdsm-4.13.0-0.8.beta1.el6ev.x86_64
libvirt-0.10.2-29.el6.x86_64

Comment 13 Yaniv Bronhaim 2013-11-17 15:18:28 UTC
Please share copy paste of output, libvirt log, and status of libvirtd. It means during vdsmd service start you could not connect to libvirt. It can be many issues related to libvirt but this bug talks about failing the start and not move forward with the other pre-start tasks. If not, lets investigate 

please provide the information asap.

Comment 14 Leonid Natapov 2013-11-17 15:57:45 UTC
[root@green-vdsc libvirt]# initctl status libvirtd
initctl: Unknown job: libvirtd
[root@green-vdsc libvirt]# service vdsmd status
VDS daemon is not running
[root@green-vdsc ~]# service vdsmd start
Starting libvirtd daemon:                                  [  OK  ]
supervdsm start                                            [  OK  ]
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running reconfigure_sanlock
vdsm: Running reconfigure_libvirt
Stopping libvirtd daemon: [  OK  ]
Reconfiguration of libvirt is done.

To start working with the new configuration, execute:
'vdsm-tool libvirt-configure-services-restart'
This will manage restarting of the following services:
libvirtd, supervdsmd

diff: /etc/init/libvirtd.conf: No such file or directory
vdsm: Running prepare_transient_repository
vdsm: Running syslog_available
vdsm: Running nwfilter
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 143, in <module>
    sys.exit(main())
  File "/usr/bin/vdsm-tool", line 140, in main
    return tool_command[cmd]["command"](*args[1:])
  File "/usr/lib64/python2.6/site-packages/vdsm/tool/nwfilter.py", line 35, in main
    conn = libvirtconnection.get(None, False)
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 132, in get
    conn = utils.retry(libvirtOpenAuth, timeout=10, sleep=0.2)
  File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 903, in retry
    return func()
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
    if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
vdsm: failed to execute nwfilter, error code 1
vdsm start                                                 [FAILED]
------------------------------------------
after unsuccessful vdsdm start
libvirt log includes only the following:

013-11-17 15:54:48.847+0000: 30726: info : libvirt version: 0.10.2, package: 29.el6 (Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>, 2013-10-09-06:25:35, x86-026.build.eng.bos.redhat.com)
2013-11-17 15:54:48.847+0000: 30726: error : virNWFilterSnoopLeaseFileRefresh:1903 : open("/var/run/libvirt/network/nwfilter.ltmp"): No such file or directory

Comment 15 Leonid Natapov 2013-11-17 16:00:09 UTC
You can easily reproduce it by cleaning the host from vdsm and libvirt yum remove vdsm*,yum remove libvirt* and installing them again from is23.

Comment 16 Leonid Natapov 2013-11-18 10:52:53 UTC
this particluar bug was fixed so I am verifying this BZ. However vdsm still unable to start when starting it after fresh install but the reason is different. Opening another BZ.

Comment 17 errata-xmlrpc 2014-01-21 16:18:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0040.html


Note You need to log in before you can comment on or make changes to this bug.