Bug 1060524

something wrong with vdsm there, it restarts in a loop by getting SIGTERM, not sure why yet. is it reproducible or relates only to that specific host?

Could you do `su - vdsm -s /bin/bash`
and then run /usr/share/vdsm/vdsm
to see why vdsm resets itself so often?

BTW, is sysvinit-tools installed on your host?

it just gets stuck like this:
[root@cinteg07 ~]# su - vdsm -s /bin/bash
-bash-4.1$ /usr/share/vdsm/vdsm
(PID: 26271) I am the actual vdsm 4.14.1-21.git291df0d.el6 cinteg07.ci.lab.tlv.redhat.com (2.6.32-431.el6.x86_64)
Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x284b320>>)
Run and protect: registerDomainStateChangeCallback, Return response: None
Starting up MOM
Setting channels' timeout to 30 seconds.
Starting VM channels listener thread.
trying to connect libvirt


and regarding to your question > yes, it is installed.

Would you make sure that `service vdsmd stop` before running a new vdsm instance?

When you run Vdsm manually, is it suddenly responsive to Engine's polling?

I stopped vdsmd but it could be that supervdsm resumed it.
how can I disable supervdsm?

supervdsm does not restart vdsm.

I do not understand your report then: with `service vdsmd start`, vdsm constantly crashes, and from the command line all is well?

When you run Vdsm manually, is it suddenly responsive to Engine's polling?

you get alot of:
libvir: XML-RPC error : authentication failed: Authorization requires authentication but no agent is available.
libvir: XML-RPC error : authentication failed: Authorization requires authentication but no agent is available.
libvir: XML-RPC error : authentication failed: Authorization requires authentication but no agent is available.
libvir: XML-RPC error : authentication failed: Authorization requires authentication but no agent is available.
libvir: XML-RPC error : authentication failed: Authorization requires authentication but no agent is available.
Vm's recovery failed
Traceback (most recent call last):
  File "/usr/share/vdsm/clientIF.py", line 390, in _recoverExistingVms
    caps.CpuTopology().cores())
  File "/usr/share/vdsm/caps.py", line 127, in __init__
    self._topology = _getCpuTopology(capabilities)
  File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 832, in __call__
    value = self.func(*args)
  File "/usr/share/vdsm/caps.py", line 147, in _getCpuTopology
    capabilities = _getCapsXMLStr()
  File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 832, in __call__
    value = self.func(*args)
  File "/usr/share/vdsm/caps.py", line 141, in _getCapsXMLStr
    return libvirtconnection.get().getCapabilities()
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 146, in get
    conn = utils.retry(libvirtOpenAuth, timeout=10, sleep=0.2)
  File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 949, in retry
    return func()
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
    if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirtError: authentication failed: Authorization requires authentication but no agent is available.

trying to connect libvirt
libvir: XML-RPC error : authentication failed: Authorization requires authentication but no agent is available.

familiar?

(In reply to Yaniv Bronhaim from comment #10)
> you get alot of:
> libvir: XML-RPC error : authentication failed: Authorization requires
> authentication but no agent is available.
> libvir: XML-RPC error : authentication failed: Authorization requires
> authentication but no agent is available.
> libvir: XML-RPC error : authentication failed: Authorization requires
> authentication but no agent is available.
> libvir: XML-RPC error : authentication failed: Authorization requires
> authentication but no agent is available.
> libvir: XML-RPC error : authentication failed: Authorization requires
> authentication but no agent is available.
> Vm's recovery failed
> Traceback (most recent call last):
>   File "/usr/share/vdsm/clientIF.py", line 390, in _recoverExistingVms
>     caps.CpuTopology().cores())
>   File "/usr/share/vdsm/caps.py", line 127, in __init__
>     self._topology = _getCpuTopology(capabilities)
>   File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 832, in
> __call__
>     value = self.func(*args)
>   File "/usr/share/vdsm/caps.py", line 147, in _getCpuTopology
>     capabilities = _getCapsXMLStr()
>   File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 832, in
> __call__
>     value = self.func(*args)
>   File "/usr/share/vdsm/caps.py", line 141, in _getCapsXMLStr
>     return libvirtconnection.get().getCapabilities()
>   File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line
> 146, in get
>     conn = utils.retry(libvirtOpenAuth, timeout=10, sleep=0.2)
>   File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 949, in retry
>     return func()
>   File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
>     if ret is None:raise libvirtError('virConnectOpenAuth() failed')
> libvirtError: authentication failed: Authorization requires authentication
> but no agent is available.
> 
> trying to connect libvirt
> libvir: XML-RPC error : authentication failed: Authorization requires
> authentication but no agent is available.
> 
> familiar?

Hi,

Can you please check if virsh tool works? 
Also, I see a report [1] about "Authorization requires authentication but no agent is available" although might be not related to this one.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=872166

Answering myself, virsh works:

# virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # list
Please enter your authentication name: vdsm@rhevh
Please enter your password: 
 Id    Name                           State
----------------------------------------------------

virsh # list --all
 Id    Name                           State
----------------------------------------------------

virsh # net-list
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     no            yes
vdsm-rhevm           active     yes           yes
vdsm-virbr0          active     yes           yes

According with scenario looks like the polkit dir doesn't have any rule so vdsm will show us:

"trying to connect to libvirt"

# ls /var/lib/polkit-1/localauthority/10-vendor.d
# 

# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.5 (Santiago)


From our vdsm.spec:
=======================
# Required paths
%if 0%{?fedora} >= 18
%global _polkitdir %{_datadir}/polkit-1/rules.d
%else
%global _polkitdir %{_localstatedir}/lib/polkit-1/localauthority/10-vendor.d
%endif


and

%install
# Install the polkit for libvirt
%if 0%{?fedora} >= 18
install -Dm 0644 vdsm/vdsm-libvirt-access.rules \
                 %{buildroot}%{_polkitdir}/10-vdsm-libvirt-access.rules
%else
install -Dm 0644 vdsm/vdsm-libvirt-access.pkla \
                 %{buildroot}%{_polkitdir}/10-vdsm-libvirt-access.pkla

More evidences:
# rpm -qa | grep -i vdsm
vdsm-python-4.14.1-24.gitb4b06ab.el6.x86_64
vdsm-hook-fileinject-4.12.0-120.git83a430b.el6.noarch
vdsm-xmlrpc-4.14.1-24.gitb4b06ab.el6.noarch
vdsm-4.14.1-24.gitb4b06ab.el6.x86_64
vdsm-python-zombiereaper-4.14.1-24.gitb4b06ab.el6.noarch
vdsm-cli-4.14.1-24.gitb4b06ab.el6.noarch


# yum downloader vdsm
# rpm2cpio ./vdsm-4.14.1-24.gitb4b06ab.el6.x86_64.rpm | cpio -div
# cd vdsm*
# find . -name *pkla*
No polkit available in the package.

- vdsm not starting

Now:
=========
- scp 10-vdsm-libvirt-access.pkla to host
# cp 10-vdsm-libvirt-access.pkla /var/lib/polkit-1/localauthority/10-vendor.d
- started vdsm manually
# service vdsmd status
VDS daemon server is running

# vdsClient -s 0 getVdsCaps (works)

# su - vdsm -s /bin/bash
-bash-4.1$ /usr/share/vdsm/vdsm 
(PID: 17731) I am the actual vdsm 4.14.1-24.gitb4b06ab.el6 cinteg07.ci.lab.tlv.redhat.com (2.6.32-431.el6.x86_64)
Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF instance at 0x2170320>>)
Run and protect: registerDomainStateChangeCallback, Return response: None
Starting up MOM
Setting channels' timeout to 30 seconds.
trying to connect libvirt (Not more blocking on this)
Starting VM channels listener thread.

still I don't understand how some managed to work with vdsm after the merge of the with_systemd patch [1].. its almost a year since then

[1] http://gerrit.ovirt.org/#/c/12086

Verified on rhevm-3.4.0-0.3.master.el6ev.noarch,host deployment for host with vdsm-4.14.2-0.4.el6ev.x86_64.rpm works fine.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0504.html

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days