Bug 1010693

Summary:	When starting vdsmd service, service asks for authentication name and password and wait forever
Product:	[Retired] oVirt	Reporter:	Nir Soffer <nsoffer>
Component:	vdsm	Assignee:	Yaniv Bronhaim <ybronhei>
Status:	CLOSED DUPLICATE	QA Contact:	Haim <hateya>
Severity:	high	Docs Contact:
Priority:	high
Version:	3.4	CC:	abaron, acathrow, bazulay, danken, iheim, jsvarova, mbourvin, mgoldboi, yeylon
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	infra
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Whereas the status command of libvirt-guests init script returned "0" value when libvirt-guests service was stopped, LSB (Linux Standard Base) required a different value ("3") in such case. Consequently, other scripts relying on the return value could not distinguish whether the service was running or not. The libvirt-guests script has been fixed to conform with LSB and the "service libvirt-guests status" command now returns the correct value in the described scenario.	Story Points:	---
Clone Of:
Clones:	1011981 (view as bug list)		Environment:
Last Closed:	2013-09-29 20:47:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1011981
Bug Blocks:

Description Nir Soffer 2013-09-22 15:10:57 UTC

Description of problem:

When starting vdsmd service, the service ask for authentication name and password and passwords and then wait for ever for input.

Version-Release number of selected component (if applicable):

host: RHEL 6.4
libvirt: 0.10.2-18.el6_4.14.x86_64
vdsm: 1b9acee00ff12d

How reproducible:
Always

Steps to Reproduce:
1. Install vdsm 4.12.1 from ovirt stable repository
   http://resources.ovirt.org/releases/stable/rpm/EL/6/x86_64/
2. Start few vms from ovirt engine
3. yum remove vdsm*
4. Build vdsm from source and perfom basic installation
   cd ~/rpmbuild/RPMS 
   yum install --enablerepo=ovirt-beta x86_64/* noarch/vdsm-xml* noarch/vdsm-cli*
5. service vdsmd stop
6. service vdsmd start (vdsm fail to connect to libvirt)
7. service vdsmd stop
8. service vdsmd start (some warnings)
9. service vdsmd stop
10. service vdsmd start

Actual results:
vdsm ask for password and wait for input

Expected results:
start without waiting for input

Additional info:

The "Please enter authentiction name" messages comes from libvirt. We get the same message when running "virsh list".

From this point vdsmd always ask for password when starting the service.

Shell output:

================================================================================================================================
 Package                Arch       Version                       Repository                                                Size
================================================================================================================================
Installing:
 vdsm                   x86_64     4.12.0-144.git1b9acee.el6     /vdsm-4.12.0-144.git1b9acee.el6.x86_64                   4.0 M
 vdsm-cli               noarch     4.12.0-144.git1b9acee.el6     /vdsm-cli-4.12.0-144.git1b9acee.el6.noarch               362 k
 vdsm-python            x86_64     4.12.0-144.git1b9acee.el6     /vdsm-python-4.12.0-144.git1b9acee.el6.x86_64            439 k
 vdsm-python-cpopen     x86_64     4.12.0-144.git1b9acee.el6     /vdsm-python-cpopen-4.12.0-144.git1b9acee.el6.x86_64      34 k
 vdsm-xmlrpc            noarch     4.12.0-144.git1b9acee.el6     /vdsm-xmlrpc-4.12.0-144.git1b9acee.el6.noarch            123 k

Transaction Summary
================================================================================================================================
Install       5 Package(s)

Total size: 5.0 M
Installed size: 5.0 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : vdsm-python-4.12.0-144.git1b9acee.el6.x86_64                                                                 1/5 
  Installing : vdsm-python-cpopen-4.12.0-144.git1b9acee.el6.x86_64                                                          2/5 
  Installing : vdsm-xmlrpc-4.12.0-144.git1b9acee.el6.noarch                                                                 3/5 
  Installing : vdsm-4.12.0-144.git1b9acee.el6.x86_64                                                                        4/5 
  Installing : vdsm-cli-4.12.0-144.git1b9acee.el6.noarch                                                                    5/5 
  Verifying  : vdsm-python-cpopen-4.12.0-144.git1b9acee.el6.x86_64                                                          1/5 
  Verifying  : vdsm-python-4.12.0-144.git1b9acee.el6.x86_64                                                                 2/5 
  Verifying  : vdsm-xmlrpc-4.12.0-144.git1b9acee.el6.noarch                                                                 3/5 
  Verifying  : vdsm-4.12.0-144.git1b9acee.el6.x86_64                                                                        4/5 
  Verifying  : vdsm-cli-4.12.0-144.git1b9acee.el6.noarch                                                                    5/5 

Installed:
  vdsm.x86_64 0:4.12.0-144.git1b9acee.el6                     vdsm-cli.noarch 0:4.12.0-144.git1b9acee.el6                       
  vdsm-python.x86_64 0:4.12.0-144.git1b9acee.el6              vdsm-python-cpopen.x86_64 0:4.12.0-144.git1b9acee.el6             
  vdsm-xmlrpc.noarch 0:4.12.0-144.git1b9acee.el6             

Complete!
[root@dhcp-2-233 RPMS]# service vdsmd stop
Shutting down vdsm daemon: 
vdsm watchdog stop                                         [  OK  ]
vdsm: not running                                          [FAILED]
vdsm: Running run_final_hooks
vdsm stop                                                  [  OK  ]
[root@dhcp-2-233 RPMS]# service vdsmd start

Running guests on default URI: fedora19, precise
Not suspending transient guests on URI: default: fedora19, precise
supervdsm start                                            [  OK  ]
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running reconfigure_sanlock
vdsm: Running reconfigure_libvirt
Stopping libvirtd daemon: [  OK  ]
Reconfiguration of libvirt is done.

To start working with the new configuration, execute:
'vdsm-tool libvirt-configure-services-restart'
This will manage restarting of the following services:
libvirtd, supervdsmd

diff: /etc/init/libvirtd.conf: No such file or directory
vdsm: Running syslog_available
vdsm: Running nwfilter
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 143, in <module>
    sys.exit(main())
  File "/usr/bin/vdsm-tool", line 140, in main
    return tool_command[cmd]["command"](*args[1:])
  File "/usr/lib64/python2.6/site-packages/vdsm/tool/nwfilter.py", line 35, in main
    conn = libvirtconnection.get(None, False)
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 125, in get
    conn = utils.retry(libvirtOpenAuth, timeout=10, sleep=0.2)
  File "/usr/lib64/python2.6/site-packages/vdsm/utils.py", line 903, in retry
    return func()
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 102, in openAuth
    if ret is None:raise libvirtError('virConnectOpenAuth() failed')
libvirt.libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
vdsm: failed to execute nwfilter, error code 1
Starting up vdsm daemon: 
vdsm start                                                 [  OK  ]
[root@dhcp-2-233 RPMS]# service vdsmd stop
Shutting down vdsm daemon: 
vdsm watchdog stop                                         [  OK  ]
vdsm: Running run_final_hooks                              [  OK  ]
vdsm stop                                                  [  OK  ]
[root@dhcp-2-233 RPMS]# service vdsmd start

Can't connect to default. Skipping.
libvirtd start/running, process 12872
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running reconfigure_sanlock
vdsm: Running reconfigure_libvirt
libvirt is already configured for vdsm
vdsm: Running syslog_available
vdsm: Running nwfilter
libvir: Network Filter Driver error : Requested operation is not valid: nwfilter is in use
vdsm: Running dummybr
vdsm: Running load_needed_modules
vdsm: Running tune_system
vdsm: Running mkdirs
vdsm: Running test_space
vdsm: Running test_lo
vdsm: Running test_conflicting_conf
SUCCESS: ssl configured to true. No conflicts
Starting up vdsm daemon: 
vdsm start                                                 [  OK  ]
[root@dhcp-2-233 RPMS]# service vdsmd stop
Shutting down vdsm daemon: 
vdsm watchdog stop                                         [  OK  ]
vdsm: Running run_final_hooks                              [  OK  ]
vdsm stop                                                  [  OK  ]
[root@dhcp-2-233 RPMS]# service vdsmd start
Please enter your authentication name: Please enter your password:

Comment 1 Nir Soffer 2013-09-22 15:19:40 UTC

Entering empty password, vdsmd service starts:

...
Please enter your authentication name: Please enter your password:

Can't connect to default. Skipping.
initctl: Job is already running: libvirtd
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running reconfigure_sanlock
vdsm: Running reconfigure_libvirt
libvirt is already configured for vdsm
vdsm: Running syslog_available
vdsm: Running nwfilter
libvir: Network Filter Driver error : Requested operation is not valid: nwfilter is in use
vdsm: Running dummybr
vdsm: Running load_needed_modules
vdsm: Running tune_system
vdsm: Running mkdirs
vdsm: Running test_space
vdsm: Running test_lo
vdsm: Running test_conflicting_conf
SUCCESS: ssl configured to true. No conflicts
Starting up vdsm daemon: 
vdsm start                                                 [  OK  ]


From the engine side, the host seems to function correctly. The vms can be stopped and started again.

Comment 2 Dan Kenigsberg 2013-09-25 13:35:01 UTC

The problem is real, due to

# service libvirt-guests status; echo $?
stopped, with no saved guests
0

returning a wrong $? == 0 even when it is stopped (should be 3)

The issue is solved in upstream libvirt by virtue of

commit 9c51de2b4e2158b93adab4467293b35626099ff6
Author: Ján Tomko <jtomko>
Date:   Tue Apr 30 13:44:33 2013 +0200

    libvirt-guests: status: return non-zero when stopped


Note: the issue is hidden if somehow there were saved guests such as in
# service libvirt-guests status; echo $?
stopped, with saved guests
3

Comment 3 Yaniv Bronhaim 2013-09-29 08:35:49 UTC

*** Bug 1010667 has been marked as a duplicate of this bug. ***

Comment 4 Yaniv Bronhaim 2013-09-29 08:38:23 UTC

Fix in libvirt-guests init script (BZ 1011981), putting bug on POST until merge to rhel6.4

Comment 5 Yaniv Bronhaim 2013-09-29 12:27:38 UTC

Attached workaround to avoid password request when starting vdsmd service.    

The workaround was part of vdsmd init script already in the past and was removed by mistake, this adds it back and will be removed once libvirt-guests init script returns the right exit code.

Comment 6 Nir Soffer 2013-09-29 16:47:23 UTC

The workardound do work on RHEL 6.4 and vdsm 5ffd93d758c3c66.

Comment 7 Dan Kenigsberg 2013-09-29 20:47:02 UTC

Closing per verification in comment 6.

Comment 8 Yaniv Bronhaim 2013-09-30 08:02:15 UTC


*** This bug has been marked as a duplicate of bug 1010667 ***