Bug 1067511

Setting target release to current version for consideration and review. please
do not push non-RFE bugs to an undefined target release to make sure bugs are
reviewed for relevancy, fix, closure, etc.

From the logs:

Command returned failure code 1 during SSH session 'root@...'
Stdout: 
Stderr: Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 145, in <module>
    sys.exit(main())
  File "/usr/bin/vdsm-tool", line 142, in main
    return tool_command[cmd]["command"](*args[1:])
  File "/usr/lib64/python2.6/site-packages/vdsm/tool/service.py", line 390, in service_restart
    return _runAlts(_srvRestartAlts, srvName)
  File "/usr/lib64/python2.6/site-packages/vdsm/tool/service.py", line 350, in _runAlts
    "%s failed" % alt.func_name, out, err)
vdsm.tool.service.ServiceOperationError: ServiceOperationError: _serviceRestart failed
Shutting down vdsm daemon: 
[FAILED]
vdsm watchdog stop[  OK  ]
vdsm: not running[FAILED]
vdsm: Running run_final_hooks
vdsm stop[  OK  ]
supervdsm start[  OK  ]
Starting iscsid: [  OK  ]
[  OK  ]
vdsm: Running mkdirs
vdsm: Running configure_coredump
vdsm: Running run_init_hooks
vdsm: Running gencerts
vdsm: Running check_is_configured
libvirt is not configured for vdsm yet
sanlock service is already configured
Modules libvirt are not configured
 vdsm: stopped during execute check_is_configured task (task returned with error code 1).
vdsm start[FAILED]

Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 145, in <module>
    sys.exit(main())
  File "/usr/bin/vdsm-tool", line 142, in main
    return tool_command[cmd]["command"](*args[1:])
  File "/usr/lib64/python2.6/site-packages/vdsm/tool/configurator.py", line 265, in isconfigured
    raise RuntimeError(msg)
RuntimeError: 

One of the modules is not configured to work with VDSM.
To configure the module use the following:
'vdsm-tool configure [module_name]'.

If all modules are not configured try to use:
'vdsm-tool configure --force'
(The force flag will stop the module's service and start it
afterwards automatically to load the new configuration.)

i tried to run suggested command and end up with the same result.
suggestions ?
(In reply to Fabian Deutsch from comment #2)
> From the logs:
> 
> Command returned failure code 1 during SSH session 'root@...'
> Stdout: 
> Stderr: Traceback (most recent call last):
>   File "/usr/bin/vdsm-tool", line 145, in <module>
>     sys.exit(main())
>   File "/usr/bin/vdsm-tool", line 142, in main
>     return tool_command[cmd]["command"](*args[1:])
>   File "/usr/lib64/python2.6/site-packages/vdsm/tool/service.py", line 390,
> in service_restart
>     return _runAlts(_srvRestartAlts, srvName)
>   File "/usr/lib64/python2.6/site-packages/vdsm/tool/service.py", line 350,
> in _runAlts
>     "%s failed" % alt.func_name, out, err)
> vdsm.tool.service.ServiceOperationError: ServiceOperationError:
> _serviceRestart failed
> Shutting down vdsm daemon: 
> [FAILED]
> vdsm watchdog stop[  OK  ]
> vdsm: not running[FAILED]
> vdsm: Running run_final_hooks
> vdsm stop[  OK  ]
> supervdsm start[  OK  ]
> Starting iscsid: [  OK  ]
> [  OK  ]
> vdsm: Running mkdirs
> vdsm: Running configure_coredump
> vdsm: Running run_init_hooks
> vdsm: Running gencerts
> vdsm: Running check_is_configured
> libvirt is not configured for vdsm yet
> sanlock service is already configured
> Modules libvirt are not configured
>  vdsm: stopped during execute check_is_configured task (task returned with
> error code 1).
> vdsm start[FAILED]
> 
> Traceback (most recent call last):
>   File "/usr/bin/vdsm-tool", line 145, in <module>
>     sys.exit(main())
>   File "/usr/bin/vdsm-tool", line 142, in main
>     return tool_command[cmd]["command"](*args[1:])
>   File "/usr/lib64/python2.6/site-packages/vdsm/tool/configurator.py", line
> 265, in isconfigured
>     raise RuntimeError(msg)
> RuntimeError: 
> 
> One of the modules is not configured to work with VDSM.
> To configure the module use the following:
> 'vdsm-tool configure [module_name]'.
> 
> If all modules are not configured try to use:
> 'vdsm-tool configure --force'
> (The force flag will stop the module's service and start it
> afterwards automatically to load the new configuration.)

This is an automated message.
Re-targeting all non-blocker bugs still open on 3.4.0 to 3.4.1.

I suspect that the main issue is the repeated log:

2014-02-18 10:07:49,902 ERROR [org.ovirt.engine.core.bll.GetoVirtISOsQuery] (ajp--127.0.0.1-8702-11) ovirt ISOs directory not found. Search in: /usr/share/ovirt-node-iso

Fabian, should it work without the isos directory? or this just for upgrades?

Hi Shai,

(In reply to Fabian Deutsch from comment #2)
> From the logs:
> 
> Command returned failure code 1 during SSH session 'root@...'
> Stdout: 
> Stderr: Traceback (most recent call last):
>   File "/usr/bin/vdsm-tool", line 145, in <module>
>     sys.exit(main())
>   File "/usr/bin/vdsm-tool", line 142, in main
>     return tool_command[cmd]["command"](*args[1:])
>   File "/usr/lib64/python2.6/site-packages/vdsm/tool/service.py", line 390,
> in service_restart
>     return _runAlts(_srvRestartAlts, srvName)
>   File "/usr/lib64/python2.6/site-packages/vdsm/tool/service.py", line 350,
> in _runAlts
>     "%s failed" % alt.func_name, out, err)
> vdsm.tool.service.ServiceOperationError: ServiceOperationError:
> _serviceRestart failed
> Shutting down vdsm daemon: 
> [FAILED]
> vdsm watchdog stop[  OK  ]
> vdsm: not running[FAILED]
> vdsm: Running run_final_hooks
> vdsm stop[  OK  ]
> supervdsm start[  OK  ]
> Starting iscsid: [  OK  ]
> [  OK  ]
> vdsm: Running mkdirs
> vdsm: Running configure_coredump
> vdsm: Running run_init_hooks
> vdsm: Running gencerts
> vdsm: Running check_is_configured
> libvirt is not configured for vdsm yet
> sanlock service is already configured
> Modules libvirt are not configured
>  vdsm: stopped during execute check_is_configured task (task returned with
> error code 1).
> vdsm start[FAILED]
> 
> Traceback (most recent call last):
>   File "/usr/bin/vdsm-tool", line 145, in <module>
>     sys.exit(main())
>   File "/usr/bin/vdsm-tool", line 142, in main
>     return tool_command[cmd]["command"](*args[1:])
>   File "/usr/lib64/python2.6/site-packages/vdsm/tool/configurator.py", line
> 265, in isconfigured
>     raise RuntimeError(msg)
> RuntimeError: 
> 
> One of the modules is not configured to work with VDSM.
> To configure the module use the following:
> 'vdsm-tool configure [module_name]'.
> 
> If all modules are not configured try to use:
> 'vdsm-tool configure --force'
> (The force flag will stop the module's service and start it
> afterwards automatically to load the new configuration.)

This report seems old 02/20 and I do believe the iso you tested doesn't contain the vdsm which fix for this issue.

From engine.log:
<snip>
Installing Host delldesktop. Yum Download/Verify: vdsm-4.14.3-0.el6.x86_64
</snip>

You report node iso: 3.0.4-1.0.201401291204.el6

Now the same iso version in ovirt.resources:
http://resources.ovirt.org/releases/3.4/iso/ovirt-node-iso-3.0.4-1.0.201401291204.vdsm34.el6.iso

# mount -o loop ./ovirt-node-iso-3.0.4-1.0.201401291204.vdsm34.el6.iso /tmp
vi /tmp/isolinux/current-manifest-rpm.txt

vdsm-4.14.6-0.el6.x86_64 ((none))
vdsm-cli-4.14.6-0.el6.noarch ((none))
vdsm-gluster-4.14.6-0.el6.noarch ((none))
vdsm-python-4.14.6-0.el6.x86_64 ((none))
vdsm-python-zombiereaper-4.14.6-0.el6.noarch ((none))
vdsm-reg-4.14.6-0.el6.noarch ((none))
vdsm-xmlrpc-4.14.6-0.el6.noarch ((none))


We had several improvements in node for the final GA release, some of patches listed below. Please re-test with this last iso available so we can close this bug. Thanks!

commit 9af7babc1007ad050f2c1f63409b50747edfe32b
Author: Douglas Schilling Landgraf <dougsland>
Date:   Fri Mar 21 03:24:08 2014 -0400

    libvirt_configure: persist cmd qemu-sanlock.conf

    On a ovirt-node if executing vdsm-tool configure --force
    with libvirt sanlock enabled it will set /etc/libvirt/qemu-sanlock.conf
    but won't use persist command so next reboot the changes will be lost
    and vdsm-tool will complain about libvirt not configured during vdsm start.

    Change-Id: I299f5cb7af4d2e09208835aeda28ec1f3c3def7d
    Signed-off-by: Douglas Schilling Landgraf <dougsland>
    Reviewed-on: http://gerrit.ovirt.org/25968
    Reviewed-by: Fabian Deutsch <fabiand>
    Reviewed-by: Dan Kenigsberg <danken>

commit b4efa0036e065031d6b66d3e86f03ea6e5141e0c
Author: Douglas Schilling Landgraf <dougsland>
Date:   Wed Mar 12 16:11:55 2014 -0400

    libvirt_configure: unpersist logrotate.d/libvirt

    On commit 247a76e7 we moved the libvirt settings to libvirt_configure.sh.in and now
    we see that vdsm cannot rename temp. file to /etc/logrotate.d/libvirtd on
    ovirt-node system at the stage we execute vdsm-tool configure --force.
    This patch will add the command unpersist before renaming the file,
    otherwise ovirt-node will complain with "Device or resource busy".

    Change-Id: Icd0e6b3ea39680ac3fc8ec5731aa6be3bd8e4deb
    Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1074257
    Signed-off-by: Douglas Schilling Landgraf <dougsland>
    Reviewed-on: http://gerrit.ovirt.org/25700
    Reviewed-by: Yaniv Bronhaim <ybronhei>
    Reviewed-by: Dan Kenigsberg <danken>

commit b2cbc32cf62ff20d00dd75d1f801058a2a903ca3
Author: Douglas Schilling Landgraf <dougsland>
Date:   Tue Mar 25 14:18:02 2014 -0400

    configurator: move usermod to spec

    vdsm-tool configure --force adds to /etc/group (qemu/kvm) sanlock
    but doesn't persist the file in ovirt node distro which will
    affect vdsm start on next reboot. This patch adds the usermod to
    spec file.

    Change-Id: I8a44a8a4d3301263beeb03e1d106c25ffe85263e
    Signed-off-by: Douglas Schilling Landgraf <dougsland>
    Reviewed-on: http://gerrit.ovirt.org/26089
    Reviewed-by: Dan Kenigsberg <danken>

Hi Shai,

  For now, I am closing this one as current release based on my above comment, fell free to re-open in case you see it again.