Description of problem: after upgrading vdsm to vdsm-4.16.22-1.el6ev.x86_64, and running 'vdsm-tool is-configured' returns: """ libvirt is not configured for vdsm yet Modules libvirt are not configured Error: One of the modules is not configured to work with VDSM. To configure the module use the following: 'vdsm-tool configure [--module module-name]'. """ The result of that is that unless running 'vdsm-tool configure --force' VDSM will not run after upgrade. Version-Release number of selected component (if applicable): vdsm-4.16.22-1.el6ev.x86_64 # uname -r 2.6.32-573.el6.x86_64 How reproducible: always Steps to Reproduce: 1. upgrade vdsm from 3.4 2. run 'vdsm-tool is-configured' 3. Actual results: libvirt appears as not configured Expected results: Since libvirt was configured before VDSM was upgraded, it shuold be also configured after the VDSM is upgraded, or VDSM will not start. Additional info:
just to clarify: # cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.7 (Santiago)
Also, after running 'vdsm-tool configure --force' and rebooting, VDSM refused to start with the same error. another 'vdsm-tool configure --force' solves it. ----------------- [root@pink-vds2 ~]# service vdsmd status VDS daemon is not running [root@pink-vds2 ~]# service supervdsmd status Super VDSM daemon server is running [root@pink-vds2 ~]# service vdsmd start libvirtd start/running, process 4545 vdsm: Running mkdirs vdsm: Running configure_coredump vdsm: Running configure_vdsm_logs vdsm: Running wait_for_network vdsm: Running run_init_hooks vdsm: Running upgraded_version_check vdsm: Running check_is_configured libvirt is not configured for vdsm yet Modules libvirt are not configured Error: One of the modules is not configured to work with VDSM. To configure the module use the following: 'vdsm-tool configure [--module module-name]'. If all modules are not configured try to use: 'vdsm-tool configure --force' (The force flag will stop the module's service and start it afterwards automatically to load the new configuration.) vdsm: stopped during execute check_is_configured task (task returned with error code 1). vdsm start [FAILED] [root@pink-vds2 ~]# vdsm-tool configure --force Checking configuration status... libvirt is not configured for vdsm yet Running configure... Reconfiguration of libvirt is done. Done configuring modules to VDSM. [root@pink-vds2 ~]# service vdsmd start libvirtd start/running, process 5190 vdsm: Running mkdirs vdsm: Running configure_coredump vdsm: Running configure_vdsm_logs vdsm: Running wait_for_network vdsm: Running run_init_hooks vdsm: Running upgraded_version_check vdsm: Running check_is_configured libvirt is already configured for vdsm vdsm: Running validate_configuration SUCCESS: ssl configured to true. No conflicts vdsm: Running prepare_transient_repository vdsm: Running syslog_available vdsm: Running nwfilter vdsm: Running dummybr vdsm: Running load_needed_modules vdsm: Running tune_system vdsm: Running test_space vdsm: Running test_lo Upgrading to unified persistence if needed Upgrading to v3.x networking if needed Starting up vdsm daemon: vdsm start [ OK ]
Its not stated from what version you upgraded - Anyhow, I just checked the following which is between latest 4.16.22 from 4.14.17 (latest 3.4 I found in resources.ovirt.org) - and it works fine. Maybe you installed the 3.4 one without running the service at all and you assume that after upgrade it will work without configurations... :/ please try to reproduce and state all the steps you did. I verified the following steps: 1. installing 4.14.17 vdsm rpms 2. running vdsm-tool configure 3. service vdsmd start (seeing that it works and response) 4. running yum upgrade vdsm after setting repo with 4.16.22 5. vdsm still up with the new version (means it was restarted alright) 6. ran vdsm-tool is-configured and saw "libvirt is already configured for vdsm" seems OK to me. -- snip Updating : vdsm-python-zombiereaper-4.16.22-0.el6.noarch 1/12 Updating : vdsm-python-4.16.22-0.el6.noarch 2/12 Updating : vdsm-xmlrpc-4.16.22-0.el6.noarch 3/12 Installing : vdsm-yajsonrpc-4.16.22-0.el6.noarch 4/12 Installing : vdsm-jsonrpc-4.16.22-0.el6.noarch 5/12 Updating : vdsm-4.16.22-0.el6.x86_64 6/12 Updating : vdsm-cli-4.16.22-0.el6.noarch 7/12 Cleanup : vdsm-4.14.17-0.el6.x86_64 8/12 Cleanup : vdsm-cli-4.14.17-0.el6.noarch 9/12 Cleanup : vdsm-xmlrpc-4.14.17-0.el6.noarch 10/12 Cleanup : vdsm-python-4.14.17-0.el6.x86_64 11/12 Cleanup : vdsm-python-zombiereaper-4.14.17-0.el6.noarch 12/12 Verifying : vdsm-xmlrpc-4.16.22-0.el6.noarch 1/12 Verifying : vdsm-python-4.16.22-0.el6.noarch 2/12 Verifying : vdsm-yajsonrpc-4.16.22-0.el6.noarch 3/12 Verifying : vdsm-python-zombiereaper-4.16.22-0.el6.noarch 4/12 Verifying : vdsm-jsonrpc-4.16.22-0.el6.noarch 5/12 Verifying : vdsm-4.16.22-0.el6.x86_64 6/12 Verifying : vdsm-cli-4.16.22-0.el6.noarch 7/12 Verifying : vdsm-xmlrpc-4.14.17-0.el6.noarch 8/12 Verifying : vdsm-python-4.14.17-0.el6.x86_64 9/12 Verifying : vdsm-cli-4.14.17-0.el6.noarch 10/12 Verifying : vdsm-python-zombiereaper-4.14.17-0.el6.noarch 11/12 Verifying : vdsm-4.14.17-0.el6.x86_64 Dependency Installed: vdsm-jsonrpc.noarch 0:4.16.22-0.el6 vdsm-yajsonrpc.noarch 0:4.16.22-0.el6 Updated: vdsm.x86_64 0:4.16.22-0.el6 Dependency Updated: vdsm-cli.noarch 0:4.16.22-0.el6 vdsm-python.noarch 0:4.16.22-0.el6 vdsm-python-zombiereaper.noarch 0:4.16.22-0.el6 vdsm-xmlrpc.noarch 0:4.16.22-0.el6 Complete! [root@dhcp-0-211 vdsm]# vdsm-tool is-configured libvirt is already configured for vdsm [root@dhcp-0-211 vdsm]# service vdsmd status VDS daemon server is running
This issue happen after running upgrade form vdsm- 4.14.18-7.el6ev.noarch to vdsm.x86_64 0:4.16.22-1.el6ev after first reboot we discovered that vdsmd is not running because libvirt is not configured, i was looking for 'vdsm-restore-net' in supervdsm.log but there was no network restoration, because vdsmd wasn't running. As Ido wrote, it's enough to run vdsm-tool or /etc/init.d/vdsmd status to see that vdsmd is not running after upgarde from 4.14.18 >> 4.16.22
To be more correct about this, vdsmd is running after upgrade, but libvirt is not configured yet for vdsm. I will attach logs.
I tried to reproduce #4 but I already failed on vdsm-tool configure with 3.4.5 vdsm version. I'm not sure running vdsm-tool configure right after vdsm is installed (without rhevm) is valid flow... - 3.4.5/vdsm-4.14.18-6.el6ev > /etc/init.d/vdsm status = not running > /etc/init.d/vdsm start = finished with FAILED [root@dell-r210ii-13 yum.repos.d]# vdsm-tool configure Checking configuration status... libvirt is not configured for vdsm yet SUCCESS: ssl configured to true. No conflicts Running configure... Reconfiguration of libvirt is done. Done configuring modules to VDSM. [root@dell-r210ii-13 yum.repos.d]# echo $? 0 [root@dell-r210ii-13 yum.repos.d]# /etc/init.d/vdsmd status VDS daemon is not running [root@dell-r210ii-13 yum.repos.d]# /etc/init.d/vdsmd start Starting multipathd daemon: [ OK ] Starting rpcbind: [ OK ] Starting wdmd: [ OK ] Starting sanlock: [ OK ] libvirtd start/running, process 7454 supervdsm start [ OK ] Starting iscsid: [ OK ] vdsm: Running mkdirs vdsm: Running configure_coredump vdsm: Running configure_vdsm_logs vdsm: Running run_init_hooks vdsm: Running gencerts Configuring a self-signed VDSM host certificatevdsm: Running upgraded_version_check vdsm: Running check_is_configured libvirt is already configured for vdsm vdsm: Running validate_configuration FAILED: conflicting vdsm and libvirt-qemu tls configuration. vdsm.conf with ssl=True requires the following changed: libvirtd.conf: listen_tcp=0, auth_tcp="sasl", qemu.conf: spice_tls=1. Modules libvirt contains invalid configuration Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 145, in <module> sys.exit(main()) File "/usr/bin/vdsm-tool", line 142, in main return tool_command[cmd]["command"](*args[1:]) File "/usr/lib64/python2.6/site-packages/vdsm/tool/configurator.py", line 372, in validate_config raise RuntimeError("Config is not valid. Check conf files") RuntimeError: Config is not valid. Check conf files vdsm: stopped during execute validate_configuration task (task returned with error code 1). vdsm start [FAILED] > upgrade > 3.5.4.1-1/vdsm-4.16.22-1.el6ev ok > vdsm-tool configure ; rc = 0 > vdsm-tool configure = rc 1 [root@dell-r210ii-13 yum.repos.d]# vdsm-tool configure Checking configuration status... libvirt is already configured for vdsm FAILED: conflicting vdsm and libvirt-qemu tls configuration. vdsm.conf with ssl=True requires the following changes: libvirtd.conf: listen_tcp=0, auth_tcp="sasl", qemu.conf: spice_tls=1. Error: Configuration of libvirt is invalid
Clarification after upgrade: > vdsm-tool configure ; rc = 0 ^^^ cut & paste error, ignore above line :) > vdsm-tool configure = rc 1
Tested with clean Red Hat Enterprise Linux Server release 6.7 (Santiago) server and vdsm-4.14.18-7.el6ev.x86_64 installed(av14.3). 1) Installed server in rhevm-3.5.4-1.1.el6ev.noarch engine with success 2) Configured bond0 and network 't1' attached him and another network 't2' attached to eth1. Approved operation. 3) [root@pink-vds2 ~]# /etc/init.d/vdsmd status VDS daemon server is running [root@pink-vds2 ~]# /etc/init.d/libvirtd status libvirtd (pid 11126) is running... 4) Verified my network configuration before upgrade and run update for vdsm root@pink-vds2 ~]# brctl show bridge name bridge id STP enabled interfaces ;vdsmdummy; 8000.000000000000 no rhevm 8000.00215e3fdb2c no eth0 t1 8000.001b21d0bb4a no bond0 t2 8000.00215e3fdb2e no eth1 virbr0 8000.525400c281cd yes virbr0-nic - ifcfg-* files were generated by VDSM version 4.14.18-7.el6ev 5) Updated vdsm with success to >> vdsm-4.16.22-1.el6ev.x86_64 [root@pink-vds2 yum.repos.d]# rpm -q vdsm vdsm-4.16.22-1.el6ev.x86_64 [root@pink-vds2 yum.repos.d]# /etc/init.d/vdsmd status VDS daemon server is running [root@pink-vds2 yum.repos.d]# /etc/init.d/libvirtd status libvirtd (pid 11126) is running... [root@pink-vds2 yum.repos.d]# vdsm-tool is-configured libvirt is already configured for vdsm (Ido i didn't managed to reproduce it without reboot^^) 6) Reboot server --> [root@pink-vds2 ~]# /etc/init.d/libvirtd status libvirtd (pid 3511) is running... [root@pink-vds2 ~]# /etc/init.d/vdsmd status VDS daemon server is running [root@pink-vds2 ~]# vdsm-tool is-configured libvirt is not configured for vdsm yet Modules libvirt are not configured Error: One of the modules is not configured to work with VDSM. To configure the module use the following: 'vdsm-tool configure [--module module-name]'. If all modules are not configured try to use: 'vdsm-tool configure --force' (The force flag will stop the module's service and start it afterwards automatically to load the new configuration.) So reproduced after first reboot after upgrade.
Created attachment 1052738 [details] vdsm logs
The machine was set with puppet agent which reran puppet script that enables libvirtd which vdsm recognize as wrong system configuration. over el6 vdsm configure libvirt to start with upstart. Jul 16 17:39:12 pink-vds2 puppet-agent[3362]: (/Stage[main]/Ssh/Sshkey[rose10.qa.lab.tlv.redhat.com]) Could not evaluate: Field 'key' is required Jul 16 17:39:13 pink-vds2 puppet-agent[3362]: (/Stage[main]/Libvirt::Service/Service[libvirtd]/enable) enable changed 'false' to 'true' Disabling the puppet run fix the issue. Closing as not a bug.
*** Bug 1252843 has been marked as a duplicate of this bug. ***