Created attachment 931891 [details] log infos and screen shot of error Description of problem: It reports error as follows when first configure hosted-engine in host. =============================================================================== [ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services. Version-Release number of selected component (if applicable): rhev-hypervisor7-7.0-20140827.0.iso ovirt-node-3.1.0-0.9.20140815gitee21d9b.el7.noarch ovirt-node-plugin-hosted-engine-0.1.0-0.0.x86_64 ovirt-host-deploy-1.3.0-0.0.1.master.el7.noarch ovirt-hosted-engine-setup-1.2.0-0.2.master.el7.noarch ovirt-hosted-engine-ha-1.2.1-0.3.master.el7.noarch How reproducible: 100% Steps to Reproduce: 1. Install rhev-hypervisor7-7.0-20140827.0.iso on the host 2. Enabled network 3. ps -aux |grep ovirt-ha 4. Configure hosted-engine in shell #ovirt-hosted-engine-setup Actual results: 1. After step3, the output is as follows. #ps -aux |grep ovirt-ha vdsm 1356 0.0 0.1 136204 12200 ? S 03:56 0:00 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker 2. After step4, it will report error as follows. [ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services. Expected results: It should run setup hosted-engine succeed. Additional info:
The broker is running, so the code works as expected, do you have any idea what started it? Is this a clean system or it was used before to deploy the HE?
Checking comment 3, it may ovirt-node-plugin-hosted-engine component bug.
> In reply to Ying Cui from comment #4) > Checking comment 3, it may ovirt-node-plugin-hosted-engine component bug. Good catch! ovirt-node-plugin-hosted-engine enables it so it's started on boot, this should be handled by the hosted engine setup, so it need to be removed from the node spec. over to ovirt-node-plugin-hosted-engine
(In reply to Jiri Moskovcak from comment #5) > > In reply to Ying Cui from comment #4) > > Checking comment 3, it may ovirt-node-plugin-hosted-engine component bug. > > Good catch! ovirt-node-plugin-hosted-engine enables it so it's started on > boot, this should be handled by the hosted engine setup, so it need to be > removed from the node spec. over to ovirt-node-plugin-hosted-engine +1 good catch!
It is not possible to enable a service on Node at runtime. That means that we normally enable all required services at build time - this is the reason why the HA service is already running. IIUIC the engine setup tries to enable the HA service during the setup, is that correct? If so, then this will fail, because enabling the service will not be persisted. Is there a way to workaroundt his from the hosted-engine side? Otherwise we could add a service which is enabled by default, and starts the HA service conditionally.
Hi wanghui, Can you please attach your /var/log/messages? From my test with rhev-hypervisor7-7.0-20140827.0.iso # ovirt-hosted-engine-setup [ INFO ] Stage: Initializing Continuing will configure this host for serving as hypervisor and create a VM where you have to install oVirt Engine afterwards. Are you sure you want to continue? (Yes, No)[Yes]: Yes It has been detected that this program is executed through an SSH connection without using screen. Continuing with the installation may lead to broken installation if the network connection fails. It is highly recommended to abort the installation and run it inside a screen session using command "screen". Do you want to continue anyway? (Yes, No)[No]: Yes [ INFO ] Generating a temporary VNC password. [ INFO ] Stage: Environment setup Configuration files: [] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20140902181447-x9whv9.log Version: otopi-1.3.0_master (otopi-1.3.0-0.0.master.el7) [ INFO ] Hardware supports virtualization [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [ ERROR ] Failed to execute stage 'Environment setup': Command '/bin/systemctl' failed to execute [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf' [ INFO ] Answer file '/etc/ovirt-hosted-engine/answers.conf' has been updated [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination Sep 2 17:50:13 localhost systemd: Starting Virtual Desktop Server Manager... Sep 2 17:50:13 localhost systemd: vdsmd.service: control process exited, code=exited status=1 Sep 2 17:50:13 localhost systemd: Failed to start Virtual Desktop Server Manager. Sep 2 17:50:13 localhost systemd: Unit vdsmd.service entered failed state. Sep 2 17:50:13 localhost vdsmd_init_common.sh: vdsm: stopped during execute check_is_configured task (task returned with error code 1). Sep 2 17:50:13 localhost vdsm-tool: module configfile could not load to vdsm-tool: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 81, in load_modules mod_absp, mod_desc) File "/usr/lib64/python2.7/site-packages/vdsm/tool/configfile.py", line 30, in <module> File "/usr/lib/python2.7/site-packages/ovirt/node/utils/__init__.py", line 22, in <module> File "/usr/lib/python2.7/site-packages/augeas.py", line 56, in <module> File "/usr/lib/python2.7/site-packages/augeas.py", line 62, in Augeas File "/usr/lib/python2.7/site-packages/augeas.py", line 53, in _dlopen ImportError: Unable to import libpython2.7! Sep 2 17:50:13 localhost vdsm-tool: module configurator could not load to vdsm-tool: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 81, in load_modules mod_absp, mod_desc) File "/usr/lib64/python2.7/site-packages/vdsm/tool/configurator.py", line 31, in <module> File "/usr/lib64/python2.7/site-packages/vdsm/tool/configfile.py", line 30, in <module> File "/usr/lib/python2.7/site-packages/ovirt/node/utils/__init__.py", line 22, in <module> File "/usr/lib/python2.7/site-packages/augeas.py", line 56, in <module> File "/usr/lib/python2.7/site-packages/augeas.py", line 62, in Augeas File "/usr/lib/python2.7/site-packages/augeas.py", line 53, in _dlopen ImportError: Unable to import libpython2.7! Sep 2 17:50:13 localhost vdsm-tool: module upgrade_300_networks could not load to vdsm-tool: Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 81, in load_modules mod_absp, mod_desc) File "/usr/lib64/python2.7/site-packages/vdsm/tool/upgrade_300_networks.py", line 30, in <module> File "/usr/share/vdsm/network/configurators/ifcfg.py", line 41, in <module> File "/usr/lib/python2.7/site-packages/ovirt/node/utils/__init__.py", line 22, in <module> File "/usr/lib/python2.7/site-packages/augeas.py", line 56, in <module> File "/usr/lib/python2.7/site-packages/augeas.py", line 62, in Augeas File "/usr/lib/python2.7/site-packages/augeas.py", line 53, in _dlopen ImportError: Unable to import libpython2.7! Sep 2 17:50:13 localhost vdsmd_init_common.sh: vdsm: Running mkdirs
Ok, I have reproduced the report generating an ISO manually. [ INFO ] Generating a temporary VNC password. [ INFO ] Stage: Environment setup Configuration files: [] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20140902212223-otzfhd.log Version: otopi-1.3.0_master (otopi-1.3.0-0.0.master.el7) [ INFO ] Hardware supports virtualization [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ ERROR ] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services. [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/etc/ovirt-hosted-engine/answers.conf' [ INFO ] Answer file '/etc/ovirt-hosted-engine/answers.conf' has been updated [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination # ps auxww | grep ovirt-ha vdsm 1489 0.0 0.4 132584 12396 ? S 21:14 0:00 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broke
Hi Sandro, Based on comment#7, can we add additional validation to such verification [1], for example: If ovirt-ha is running and there is no /etc/ovirt-hosted-engine/vm.conf can we continue the setup? or even similar validation in other .conf file? I do believe, this kind of issue (enabling/starting the service) before the setup could be happened in a non ovirt-node environment as well. Please let me know your thoughts, thanks! [1] https://github.com/oVirt/ovirt-hosted-engine-setup/blob/master/src/plugins/ovirt-hosted-engine-setup/ha/ha_services.py#L62
I guess we can add a check in setup and automatically stop the ha services if we find them running but not configured instead of telling the user to stop them manually.
(In reply to Sandro Bonazzola from comment #11) > I guess we can add a check in setup and automatically stop the ha services > if we find them running but not configured instead of telling the user to > stop them manually. Thanks a lot Sandro, that should help, I am going to verify!
Test version: rhev-hypervisor7-7.0-20140904.0.iso ovirt-node-3.1.0-0.10.20140904gitb828c37.el7.noarch ovirt-hosted-engine-setup-1.2.0-0.2.master.el7.noarch ovirt-hosted-engine-ha-1.2.1-0.3.master.el7.noarch ovirt-node-plugin-hosted-engine-0.1.0-0.0.x86_64 ovirt-host-deploy-1.3.0-0.0.1.master.el7.noarch Test steps: 1. Clean install rhev-hypervisor7-7.0-20140904.0.iso 2. Configure network 3. #ps -aux |grep ovirt-ha 4. #ovirt-hosted-engine-setup Test result: 1. After step3, it still has Hosted Engine HA service running. # ps -aux |grep ovirt-ha vdsm 1377 0.0 0.1 136204 12188 ? S 03:18 0:00 /usr/bin/python /usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker 2. After step4, it still report error as follows. [ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services. So this issue is not fixed in rhev-hypervisor7-7.0-20140904.0.iso. Change the status to assign.
Change this bug back to MODIFIED, because there is no updated ovirt-hosted-engine-setup package in downstream in build rhev-hypervisor7-7.0-20140904.0.iso. Here needinfo Sandro, when are you going to rebuild downstream packages for QE testing? then RHEVH will pick them into.
Verified ovirt-hosted-engine-setup-1.2.0-1.el7.noarch Finish deploy process and have running hosted-engine environment.
Ok installation was done on clean host, if I try to redeploy it failed with the above message(reboot) not help, after erase of all packages include vdsm and restart host, hosted-engine --deploy works again.
Created attachment 944040 [details] setup log
I'm not sure it should fail the QA: preventing to redeploy on an host that has already been deployed it's just the aim of that check. So it should work and a clean system while it should fail on an host witch has already been deployed with this error: [ERROR] Failed to execute stage 'Programs detection': Hosted Engine HA services are already running on this system. Hosted Engine cannot be deployed on a host already running those services. The problem is another: there is not an easy way to completely cleanup a deployed system and you have to manually remove packages to deconfigure them. We have another RFE for that: please see https://bugzilla.redhat.com/1001181 till that we have to manually cleanup a deployed system before we can deploy it a second time.
Verified on ovirt-hosted-engine-setup-1.2.0-1.el7.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-0161.html