Description of problem: If adding new host with newly installed RHEL7.2 (with all needed repos, of course) through Python SDK installation of host always ends as failed. On the other hand, adding this clean host to engine trough web-interface works fine. Version-Release number of selected component (if applicable): Engine version: 4.0.2.6-0.1.el7ev Python SDK version: latest 4.0 from github.com How reproducible: 100% Steps to Reproduce: 1. get some hosts 2. get engine 3. add this hosts through python-sdk Actual results: Host installation fails. Expected results: Host is up. Additional info: Script I'm using to add hosts: https://gist.github.com/slaykovsky/3bdcaa305edd4d04f1b38b11ac65615e
(In reply to Aleksei Slaikovskii from comment #0) > Description of problem: > If adding new host with newly installed RHEL7.2 (with all needed repos, of > course) through Python SDK installation of host always ends as failed. On > the other hand, adding this clean host to engine trough web-interface works > fine. > > Version-Release number of selected component (if applicable): > Engine version: 4.0.2.6-0.1.el7ev > Python SDK version: latest 4.0 from github.com > > How reproducible: > 100% > > Steps to Reproduce: > 1. get some hosts > 2. get engine > 3. add this hosts through python-sdk > > Actual results: > Host installation fails. > > Expected results: > Host is up. > > Additional info: > Script I'm using to add hosts: > https://gist.github.com/slaykovsky/3bdcaa305edd4d04f1b38b11ac65615e There's vdsm on hosts: vdsm-4.18.11-1.el7ev.x86_64(In reply to Aleksei Slaikovskii from comment #0) > Description of problem: > If adding new host with newly installed RHEL7.2 (with all needed repos, of > course) through Python SDK installation of host always ends as failed. On > the other hand, adding this clean host to engine trough web-interface works > fine. > > Version-Release number of selected component (if applicable): > Engine version: 4.0.2.6-0.1.el7ev > Python SDK version: latest 4.0 from github.com > > How reproducible: > 100% > > Steps to Reproduce: > 1. get some hosts > 2. get engine > 3. add this hosts through python-sdk > > Actual results: > Host installation fails. > > Expected results: > Host is up. > > Additional info: > Script I'm using to add hosts: > https://gist.github.com/slaykovsky/3bdcaa305edd4d04f1b38b11ac65615e vdsm-4.18.11-1.el7ev.x86_64
Can you attach engine.log, server.log?
Created attachment 1191124 [details] engine.log-20160816.xz and server.log
Which server is it that you are trying to install? Assuming it's _test_0, you can find its logs (on the engine) at /var/log/ovirt-engine/host-deploy/ovirt-host-deploy-20160815144925-10.34.61.74-6d6f1229.log (and in general, in any deploy failures, you can look @ /var/log/ovirt-engine/host-deploy/ to find out more information).
(In reply to Yaniv Kaul from comment #4) > Which server is it that you are trying to install? And you clearly have other issues that need to be solved: 2016-08-15 14:40:10,766 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-5) [8e8e747] Host installation failed for host '34b2d3ab-77a8-4275-84ba-2ff6bf83d818', 'aslaikov_test_3':Host is not reachable Let's try to narrow down the scenario (and the log file - I shouldn't be going over 60MB log file).
(In reply to Yaniv Kaul from comment #6) > (In reply to Yaniv Kaul from comment #4) > > Which server is it that you are trying to install? > > And you clearly have other issues that need to be solved: > 2016-08-15 14:40:10,766 ERROR > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] > (org.ovirt.thread.pool-6-thread-5) [8e8e747] Host installation failed for > host '34b2d3ab-77a8-4275-84ba-2ff6bf83d818', 'aslaikov_test_3':Host is not > reachable Yes, installation fails because of "host is not reachable". > Let's try to narrow down the scenario (and the log file - I shouldn't be > going over 60MB log file). Ok, I will try to install the other one now and give you logs.
Created attachment 1191185 [details] host-deploy, engine, server and vdsm logs.
Looking at the engine log I can see that during protocol detection in the engine. Both protocol failed to connect to vdsm due to: Caused by: java.net.NoRouteToHostException: No route to host I do not see in host deploy log that firewall was configured. Please make sure that during host installation firewall is reconfigured to allow engine <-> vdsm communication or open 54321 port manually.
So this problem was successfully fixed by adding override_iptables=True to Host constructor. E.g. like: h = Host(name=name, address=address, root_password=self._host_password, override_iptables=True)
So I close it as it's not a bug eventually :)
We need to make sure it's the default, or added. Just failed our Ansible scripts, for example.
Yaniv, please provide engine, host deploy and vdsm logs for your failure when using Ansible scripts.
Created attachment 1250148 [details] engine log- fail on api vs success on ui
(In reply to Piotr Kliczewski from comment #14) > Yaniv, please provide engine, host deploy and vdsm logs for your failure > when using Ansible scripts. Piotr - no need. Just revert https://gerrit.ovirt.org/#/c/71772/3/basic-suite-4.1/test-scenarios/002_bootstrap.py and see it happening in ovirt-system-test. Without setting the iptables, it'll fail installation.
Yaniv, if we do not update firewall settings we assume that admin need to do it manually. It seems to be expected behaviour.
(In reply to Piotr Kliczewski from comment #17) > Yaniv, if we do not update firewall settings we assume that admin need to do > it manually. It seems to be expected behaviour. Right, if you do not allow overriding iptables during host-deploy, we expect that admin already performed this change manually. Inside UI code we have a checkbox (checked by default) which enables overriding iptables on host during installation. AFAIK SDK does not send the default value for override_iptables parameter, the default is stored on backend in VdsOperationActionParameters. Also as SDKs are autogenerated, we cannot easily add default value into each SDK. So the only way how to change it is to change the default in backend class VdsOperationActionParameters. But not sure if we don't break backward compatibility of API by that change. @Juan?
Changing default values breaks backwards compatibility. If we take that seriously then we shouldn't change them, till version 5 of the API. The only thing we can (and probably should) do in version 4 of the API is document it correctly. Currently it isn't.
(In reply to Juan Hernández from comment #19) > Changing default values breaks backwards compatibility. If we take that > seriously then we shouldn't change them, till version 5 of the API. The only > thing we can (and probably should) do in version 4 of the API is document it > correctly. Currently it isn't. I think in this case, we can make an exception to the rule.
No objection, if it is correctly documented, in a release note and in the specification of the API.
Martin - I just want to make sure that Ansible is going to be OK, when we GA (or when Ansible 2.3 does).
(In reply to Yaniv Kaul from comment #22) > Martin - I just want to make sure that Ansible is going to be OK, when we GA > (or when Ansible 2.3 does). Ansible is behaving the same way as SDK: if you don't specify override_iptables option when invoking ovirt_hosts module, the default value is defined in backend code (same as for API request). ovirt_hosts module doesn't contain any default value for that option. So when we change default on backend it will not affect only API/SDKs but also Ansible module. Unfortunately we don't have 'override_iptables: True' in examples inside ovirt_hosts module, but we will post a PR to fix that (hopefully it will get into Ansible 2.3.0 release).
It makes sense to override firewall rules by default also when reinstalling the host. With this change we will have the same behaviour in both UI and API.
Verified on python-ovirt-engine-sdk4-4.1.3-1.el7ev.x86_64