Created attachment 1841074 [details] sosreport from host alma03 the host Description of problem: Default "hosted-engine --deploy" fails on dualstack host, while "hosted-engine --deploy --4" works fine. [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Wait for the host to be up] [ INFO ] TASK [ovirt.ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Host is not up, please check logs, perhaps also on the engine machine"} [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} [ ERROR ] Failed to execute stage 'Closing up': Failed executing ansible-playbook [ ERROR ] Hosted Engine deployment failed: please check the logs for the issue, fix accordingly or re-deploy from scratch. Looks like that default deployment method is not working properly for otopi using IPv4. Version-Release number of selected component (if applicable): ovirt-hosted-engine-ha-2.4.9-1.el8ev.noarch ovirt-hosted-engine-setup-2.5.4-2.el8ev.noarch rhvm-appliance-4.4-20210827.0.el8ev.x86_64 Linux 4.18.0-348.1.1.el8_5.x86_64 #1 SMP Wed Nov 3 12:27:39 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux release 8.5 (Ootpa) How reproducible: 100% Steps to Reproduce: 1.Have a host with assigned IPv4 and IPv6 addresses, but it should work using IPv4 and be configured by DHCP. 2.Use FQDN for the engine VM which resolves by DNS with two IPs, one IPv4 and another IPv6. 3.Have NFS storage resolvable with IPv4 and IPv6 or simply use FC to omit this step. 4.Deploy HE using regular and default command "hosted-engine --deploy". 5.Deployment will fail, but if you will try to deploy using ""hosted-engine --deploy --4" it will pass. Actual results: Default deployment is not working properly and fails over IPv4 if environment is dualstack. Expected results: Default deployment should not fail on dualstack with working IPv4. Additional info: Sosreport is attached.
In this case the DNS resolution prefers IPv6, while IPv4 is used for the deployment. The issue is we have no way of telling if the user wants to run IPv4 or IPv6 deployment with "hosted-engine --deploy" without a specified "--4" or "--6".
(In reply to Asaf Rachmani from comment #3) > In this case the DNS resolution prefers IPv6, while IPv4 is used for the > deployment. > The issue is we have no way of telling if the user wants to run IPv4 or IPv6 > deployment with "hosted-engine --deploy" without a specified "--4" or "--6". I don't understand the need for this change. Why require it? In dual stack IPv6 is preferred, and that's what the user "wants". If they don't want it then they shouldn't configure IPv6, shouldn't resolve to IPv6 or pass -4 explicitly.
In HE deployment, IPv4 is preferred (also in dual-stack) and this is the reason that the deployment fails We don't really know what the user wants, why not be on the safe side by asking the users to specify --6 or --4?
where is IPv4 preferred? I specifically changed that in https://github.com/oVirt/ovirt-hosted-engine-setup/commit/ac444b5ddac08b89583085603fb67a7c4f80596a Everything on our hosts prefer IPv6, we should do so too by default. We do not really fully support dual stack, this change was for OST, and it does work flawlessly there (dual stack network, setup chooses IPv6-only automatically. We do not really have support for dual stack inside HE)
I prefer to revert this, there already is a way how to force one or the other - using he_force_ip4 and he_force_ip6 variables
So there are a few issues here, if the default should be IPv6 we need to suggest IPv6 default gateway and not IPv4 as we currently do. We should also avoid asking for DHCP since we don't support DHCPv6. We probably need to consider closing this bug as we don't support dual-stack, in the first place.
(In reply to Asaf Rachmani from comment #8) > So there are a few issues here, if the default should be IPv6 we need to > suggest IPv6 default gateway and not IPv4 as we currently do. possibly. indeed it shows ipv4 currently. Didi also suggested to make it more clear by using a WARN level where we say [ INFO ] The engine VM will be configured to use fd8f:1391:3a82:209::c0a8:d163/64 or even have another line somewhere to tell people what we selected. > We should also avoid asking for DHCP since we don't support DHCPv6. > We probably need to consider closing this bug as we don't support > dual-stack, in the first place. We just chose to ignore dual-stack and dhcpv6 because there are too many misconfigurations everywhere, and decided to support only IPv6 with static addresses. But if there's a chance it works in a properly configured dual-stack environments then so be it, we can leave the possibility open in the code. (keeping bug open for tracking the revert)
(In reply to Michal Skrivanek from comment #6) > where is IPv4 preferred? I specifically changed that in > https://github.com/oVirt/ovirt-hosted-engine-setup/commit/ > ac444b5ddac08b89583085603fb67a7c4f80596a > > Everything on our hosts prefer IPv6, we should do so too by default. We do > not really fully support dual stack, this change was for OST, and it does > work flawlessly there (dual stack network, setup chooses IPv6-only > automatically. We do not really have support for dual stack inside HE) Why do we have both "Force IPv4/IPv6 on dual stack env." options in the hosted-engine deployment menu if we do not really have support for dual stack inside HE? (In reply to Michal Skrivanek from comment #7) > I prefer to revert this, there already is a way how to force one or the > other - using he_force_ip4 and he_force_ip6 variables alma04 ~]# hosted-engine --deploy --help Usage: /usr/sbin/hosted-engine --deploy [args] Run ovirt-hosted-engine deployment. --4 Force IPv4 on dual stack env. --6 Force IPv6 on dual stack env. Do you mean "hosted-engine --deploy --4" = "he_force_ip4" and "hosted-engine --deploy --6" = "he_force_ip6"?
(In reply to Nikolai Sednev from comment #10) > (In reply to Michal Skrivanek from comment #6) > > where is IPv4 preferred? I specifically changed that in > > https://github.com/oVirt/ovirt-hosted-engine-setup/commit/ > > ac444b5ddac08b89583085603fb67a7c4f80596a > > > > Everything on our hosts prefer IPv6, we should do so too by default. We do > > not really fully support dual stack, this change was for OST, and it does > > work flawlessly there (dual stack network, setup chooses IPv6-only > > automatically. We do not really have support for dual stack inside HE) > > Why do we have both "Force IPv4/IPv6 on dual stack env." options in the > hosted-engine deployment menu if we do not really have support for dual > stack inside HE? what menu? if you're referring to cockpit it has been dropped from RHV and only CLI is supported. > (In reply to Michal Skrivanek from comment #7) > > I prefer to revert this, there already is a way how to force one or the > > other - using he_force_ip4 and he_force_ip6 variables > > alma04 ~]# hosted-engine --deploy --help > Usage: /usr/sbin/hosted-engine --deploy [args] > Run ovirt-hosted-engine deployment. > > --4 > Force IPv4 on dual stack env. > --6 > Force IPv6 on dual stack env. > > Do you mean "hosted-engine --deploy --4" = "he_force_ip4" and "hosted-engine > --deploy --6" = "he_force_ip6"? There are --4 and --6 parameters passing the variable to ansible role. The change here was just about enforcing them.
we can't support all the weird combinations of dual-stack networks. If there is IPv6 we presume it's configured correctly, if not the setup is going to fail unless you explicitly choose e.g. --4 to force IPv4.
(In reply to Michal Skrivanek from comment #12) > we can't support all the weird combinations of dual-stack networks. If there > is IPv6 we presume it's configured correctly, if not the setup is going to > fail unless you explicitly choose e.g. --4 to force IPv4. Then you should probably close https://bugzilla.redhat.com/show_bug.cgi?id=2048416 as WONTFIX too. (In reply to Michal Skrivanek from comment #11) > (In reply to Nikolai Sednev from comment #10) > > (In reply to Michal Skrivanek from comment #6) > > > where is IPv4 preferred? I specifically changed that in > > > https://github.com/oVirt/ovirt-hosted-engine-setup/commit/ > > > ac444b5ddac08b89583085603fb67a7c4f80596a > > > > > > Everything on our hosts prefer IPv6, we should do so too by default. We do > > > not really fully support dual stack, this change was for OST, and it does > > > work flawlessly there (dual stack network, setup chooses IPv6-only > > > automatically. We do not really have support for dual stack inside HE) > > > > Why do we have both "Force IPv4/IPv6 on dual stack env." options in the > > hosted-engine deployment menu if we do not really have support for dual > > stack inside HE? > > what menu? CLI, alma04 ~]# hosted-engine --deploy --help Usage: /usr/sbin/hosted-engine --deploy [args] Run ovirt-hosted-engine deployment. --config-append=<file> Load extra configuration files. --generate-answer=<file> Generate answer file. --restore-from-file=<file> Restore an engine backup file during the deployment. --4 Force IPv4 on dual stack env. --6 Force IPv6 on dual stack env. --ansible-extra-vars=DATA Pass '--extra-vars=DATA' to all ansible calls. DATA can be anything that ansible can accept - var=value, @file, JSON/YAML. Please note: Using this option is supported only with specific values as documented elsewhere. Passing arbitrary values might conflict with existing variables. > if you're referring to cockpit it has been dropped from RHV and only CLI is > supported. > > > > (In reply to Michal Skrivanek from comment #7) > > > I prefer to revert this, there already is a way how to force one or the > > > other - using he_force_ip4 and he_force_ip6 variables > > > > alma04 ~]# hosted-engine --deploy --help > > Usage: /usr/sbin/hosted-engine --deploy [args] > > Run ovirt-hosted-engine deployment. > > > > --4 > > Force IPv4 on dual stack env. > > --6 > > Force IPv6 on dual stack env. > > > > Do you mean "hosted-engine --deploy --4" = "he_force_ip4" and "hosted-engine > > --deploy --6" = "he_force_ip6"? > > There are --4 and --6 parameters passing the variable to ansible role. The > change here was just about enforcing them. That was exactly what I used during the deployment, --4 and in that case it worked fine.
*** Bug 2066416 has been marked as a duplicate of this bug. ***
Oh wow - we should really document --4 and --6 in the official docs somewhere. It's nice that it's in the help, but let's put all the different switches and possibilities in the docs.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days