Description of problem: The self-hosted engine setup requires ssh pubkey access from the engine VM into the root account of the bare metal host/node. If the user sets 'PermitRootLogin no' in sshd_config of the host/node, then the engine deployment will hang (and eventually time out) at 'Waiting for the host to be up'. There are no clear error messages in the logs about the possible cause of the problem. Version-Release number of selected component (if applicable): # rpm -qf /usr/sbin/hosted-engine ovirt-hosted-engine-setup-2.4.4-1.el8.noarch How reproducible: Always Steps to Reproduce: 1. Per the ovirt 4.4 installation instructions, install a minimal CentOS 8 host, or an oVirt node. 2. Set 'PermitRootLogin no' in /etc/ssh/sshd_config and restart sshd 3. Continue the ovirt 4.4 installation and run 'hosted-engine --deploy' Actual results: Installation hangs, and eventually times and fails directing the user to check the ansible logs: [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/pass word credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Wait for the host to be up] [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": false, "ov irt_hosts": []} [...cleanup...] [ INFO ] TASK [ovirt.hosted_engine_setup : Notify the user about a failure] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} Expected results: Installation fails with a message (either on stdout or in a log file) that says there is a ssh problem between the newly deployed engine and the bare metal host/node. Additional info: In addition to clarifying the error message/logs in the hosted-engine application, it would be nice to have the online documentation describe the sshd config requirements for the host/node. Something along the lines of: "The oVirt engine requires root ssh access into the hypervisor host/node. ssh is done using pubkey authentication. At a minimum you will need to allow ssh pubkey auth into the root account of the hypervisor host/node with the following ssh configuration: Match Address engine.ip.address PermitRootLogin without-password PubkeyAuthentication yes Setting a less restrictive 'PermitRootLogin yes' at the global level will also work. "
Following another post, by Paul-Erik Törrönen, on the thread leading to creation of this bug: We should also mention that sshd must allow using RSA keypairs for authentication. You can't use e.g. ed25519.
Comment 0 describes three different bugs: 1. There are no clear error messages in the logs about the possible cause of the problem. 2. The deployment script should state that ssh public key access to root is required. 3. The documentation should state pubkey access as a prerequisite. This current bug only addresses documentation. I can very easily add a prerequisite to the following topics: - Deploying the Self-hosted Engine using Cockpit [1] - Deploying the Self-Hosted Engine Using the Command Line [2] How's this for the text of the prerequisite: ---- The self-hosted engine setup script requires ssh public key access from the engine virtual machine to the root account of its bare metal host. In `/etc/ssh/sshd_config`, ensure that these values are set as follows: * `PermitRoolLogin` is set to `without-password` or `yes` * `PubkeyAuthentication` is set to `yes` * `HostKey` is set to `/etc/ssh/ssh_host_rsa_key` ---- [1] https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html-single/installing_red_hat_virtualization_as_a_self-hosted_engine_using_the_cockpit_web_interface/index#Deploying_the_Self-Hosted_Engine_Using_Cockpit_install_RHVM [2] https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html-single/installing_red_hat_virtualization_as_a_self-hosted_engine_using_the_command_line/index#Deploying_the_Self-Hosted_Engine_Using_the_CLI_install_RHVM
The documentation text flag should only be set after 'doc text' field is provided. Please provide the documentation text and set the flag to '?' again.
(In reply to Steve Goodman from comment #2) > Comment 0 describes three different bugs: > 1. There are no clear error messages in the logs about the possible cause of > the problem. That's indeed a separate issue, perhaps worth its own bug. I hope to have at least partially improved this in bug 1893385, let's see. > 2. The deployment script should state that ssh public key access to root is > required. Perhaps this makes sense as well, but let's assume that it's a doc issue and is part of current bug. > 3. The documentation should state pubkey access as a prerequisite. > > This current bug only addresses documentation. > > I can very easily add a prerequisite to the following topics: > - Deploying the Self-hosted Engine using Cockpit [1] > - Deploying the Self-Hosted Engine Using the Command Line [2] > > How's this for the text of the prerequisite: > > ---- > The self-hosted engine setup script requires ssh public key access from the > engine virtual machine to the root account of its bare metal host. In > `/etc/ssh/sshd_config`, ensure that these values are set as follows: > * `PermitRoolLogin` is set to `without-password` or `yes` > * `PubkeyAuthentication` is set to `yes` +1 > * `HostKey` is set to `/etc/ssh/ssh_host_rsa_key` Not sure this is the point raised in comment 1 (and [1], originally). He didn't provide details, and it's a bit hard to guess. I think he changed PubkeyAcceptedKeyTypes, perhaps via crypto-policies (which is also mentioned in sshd_config manpage for this option). I'd say something more general, such as: "... requires ssh public key access using 2048 bit RSA keys", and then mention crypto-policies, e.g. * PubkeyAcceptedKeyTypes should allow 2048 bit RSA keys. It is controlled by default using system-wide crypto policies. See also the manual page crypto-policies(7). [1] https://lists.ovirt.org/archives/list/users@ovirt.org/message/PYY7KT3UYCLDPPPBEK2ZDFEY3AEKYJW5/ > ---- > > [1] > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/ > html-single/installing_red_hat_virtualization_as_a_self- > hosted_engine_using_the_cockpit_web_interface/index#Deploying_the_Self- > Hosted_Engine_Using_Cockpit_install_RHVM > [2] > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/ > html-single/installing_red_hat_virtualization_as_a_self- > hosted_engine_using_the_command_line/index#Deploying_the_Self- > Hosted_Engine_Using_the_CLI_install_RHVM
> > * `HostKey` is set to `/etc/ssh/ssh_host_rsa_key` > > Not sure this is the point raised in comment 1 (and [1], originally). > > He didn't provide details, and it's a bit hard to guess. I think he changed > PubkeyAcceptedKeyTypes, perhaps via crypto-policies (which is also mentioned > in sshd_config manpage for this option). > > I'd say something more general, such as: > > "... requires ssh public key access using 2048 bit RSA keys", and then > mention crypto-policies, e.g. > > * PubkeyAcceptedKeyTypes should allow 2048 bit RSA keys. It is controlled by > default using system-wide crypto policies. See also the manual page > crypto-policies(7). > > [1] > https://lists.ovirt.org/archives/list/users@ovirt.org/message/ > PYY7KT3UYCLDPPPBEK2ZDFEY3AEKYJW5/ > > > ---- > > > > [1] > > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/ > > html-single/installing_red_hat_virtualization_as_a_self- > > hosted_engine_using_the_cockpit_web_interface/index#Deploying_the_Self- > > Hosted_Engine_Using_Cockpit_install_RHVM > > [2] > > https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/ > > html-single/installing_red_hat_virtualization_as_a_self- > > hosted_engine_using_the_command_line/index#Deploying_the_Self- > > Hosted_Engine_Using_the_CLI_install_RHVM Wart, can you comment?
(In reply to Yedidyah Bar David from comment #4) > (In reply to Steve Goodman from comment #2) > > Comment 0 describes three different bugs: > > 1. There are no clear error messages in the logs about the possible cause of > > the problem. > > That's indeed a separate issue, perhaps worth its own bug. I hope to have at > least partially improved this in bug 1893385, let's see. > > > 2. The deployment script should state that ssh public key access to root is > > required. > > Perhaps this makes sense as well, but let's assume that it's a doc issue and > is part of current bug. > > > 3. The documentation should state pubkey access as a prerequisite. > > > > This current bug only addresses documentation. > > > > I can very easily add a prerequisite to the following topics: > > - Deploying the Self-hosted Engine using Cockpit [1] > > - Deploying the Self-Hosted Engine Using the Command Line [2] > > > > How's this for the text of the prerequisite: > > > > ---- > > The self-hosted engine setup script requires ssh public key access from the > > engine virtual machine to the root account of its bare metal host. In > > `/etc/ssh/sshd_config`, ensure that these values are set as follows: > > * `PermitRoolLogin` is set to `without-password` or `yes` > > * `PubkeyAuthentication` is set to `yes` > > +1 +1 from me as well. > > * `HostKey` is set to `/etc/ssh/ssh_host_rsa_key` > > Not sure this is the point raised in comment 1 (and [1], originally). > > He didn't provide details, and it's a bit hard to guess. I think he changed > PubkeyAcceptedKeyTypes, perhaps via crypto-policies (which is also mentioned > in sshd_config manpage for this option). [...] In my case, it was the 'PermitRootLogin = no' that was causing the login failure. I had not made any modifications to the HostKey setting, but it would be good to state if there are requirements for that as well.
Re-targeting to 4.4.7 since oVirt 4.4.6 was released.
This issue is past ovirt-4.4.8-1 development cycle. Moving to ovirt-4.4.9. If you believe this issue should be closed or moved back to 4.4.8-1 please do so. Otherwise, I would like to ask for updating tickets ahead of the development cycle deadline (release date).
Richard, can you please do a peer review?
Two small comments -- otherwise, LGTM.
Merged.
Published to: https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html-single/installing_red_hat_virtualization_as_a_self-hosted_engine_using_the_command_line/index#Deploying_the_Self-Hosted_Engine_Using_the_CLI_install_RHVM https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html-single/installing_red_hat_virtualization_as_a_self-hosted_engine_using_the_cockpit_web_interface/index#Deploying_the_Self-Hosted_Engine_Using_Cockpit_install_RHVM