Bug 1415822
| Summary: | RHV self hosted deployment fails during ansible task [self_hosted_first_host : Execute hosted-engine setup] Failed to execute stage 'Environment setup': Cannot setup Hosted Engine with other VMs running | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Quickstart Cloud Installer | Reporter: | Landon LaSmith <llasmith> | ||||||
| Component: | Installation - RHEV | Assignee: | Derek Whatley <dwhatley> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Tasos Papaioannou <tpapaioa> | ||||||
| Severity: | unspecified | Docs Contact: | Dan Macpherson <dmacpher> | ||||||
| Priority: | unspecified | ||||||||
| Version: | 1.1 | CC: | bthurber, dymurray, fabian, qci-bugzillas, tpapaioa | ||||||
| Target Milestone: | --- | Keywords: | Triaged | ||||||
| Target Release: | 1.1 | ||||||||
| Hardware: | All | ||||||||
| OS: | All | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2017-02-28 01:45:26 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Just to note, it seems the "other VMs detected" issue is actually due to fusor repeatedly + automatically retrying the ansible-ovirt deployment. The original error that caused the failure comes after the engine VM has been successfully set up (hence the vm detection in future runs):
[ INFO ] Engine-setup successfully completed
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ INFO ] Engine is still not reachable, waiting...
[ INFO ] Engine is still unreachable
[ ERROR ] Engine is still not reachable
[ ERROR ] Failed to execute stage 'Closing up': Engine is still not reachable
[ INFO ] Stage: Clean up
[ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20170123113113.conf'
[ INFO ] Stage: Pre-termination
[ INFO ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed: this system is not reliable, please check the issue,fix and redeploy
Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20170123105043-kvzx84.log
It seems the engine VM fails to come back up after being launched, though I don't yet see any reason for that to happen.
Created attachment 1245909 [details]
self-hosted engine yum update output
Issue may be related to self-hosted engine being updated via `yum update` command during VM bringup.
Moved self-hosted engine provisioning from cloud-init to Ansible. https://github.com/fusor/ansible-ovirt/pull/23 https://github.com/fusor/fusor/pull/1289 This fix is in QCI-1.1-RHEL-7-20170130.t.0. Verified on QCI-1.1-RHEL-7-20170130.t.0. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:0335 |
Created attachment 1243756 [details] Deployment logs Description of problem: During a clean deployment of RHV self hosted, the deployment fails during the ansible task [self_hosted_first_host : Execute hosted-engine setup] with the error below [ INFO ] Stage: Initializing [ INFO ] Generating a temporary VNC password. [ INFO ] Stage: Environment setup During customization use CTRL-D to abort. [ INFO ] Hardware supports virtualization Configuration files: ['/etc/qci//answers'] Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20170123133110-h66rr6.log Version: otopi-1.5.2 (otopi-1.5.2-1.el7ev) [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup [WARNING] Cannot locate gluster packages, Hyper Converged setup support will be disabled. [ INFO ] Please abort the setup and install vdsm-gluster, gluster-server >= 3.7.2 and restart vdsmd service in order to gain Hyper Converged setup support. [ ERROR ] The following VMs have been found: 26e0c0b5-431c-4b39-b90c-b60ad4a6c093 [ ERROR ] Failed to execute stage 'Environment setup': Cannot setup Hosted Engine with other VMs running [ INFO ] Stage: Clean up [ INFO ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20170123133121.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Hosted Engine deployment failed Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20170123133110-h66rr6.log QCI Media Version: QCI-1.1-RHEL-7-20170120.t.0 How reproducible: A few occurrences during automated runs Steps to Reproduce: 1. Install QCI from ISO 2. Start a RHV selfhosted deployment Actual results: RHV deployment fails during hosted engine setup Expected results: RHV deployment foreman task succeeds in setting up the hosted engine VM