Bug 1576451 - ovirt-hosted-engine-cleanup must undefine the HostedEngine VM
Summary: ovirt-hosted-engine-cleanup must undefine the HostedEngine VM
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-hosted-engine-setup
Classification: oVirt
Component: Tools
Version: 2.2.19
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ovirt-4.2.4
: 2.2.21
Assignee: Simone Tiraboschi
QA Contact: Nikolai Sednev
URL:
Whiteboard:
Depends On:
Blocks: ovirt-hosted-engine-setup-2.2.22
TreeView+ depends on / blocked
 
Reported: 2018-05-09 13:33 UTC by Polina
Modified: 2018-06-26 08:38 UTC (History)
2 users (show)

Fixed In Version: ovirt-hosted-engine-setup-2.2.21
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-26 08:38:10 UTC
oVirt Team: Integration
rule-engine: ovirt-4.2+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 91102 0 master MERGED cleanup: properly undefine HE VM on cleanup 2020-07-23 18:21:38 UTC
oVirt gerrit 91148 0 ovirt-hosted-engine-setup-2.2 MERGED cleanup: properly undefine HE VM on cleanup 2020-07-23 18:21:38 UTC

Description Polina 2018-05-09 13:33:16 UTC
Description of problem:
If the previous deploy was failed with some error, and the cleanup procedure was run, the deploy fails with error 
"ERROR The MAC address '...' is in use by another virtual machine."
Running the  <virsh -r list --all> we find that HostedEngine machine is still defined with State=" shut off"

Version-Release number of selected component (if applicable):
rhv-release-4.2.3-6-001.noarch

How reproducible:100%

Steps to Reproduce:
1. Run hosted-engine deploy(let's say it fails with error).
2. Run /usr/sbin/ovirt-hosted-engine-cleanup.
3. Run hosted-engine deploy after the error is fixed.

Actual results:
The second deploy fails with error "ERROR The MAC address '...' is in use by another virtual machine" Run on host <virsh -r list --all> returns  HostedEngine shut off.

Expected results:
HostedEngine must be undefined after cleanup.

Additional info: workaround: manually run on host<virsh undefine HostedEngine>

Comment 1 Nikolai Sednev 2018-06-04 14:37:03 UTC
[ INFO  ] changed: [localhost]
[ INFO  ] TASK [Wait for the host to be up]
^C[ ERROR ] Failed to execute stage 'Closing up': SIG2
[ INFO  ] Stage: Clean up
[ INFO  ] Cleaning temporary resources
^C[ ERROR ] Failed to execute stage 'Clean up': SIG2
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20180604173237.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed: please check the logs for the issue, fix accordingly or re-deploy from scratch.
          Log file is located at /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20180604172012-vb0flx.log
[root@alma04 ~]# virsh -r list --all
 Id    Name                           State
----------------------------------------------------
 1     HostedEngineLocal              running

[root@alma04 ~]# ovirt-hosted-engine-cleanup
 This will de-configure the host to run ovirt-hosted-engine-setup from scratch. 
Caution, this operation should be used with care.

Are you sure you want to proceed? [y/n]
y
  -=== Destroy hosted-engine VM ===- 
You must run deploy first
  -=== Killing left-behind HostedEngine processes ===- 
error: failed to get domain 'HostedEngine'
error: Domain not found: no domain with matching name 'HostedEngine'

  -=== Stop HA services ===- 
  -=== Shutdown sanlock ===- 
shutdown force 1 wait 0
shutdown done 0
  -=== Disconnecting the hosted-engine storage domain ===- 
You must run deploy first
  -=== De-configure VDSM networks ===- 
  -=== Stop other services ===- 
  -=== De-configure external daemons ===- 
  -=== Removing configuration files ===- 
? /etc/init/libvirtd.conf already missing
- removing /etc/libvirt/nwfilter/vdsm-no-mac-spoofing.xml
? /etc/ovirt-hosted-engine/answers.conf already missing
? /etc/ovirt-hosted-engine/hosted-engine.conf already missing
- removing /etc/vdsm/vdsm.conf
- removing /etc/pki/vdsm/certs/cacert.pem
- removing /etc/pki/vdsm/certs/vdsmcert.pem
- removing /etc/pki/vdsm/keys/vdsmkey.pem
- removing /etc/pki/vdsm/libvirt-spice/ca-cert.pem
- removing /etc/pki/vdsm/libvirt-spice/server-cert.pem
- removing /etc/pki/vdsm/libvirt-spice/server-key.pem
- removing /etc/pki/CA/cacert.pem
- removing /etc/pki/libvirt/clientcert.pem
- removing /etc/pki/libvirt/private/clientkey.pem
? /etc/pki/ovirt-vmconsole/*.pem already missing
- removing /var/cache/libvirt/qemu
? /var/run/ovirt-hosted-engine-ha/* already missing

alma04 ~]# virsh -r list --all
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory

[root@alma04 ~]# hosted-engine --deploy 

successfully deployed.

Works for me on these components:
ovirt-hosted-engine-ha-2.2.13-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.22-1.el7ev.noarch
rhvm-appliance-4.2-20180601.0.el7.noarch
Red Hat Enterprise Linux Server release 7.5 (Maipo)
Linux 3.10.0-862.3.2.el7.x86_64 #1 SMP Tue May 15 18:22:15 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

Moving to verified.

Comment 2 Nikolai Sednev 2018-06-04 15:01:34 UTC
During redeployment I've failed with another error, hence opened separate bug https://bugzilla.redhat.com/show_bug.cgi?id=1585739.

Comment 3 Sandro Bonazzola 2018-06-26 08:38:10 UTC
This bugzilla is included in oVirt 4.2.4 release, published on June 26th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.