Bug 1772477 - [IPI][Openstack] First attempt to destroy cluster sometimes fails with msg="Resource not found"
Summary: [IPI][Openstack] First attempt to destroy cluster sometimes fails with msg="R...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.2.z
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.3.0
Assignee: Pierre Prinetti
QA Contact: David Sanz
URL:
Whiteboard:
: 1773294 (view as bug list)
Depends On:
Blocks: 1773909
TreeView+ depends on / blocked
 
Reported: 2019-11-14 12:48 UTC by Filip Brychta
Modified: 2020-01-23 11:13 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1773909 (view as bug list)
Environment:
Last Closed: 2020-01-23 11:12:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
installation dir (1.03 MB, application/gzip)
2019-11-14 12:48 UTC, Filip Brychta
no flags Details
installation log (77.22 KB, text/plain)
2019-11-14 12:49 UTC, Filip Brychta
no flags Details
removal log from failed attempt (3.66 KB, text/plain)
2019-11-14 12:49 UTC, Filip Brychta
no flags Details
removal log from second successful attempt (9.57 KB, text/plain)
2019-11-14 12:50 UTC, Filip Brychta
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 2408 0 'None' closed Destroy cluster fails for a missing FIP 2020-08-13 03:41:22 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:13:06 UTC

Description Filip Brychta 2019-11-14 12:48:26 UTC
Created attachment 1636125 [details]
installation dir

Description of problem:
First attempt to destroy cluster in OpenStack sometimes fail with:
level=fatal msg="Resource not found"
Second attempt works fine and the cluster is removed.

Following error is shown:
level=debug msg="Deleting Floating IP: d1f931dd-6df1-4f56-ae0c-8095d64e5a7a"
level=fatal msg="Resource not found"


Version-Release number of the following components:
OCP 4.2.2

How reproducible:
Sometimes

Steps to Reproduce:
1. Install OCP 4.2.2 in OpenStack: openshift-install --log-level debug create cluster --dir /home/jenkins/workspace/install-ocp4-2-plus/openshift/cluster-installation/psi/fbr-42-t1
2. wait for installation to finish
3. try to destroy the cluster: openshift-install --log-level debug destroy cluster --dir /home/jenkins/workspace/destroy-ocp4-2-plus/openshift/cluster-installation/psi/fbr-42-t1

Actual results:
Removal fails.
Full logs attached but here is the error msg:
level=debug msg="Deleting Floating IP: d1f931dd-6df1-4f56-ae0c-8095d64e5a7a"
level=fatal msg="Resource not found"

Expected results:
Cluster is removed without any errors.

Additional info:
Also saw following error from different cluster:
level=debug msg="Deleting container: ocp-jqe2-lzc2g-image-registry-ptmfjgligviifudscepaghxhcthtggmu\n"
level=fatal msg="Resource not found"

I don't know if the err:
level=fatal msg="Resource not found" 
is related to previous message:
level=debug msg="Deleting Floating IP: d1f931dd-6df1-4f56-ae0c-8095d64e5a7a"
but if yes, I can see in install log that this IP was created by installer:
level=debug msg="module.topology.openstack_networking_floatingip_associate_v2.api_fip[0]: Creation complete after 5s [id=d1f931dd-6df1-4f56-ae0c-8095d64e5a7a]"

Full debug log from installation, from failed removal and from second successful removal are attached. Installation dir is attached too.

Comment 1 Filip Brychta 2019-11-14 12:49:01 UTC
Created attachment 1636126 [details]
installation log

Comment 2 Filip Brychta 2019-11-14 12:49:29 UTC
Created attachment 1636127 [details]
removal log from failed attempt

Comment 3 Filip Brychta 2019-11-14 12:50:09 UTC
Created attachment 1636128 [details]
removal log from second successful attempt

Comment 4 Martin André 2019-11-18 14:53:04 UTC
We've fixed a number of issues with the destroy command on Openstack platform in the master branch and I'm confident this is fixed in the 4.3 builds. We should consider backporting those fixes to 4.2.z.

Comment 5 Martin André 2019-11-18 14:57:11 UTC
*** Bug 1773294 has been marked as a duplicate of this bug. ***

Comment 6 Martin André 2019-11-18 15:45:55 UTC
This should have been fixed with https://github.com/openshift/installer/pull/2408

Targeting the bug to 4.3.0 so that QE can confirm the issue is fixed and we'll backport to 4.2.z after cloning the BZ.

Comment 7 Mike Fedosin 2019-11-21 12:36:14 UTC
https://github.com/openshift/installer/pull/2408 doesn't completely solve the issue and therefore https://github.com/openshift/installer/pull/2561 is also required

Comment 8 David Sanz 2019-12-02 12:00:09 UTC
Verified on 4.3.0-0.nightly-2019-12-02-055401

Comment 10 errata-xmlrpc 2020-01-23 11:12:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.