Bug 1810018 - [IPI] [OSP] openshift-install got failed for floatingip creation if rhcos base image uploading take more than 10m0s
Summary: [IPI] [OSP] openshift-install got failed for floatingip creation if rhcos bas...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.3.z
Assignee: Mike Fedosin
QA Contact: David Sanz
URL:
Whiteboard:
Depends On: 1780534
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-04 12:17 UTC by weiwei jiang
Modified: 2020-03-24 14:34 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-24 14:34:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 3281 0 None closed [release-4.3] Bug 1810018: create bootstrap floating ip after the bootstrap machine 2020-03-17 00:44:54 UTC
Red Hat Product Errata RHBA-2020:0858 0 None None None 2020-03-24 14:34:52 UTC

Description weiwei jiang 2020-03-04 12:17:00 UTC
This bug was initially created as a copy of Bug #1780534

I am copying this bug because: 



Description of problem:
openshift-install got failed if rhcos base image upload take more than 10m0s to upload 
level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Still creating... [9m40s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m20s elapsed]"
level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Still creating... [9m50s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m30s elapsed]"
level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Still creating... [10m0s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m40s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m50s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [11m0s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Creation complete after 11m8s [id=06881570-3772-44b7-a288-025e85eda912]"
level=debug msg="data.openstack_images_image_v2.base_image: Refreshing state..."
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Creating..."
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [10s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [30s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [30s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [30s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [30s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [50s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [50s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [50s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [50s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Creation complete after 1m1s [id=3d0135f5-b182-43e6-a33a-aed42c75ed95]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Creation complete after 1m3s [id=a315e92e-4bd1-469d-b584-29ff35dff579]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Creation complete after 1m3s [id=7edc5a1b-2970-4f75-9f35-613f95800b2d]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [1m10s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Creation complete after 1m13s [id=af06cfb2-37e4-4eb4-857e-93867f3cec74]"
level=error
level=error msg="Error: Error waiting for openstack_networking_floatingip_v2 926cc4a4-7adb-45d9-8f06-df644c8356f8 to become available: timeout while waiting for state to become 'ACTIVE, DOWN' (timeout: 10m0s)"
level=error
level=error msg="  on ../../../../../tmp/openshift-install-994566403/bootstrap/main.tf line 27, in resource \"openstack_networking_floatingip_v2\" \"bootstrap_fip\":"
level=error msg="  27: resource \"openstack_networking_floatingip_v2\" \"bootstrap_fip\" {"
level=error
level=error
level=fatal msg="failed to fetch Cluster: failed to generate asset \"Cluster\": failed to create cluster: failed to apply using Terraform"


Version-Release number of the following components:
4.3.0-0.nightly-2019-12-06-014514

How reproducible:
It happen when rhcos base image take more than 10m0s to upload

Steps to Reproduce:
1. Create ipi on osp cluster and make rhcos base image uploading take more than 10m0s
2.
3.

Actual results:
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m20s elapsed]"
level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Still creating... [9m50s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m30s elapsed]"
level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Still creating... [10m0s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m40s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m50s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [11m0s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Creation complete after 11m8s [id=06881570-3772-44b7-a288-025e85eda912]"
level=debug msg="data.openstack_images_image_v2.base_image: Refreshing state..."
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Creating..."
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [10s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [30s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [30s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [30s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [30s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [50s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [50s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [50s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [50s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Creation complete after 1m1s [id=3d0135f5-b182-43e6-a33a-aed42c75ed95]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Creation complete after 1m3s [id=a315e92e-4bd1-469d-b584-29ff35dff579]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Creation complete after 1m3s [id=7edc5a1b-2970-4f75-9f35-613f95800b2d]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [1m10s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Creation complete after 1m13s [id=af06cfb2-37e4-4eb4-857e-93867f3cec74]"
level=error
level=error msg="Error: Error waiting for openstack_networking_floatingip_v2 926cc4a4-7adb-45d9-8f06-df644c8356f8 to become available: timeout while waiting for state to become 'ACTIVE, DOWN' (timeout: 10m0s)"
level=error
level=error msg="  on ../../../../../tmp/openshift-install-994566403/bootstrap/main.tf line 27, in resource \"openstack_networking_floatingip_v2\" \"bootstrap_fip\":"
level=error msg="  27: resource \"openstack_networking_floatingip_v2\" \"bootstrap_fip\" {"
level=error
level=error
level=fatal msg="failed to fetch Cluster: failed to generate asset \"Cluster\": failed to create cluster: failed to apply using Terraform"

Expected results:
It should not fail

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 4 weiwei jiang 2020-03-16 02:28:49 UTC
Checked with 4.3.0-0.nightly-2020-03-15-112942, now bootstrap fip is created after base_image done


level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [8m30s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [8m40s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [8m50s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [9m0s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [9m10s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [9m20s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [9m30s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [9m40s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [9m50s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m0s elapsed]"
level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m10s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m20s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m30s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m40s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [10m50s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [11m0s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [11m10s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [11m20s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Still creating... [11m30s elapsed]"

level=debug msg="openstack_images_image_v2.base_image[0]: Creation complete after 11m40s [id=89b605f9-9a47-48e9-8606-844a7bb636a1]"
level=debug msg="data.openstack_images_image_v2.base_image: Refreshing state..."

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Creating..."
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Creating..."
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Creating..."

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [10s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [10s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [10s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [20s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [20s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [20s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [30s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [30s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [30s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [30s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [40s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [40s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [40s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [50s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [50s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [50s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [50s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [1m0s elapsed]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Still creating... [1m0s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [1m0s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[2]: Creation complete after 1m3s [id=52735ea5-9801-4f8e-950d-23f3d4d4f6c4]"
level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[1]: Creation complete after 1m3s [id=b8d49367-536f-4eee-8c12-891eef17554e]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Still creating... [1m10s elapsed]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Still creating... [1m10s elapsed]"

level=debug msg="module.masters.openstack_compute_instance_v2.master_conf[0]: Creation complete after 1m13s [id=0d9e63b6-182b-44ea-a416-e3ddc0f8d8bb]"
level=debug msg="module.bootstrap.openstack_compute_instance_v2.bootstrap: Creation complete after 1m13s [id=0ac209cf-9d5b-4b37-aed6-7fde016fc2e3]"
level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Creating..."

level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Still creating... [10s elapsed]"

level=debug msg="module.bootstrap.openstack_networking_floatingip_v2.bootstrap_fip: Creation complete after 11s [id=e6a7ab84-6349-4ee1-9ee3-00361b6bc6ec]"
level=debug
level=debug msg="Apply complete! Resources: 72 added, 0 changed, 0 destroyed."

Comment 6 errata-xmlrpc 2020-03-24 14:34:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0858


Note You need to log in before you can comment on or make changes to this bug.