Bug 1571818 – Cloud-init randomly fail to get nova metadata

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1571818 - Cloud-init randomly fail to get nova metadata

Summary:	Cloud-init randomly fail to get nova metadata

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	python-novajoin (Show other bugs)
Sub Component:
Version:	13.0 (Queens)
Hardware:	Unspecified Unspecified

Priority	high Severity high
Target Milestone:	beta
Target Release:	13.0 (Queens)
Assigned To:	Ade Lee
QA Contact:	Prasanth Anbalagan
Docs Contact:

URL:
Whiteboard:
Keywords:	Triaged

Depends On:
Blocks:
	Show dependency tree / graph

Reported:	2018-04-25 09:13 EDT by Marian Krcmarik
Modified:	2018-06-27 09:55 EDT (History)
CC List:	7 users (show)

See Also:
Fixed In Version:	python-novajoin-1.0.18-2.el7ost
Doc Type:	If docs needed, set a value
Doc Text:
Story Points:	---
Clone Of:
Environment:
Last Closed:	2018-06-27 09:53:37 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Launchpad	1766824	None	None	None	2018-04-25 09:13 EDT
OpenStack gerrit	564137	None	master: MERGED	novajoin: Add a retry loop when fetching vendor_data from novajoin (I728c2cb0c8a7433b68dd7de2de242e922974d713)	2018-05-03 08:04 EDT
Red Hat Product Errata	RHEA-2018:2086	None	None	None	2018-06-27 09:55 EDT

Groups: None (edit)

Description Marian Krcmarik 2018-04-25 09:13:35 EDT

Description of problem:
From time to time it seems that we experience contention on novajoin, which makes cloud-init fail to retrieve metadata when doing:

curl -s http://169.254.169.254/openstack/2016-10-06/vendor_data2.json

To workaround the chances of contention we currently have a random sleep before that call so lower the odds that many nodes call novajoin at the same time.

This random sleep reduces the failure rate at deploy time but we can still experience it fairly often.

The end result often is that overcloud nodes end up without cert and overcloud deployment fails, the probability raises with number of nodes

Version-Release number of selected component (if applicable):


How reproducible:
often

Steps to Reproduce:
1. Deploy overcloud of Openstack with TLS everywhere.

Actual results:
Overcloud deployment fails since some certs are missing on some overcloud nodes


Additional info:

Comment 1 Damien Ciabrini 2018-04-25 10:46:38 EDT

Proposed https://review.openstack.org/#/c/564137/3 to overcome the failures.

Comment 9 errata-xmlrpc 2018-06-27 09:53:37 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086

Note You need to log in before you can comment on or make changes to this bug.