1982002 – [4.8.z] On a Azure IPI installation MCO fails to create new nodes

Bug 1982002 - [4.8.z] On a Azure IPI installation MCO fails to create new nodes

Summary: [4.8.z] On a Azure IPI installation MCO fails to create new nodes

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	RHCOS
Sub Component:
Version:	4.7
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	medium
Target Milestone:	---
Target Release:	4.8.z
Assignee:	Benjamin Gilbert
QA Contact:	Michael Nguyen
Docs Contact:
URL:
Whiteboard:
Depends On:	1980679 1982001
Blocks:	1982003 1982004
TreeView+	depends on / blocked

Reported:	2021-07-13 23:29 UTC by Benjamin Gilbert
Modified:	2021-10-12 06:01 UTC (History)
CC List:	8 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1980679
Clones:	1982003 (view as bug list)
Environment:
Last Closed:	2021-10-12 06:01:19 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Github	openshift os pull 582	0	None	closed	Bug 1982002: [4.8] rhcos-afterburn-checkin: wait for Ignition fetch stage	2021-07-15 22:39:41 UTC
Red Hat Product Errata	RHBA-2021:3682	0	None	None	None	2021-10-12 06:01:56 UTC

Comment 2 Benjamin Gilbert 2021-07-15 20:38:11 UTC

Needs bootimage bump; moving back to POST.

Comment 3 Benjamin Gilbert 2021-07-15 22:39:45 UTC

Landed in Git; waiting for bootimage bump.

Comment 4 RHCOS Bug Bot 2021-09-28 14:05:25 UTC

The fix for this bug has landed in a bootimage bump, as tracked in bug 1982001 (now in status MODIFIED).  Moving this bug to MODIFIED.

Comment 7 Michael Nguyen 2021-10-01 13:13:17 UTC

Verified on 4.8.0-0.nightly-2021-09-30-111446. rhcos-afterburn-checkin starts after ignition-fetch

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-09-30-111446   True        False         93s     Cluster version is 4.8.0-0.nightly-2021-09-30-111446
$ oc get nodes
NAME                                            STATUS   ROLES    AGE   VERSION
ci-ln-m1mn1ib-002ac-ql4w2-master-0              Ready    master   22m   v1.21.1+a620f50
ci-ln-m1mn1ib-002ac-ql4w2-master-1              Ready    master   22m   v1.21.1+a620f50
ci-ln-m1mn1ib-002ac-ql4w2-master-2              Ready    master   22m   v1.21.1+a620f50
ci-ln-m1mn1ib-002ac-ql4w2-worker-westus-9d4q6   Ready    worker   14m   v1.21.1+a620f50
ci-ln-m1mn1ib-002ac-ql4w2-worker-westus-cfdhj   Ready    worker   14m   v1.21.1+a620f50
ci-ln-m1mn1ib-002ac-ql4w2-worker-westus-dzcfz   Ready    worker   15m   v1.21.1+a620f50
$ oc debug node/ci-ln-m1mn1ib-002ac-ql4w2-worker-westus-9d4q6
Starting pod/ci-ln-m1mn1ib-002ac-ql4w2-worker-westus-9d4q6-debug ...
To use host binaries, run `chroot /host`
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# grep ^After /usr/lib/dracut/modules.d/30rhcos-afterburn-checkin/rhcos-afterburn-checkin.service
After=ignition-fetch.service
sh-4.4# 
sh-4.4# journalctl -u ignition-fetch  | grep -i start; journalctl | grep coreos-kargs-reboot; journalctl -u rhcos-afterburn-checkin | grep -i start
Oct 01 12:50:23 localhost systemd[1]: Starting Ignition (fetch)...
Oct 01 12:50:23 localhost ignition[769]: op(1): [started]  mounting "/dev/disk/by-id/ata-Virtual_CD" at "/tmp/ignition-azure479462508"
Oct 01 12:50:23 localhost ignition[769]: op(2): [started]  unmounting "/dev/disk/by-id/ata-Virtual_CD" at "/tmp/ignition-azure479462508"
Oct 01 12:51:30 localhost systemd[1]: Started Ignition (fetch).
Oct 01 12:51:30 localhost systemd[1]: Starting Afterburn (Check In - from the initramfs)...
Oct 01 12:51:59 localhost systemd[1]: Started Afterburn (Check In - from the initramfs).
sh-4.4# cat /etc/os-release 
NAME="Red Hat Enterprise Linux CoreOS"
VERSION="48.84.202109241901-0"
ID="rhcos"
ID_LIKE="rhel fedora"
VERSION_ID="4.8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux CoreOS 48.84.202109241901-0 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8::coreos"
HOME_URL="https://www.redhat.com/"
DOCUMENTATION_URL="https://docs.openshift.com/container-platform/4.8/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
REDHAT_BUGZILLA_PRODUCT_VERSION="4.8"
REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
REDHAT_SUPPORT_PRODUCT_VERSION="4.8"
OPENSHIFT_VERSION="4.8"
RHEL_VERSION="8.4"
OSTREE_VERSION='48.84.202109241901-0'
sh-4.4# rpm-ostree status
State: idle
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:57f6acfd48e833ef07a505b25a296849b74631a2b058814d618a347c9304308d
              CustomOrigin: Managed by machine-config-operator
                   Version: 48.84.202109241901-0 (2021-09-24T19:04:29Z)

  ostree://13c18da5e6fee09fade484c3903209730cbb73e9ebcab806b9e9000cf97fd719
                   Version: 48.84.202109241901-0 (2021-09-24T19:04:29Z)
sh-4.4#

Comment 9 errata-xmlrpc 2021-10-12 06:01:19 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.14 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3682

Note You need to log in before you can comment on or make changes to this bug.