Bug 1888467

setting severity and priority to high as this is an OKD 4.6 release blocker

Verified on 4.6.0-0.nightly-2020-11-28-204928 -- machine-config-daemon-firstboot.service has the fix in the unit file.



$ oc debug node/ip-10-0-128-247.us-west-2.compute.internal
Starting pod/ip-10-0-128-247us-west-2computeinternal-debug ...
To use host binaries, run `chroot /host`
If you don't see a command prompt, try pressing enter.
sh-4.2# chroot /host
sh-4.4# systemctl cat machine-config-daemon-firstboot.service
# /etc/systemd/system/machine-config-daemon-firstboot.service
[Unit]
Description=Machine Config Daemon Firstboot
# Make sure it runs only on OSTree booted system
ConditionPathExists=/run/ostree-booted
# Removal of this file signals firstboot completion
ConditionPathExists=/etc/ignition-machine-config-encapsulated.json
After=machine-config-daemon-pull.service
Before=crio.service crio-wipe.service
Before=kubelet.service

[Service]
Type=oneshot
RemainAfterExit=yes
# Disable existing repos (if any) so that OS extensions would use embedded RPMs only
ExecStartPre=-/usr/bin/sh -c "sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/*.repo"
ExecStart=/run/bin/machine-config-daemon firstboot-complete-machineconfig

[Install]
WantedBy=multi-user.target
RequiredBy=crio.service kubelet.service
sh-4.4# journalctl -u machine-config-daemon-firstboot
-- Logs begin at Mon 2020-11-30 22:15:10 UTC, end at Mon 2020-11-30 22:28:25 UTC. --
Nov 30 22:16:16 ip-10-0-128-247 systemd[1]: Starting Machine Config Daemon Firstboot...
Nov 30 22:16:16 ip-10-0-128-247 sh[1941]: sed: can't read /etc/yum.repos.d/*.repo: No such file or directory
Nov 30 22:16:17 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:17.017716    1943 rpm-ostree.go:261] Running captured: rpm-ostree status --json
Nov 30 22:16:22 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:22.288033    1943 daemon.go:226] Booted osImageURL:  (46.82.202010011740-0)
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.018090    1943 daemon.go:233] Installed Ignition binary version: 2.6.0
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.019571    1943 update.go:423] Checking Reconcilable for config mco-empty-mc to rendered-worker-e2825f140e84a200a54950101efa3790
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.020175    1943 update.go:1653] Starting update from mco-empty-mc to rendered-worker-e2825f140e84a200a54950101efa3790: &{osUpdate:true kargs:false fips:false >
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.023861    1943 update.go:1040] Updating files
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.024188    1943 update.go:1113] Deleting stale data
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.034266    1943 controlplane.go:52] Set root blockdev /sys/devices/pci0000:00/0000:00:04.0/nvme/nvme0/nvme0n1 to use scheduler bfq
Nov 30 22:16:28 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:16:28.034491    1943 run.go:18] Running: nice -- ionice -c 3 oc image extract --path /:/run/mco-machine-os-content/os-content-024229197 --registry-config /var/lib/>
Nov 30 22:17:40 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:17:40.424536    1943 update.go:1531] Updating OS to quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ccf308bdfbc7e0d076a94be9fc0b1671703c7be56a9de3fbbb46e880d>
Nov 30 22:17:40 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:17:40.424719    1943 rpm-ostree.go:261] Running captured: rpm-ostree status --json
Nov 30 22:17:40 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:17:40.487729    1943 rpm-ostree.go:184] Current origin is not custom
Nov 30 22:17:42 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:17:42.104810    1943 rpm-ostree.go:211] Pivoting to: 46.82.202011281819-0 (8398b4ee2024e71d9de24e95b5a97da3bc7e4a010a779f551f640d5f46d20739)
Nov 30 22:17:42 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:17:42.104833    1943 rpm-ostree.go:243] Executing rebase from repo path /run/mco-machine-os-content/os-content-024229197/srv/repo with customImageURL pivot://quay.>
Nov 30 22:17:42 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:17:42.104846    1943 rpm-ostree.go:261] Running captured: rpm-ostree rebase --experimental /run/mco-machine-os-content/os-content-024229197/srv/repo:8398b4ee2024e7>
Nov 30 22:18:03 ip-10-0-128-247 machine-config-daemon[1943]: I1130 22:18:03.219052    1943 update.go:1653] initiating reboot: Completing firstboot provisioning to rendered-worker-e2825f140e84a200a54950101efa3790
Nov 30 22:18:03 ip-10-0-128-247 systemd[1]: machine-config-daemon-firstboot.service: Main process exited, code=killed, status=15/TERM
Nov 30 22:18:03 ip-10-0-128-247 systemd[1]: machine-config-daemon-firstboot.service: Failed with result 'signal'.
Nov 30 22:18:03 ip-10-0-128-247 systemd[1]: Stopped Machine Config Daemon Firstboot.
Nov 30 22:18:03 ip-10-0-128-247 systemd[1]: machine-config-daemon-firstboot.service: Consumed 17.711s CPU time
sh-4.4# exit
exit
sh-4.2# exit
exit

Removing debug pod ...

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.6.8 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5259