Bug 1948705 - [assisted operator] SNO deployment fails - ClusterDeployment shows `bootstrap.ign was not found`
Summary: [assisted operator] SNO deployment fails - ClusterDeployment shows `bootstrap...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: assisted-installer
Version: 4.8
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: 4.8.0
Assignee: David Zager
QA Contact: Chad Crum
URL:
Whiteboard: AI-Team-Platform
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-12 18:55 UTC by Chad Crum
Modified: 2021-07-27 22:59 UTC (History)
8 users (show)

Fixed In Version: OCP-Metal-v1.0.19.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 22:59:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift assisted-service pull 1505 0 None open OCPBUGSM-27640: localjob when storage=filesystem 2021-04-14 18:55:07 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:59:57 UTC

Description Chad Crum 2021-04-12 18:55:10 UTC
Description of problem:
[assisted operator] SNO deployment fails - ClusterDeployment shows `bootstrap.ign was not found`

Version-Release number of selected component (if applicable):
ipv4/ connected / libvirt env

Assisted Operator catalog source: 
oc get catalogsource assisted-installer-index -o yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: assisted-installer-index
  namespace: openshift-marketplace
spec:
  displayName: assisted-installer-index
  image: quay.io/ocpmetal/assisted-service-index:913aba63aae8923400e8ff4e040c8cfc0d59897e

Hub cluster: oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-04-09-222447   True        False         3h35m   Cluster version is 4.8.0-0.nightly-2021-04-09-222447

Openshift-versions: "4.8":{"display_name":"4.8","release_image":"registry.ci.openshift.org/ocp/release:4.8.0-0.nightly-2021-04-09-140229","rhcos_image":"https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.7/4.7.0/rhcos-4.7.0-x86_64-live.x86_64.iso","rhcos_version":"47.83.202102090044-0","support_level":"production"}}


How reproducible:
100% (so far 2 times)

Steps to Reproduce:
1. Deploy 4.8 ocp hub cluster (libvirt/ipv4/connected)
2. Deploy hive community operator
3. Deploy assisted operator using index image quay.io/ocpmetal/assisted-service-index:913aba63aae8923400e8ff4e040c8cfc0d59897e
4. Create clusterdeployment / installenv objects for an SNO libvirt deployment 5. Create discovery iso / have SNO libvirt start and register with assisted service operand
6. Approve agent cr which should trigger installation of openshift on SNO vm
7. Run oc describe clusterdeployment name-of-cluster-deployment

Actual results:
Status:
  Cli Image:  quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:711a024fb5e8dce628946e486f5d870452a3dfd04676f672e32034dbe55a1920
  Conditions:
    Last Probe Time:       2021-04-12T18:36:52Z
    Last Transition Time:  2021-04-12T18:36:52Z
    Message:               error
    Reason:                AgentPlatformState
    Status:                Unknown
    Type:                  Unreachable
    Last Probe Time:       2021-04-12T18:36:52Z
    Last Transition Time:  2021-04-12T18:36:52Z
    Message:               object e14fc97e-92b2-4287-a4c6-c8d6c4571df3/bootstrap.ign was not found


Expected results:
Install should complete

Additional info:
Attached bz_info.txt has oc get commands for every object I could think of

Comment 2 Chad Crum 2021-04-12 19:24:01 UTC
This is the assisted service image cache dir content:

[root@sealusa12 pv1]# tree
.
├── cache
│   └── public-rhcos-47.83.202102090044-0-minimal.iso
├── discovery-image-e14fc97e-92b2-4287-a4c6-c8d6c4571df3.iso
├── e14fc97e-92b2-4287-a4c6-c8d6c4571df3
│   └── manifests
│       └── openshift
├── installercache
│   └── registry.ci.openshift.org
│       └── ocp
│           └── release:4.8.0-0.nightly-2021-04-09-140229
│               └── openshift-baremetal-install
├── isoutil153855964
│   ├── EFI
│   │   └── redhat
│   ├── images
│   │   └── pxeboot
│   └── isolinux
├── isoutil218845290
│   ├── EFI
│   │   └── redhat
│   ├── images
│   │   └── pxeboot
│   └── isolinux
├── minimal_templates_version.json
├── rhcos-46.82.202012051820-0.initrd.img
├── rhcos-46.82.202012051820-0.iso
├── rhcos-46.82.202012051820-0-minimal.iso
├── rhcos-46.82.202012051820-0.rootfs.img
├── rhcos-46.82.202012051820-0.vmlinuz
├── rhcos-47.83.202102090044-0.initrd.img
├── rhcos-47.83.202102090044-0.iso
├── rhcos-47.83.202102090044-0-minimal.iso
├── rhcos-47.83.202102090044-0.rootfs.img
└── rhcos-47.83.202102090044-0.vmlinuz

20 directories, 14 files

Comment 5 Chad Crum 2021-04-27 19:01:16 UTC
Verified this is fixed in quay.io/ocpmetal/assisted-service:961906c4c288ea696ed069c101186c95da3126fd

bootstrap.ign gets generated (I have multiple examples)

find /mnt/ | grep bootstrap.ign
/mnt/pv0/f44b738c-498a-4a84-aae1-bd96ee5032a6/bootstrap.ign
/mnt/pv0/364fcad7-e55f-4ab1-a7b6-ae82ef0e474a/bootstrap.ign
/mnt/pv0/c5f0e8a4-fed2-4793-b6dc-3328e21017a9/bootstrap.ign
/mnt/pv0/9686de9d-944e-4939-acb0-b9f28e5497ee/bootstrap.ign
/mnt/pv0/3d0a8969-2e26-42ab-ac78-ca00d2c4a920/bootstrap.ign
/mnt/pv0/bc5e0e5d-9951-4a23-a871-d82937d3bcdf/bootstrap.ign
/mnt/pv0/9e1fa62c-2eee-442d-b6ff-274e522b7cae/bootstrap.ign

And I am able to get an SNO cluster deployment past this error now.

Comment 8 errata-xmlrpc 2021-07-27 22:59:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.