Bug 1913736 - Regression: fails to deploy computes when using root volumes
Summary: Regression: fails to deploy computes when using root volumes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.7
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.7.0
Assignee: Mike Fedosin
QA Contact: weiwei jiang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-07 13:46 UTC by Martin André
Modified: 2021-02-24 15:51 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:50:58 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-api-provider-openstack pull 148 0 None closed Bug 1913736: do not fetch image id when booting from volume 2021-01-19 17:26:07 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:51:13 UTC

Description Martin André 2021-01-07 13:46:49 UTC
Deployment when using Boot from Volumes by setting `rootVolume` in your `install-config.yaml`.

moc-dev ❯ oc describe machine -n openshift-machine-api mandre-srkxp-worker-0-bsrrd | tail -1
  Warning  FailedCreate  137m  openstack_controller  InvalidConfiguration


CAPO logs show:
E0107 11:22:28.800950       1 actuator.go:574] Machine error mandre-srkxp-worker-0-b9qqd: Machine validation failed: Unable to find image with name 
W0107 11:22:28.800996       1 controller.go:317] mandre-srkxp-worker-0-b9qqd: failed to create machine: Machine validation failed: Unable to find image with name 
I0107 11:22:28.801037       1 controller.go:417] Actuator returned invalid configuration error: Machine validation failed: Unable to find image with name 
I0107 11:22:28.801054       1 controller.go:426] mandre-srkxp-worker-0-b9qqd: going into phase "Failed"


It's expected that we have an empty image attribute in the machinespec when using boot from volume:

moc-dev ❯ oc get machine -n openshift-machine-api mandre-srkxp-worker-0-bsrrd -o json | jq .spec
{
  "metadata": {},
  "providerSpec": {
    "value": {
      "apiVersion": "openstackproviderconfig.openshift.io/v1alpha1",
      "cloudName": "openstack",
      "cloudsSecret": {
        "name": "openstack-cloud-credentials",
        "namespace": "openshift-machine-api"
      },
      "flavor": "m1.s2.large",
      "image": "",
      "kind": "OpenstackProviderSpec",
      "metadata": {
        "creationTimestamp": null
      },
      "networks": [
        {
          "filter": {},
          "subnets": [
            {
              "filter": {
                "name": "mandre-srkxp-nodes",
                "tags": "openshiftClusterID=mandre-srkxp"
              }
            }
          ]
        }
      ],
      "rootVolume": {
        "deviceType": "",
        "diskSize": 25,
        "sourceType": "image",
        "sourceUUID": "rhcos-4.7",
        "volumeType": "performance"
      },
      "securityGroups": [
        {
          "filter": {},
          "name": "mandre-srkxp-worker"
        }
      ],
      "serverMetadata": {
        "Name": "mandre-srkxp-worker",
        "openshiftClusterID": "mandre-srkxp"
      },
      "tags": [
        "openshiftClusterID=mandre-srkxp"
      ],
      "trunk": true,
      "userDataSecret": {
        "name": "worker-user-data"
      }
    }
  }
}


The error is caused by https://github.com/openshift/cluster-api-provider-openstack/pull/121 that doesn't take that into account, and wasn't silently ignore until https://github.com/openshift/cluster-api-provider-openstack/pull/137.

Comment 2 weiwei jiang 2021-01-11 09:42:28 UTC
Checked with 4.7.0-0.nightly-2021-01-10-070949, and it's fixed.

$ ./openshift-install-4.7 version
./openshift-install-4.7 4.7.0-0.nightly-2021-01-10-070949
built from commit 69f0bbc18e8c6b1a6e278c54efa2def9b210033a
release image registry.ci.openshift.org/ocp/release@sha256:62768f2e8bb5850c469c1d15f6cd4337bd0f3f7ecb560b009c61e6d36a134fe8

$ cat install-config.yaml
---
apiVersion: v1
baseDomain: 10.0.103.245.nip.io
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform:
    openstack:
      type: m1.large
      rootVolume:
        type: tripleo
        size: 25
  replicas: 2
...

$ oc -n openshift-machine-api get machineset wj47ios111a-7xdqh-worker-0 -o json | jq .spec.template.spec
{
  "metadata": {},
  "providerSpec": {
    "value": {
      "apiVersion": "openstackproviderconfig.openshift.io/v1alpha1",
      "cloudName": "openstack",
      "cloudsSecret": {
        "name": "openstack-cloud-credentials",
        "namespace": "openshift-machine-api"
      },
      "flavor": "m1.large",
      "image": "",
      "kind": "OpenstackProviderSpec",
      "metadata": {
        "creationTimestamp": null
      },
      "networks": [
        {
          "filter": {},
          "subnets": [
            {
              "filter": {
                "name": "wj47ios111a-7xdqh-nodes",
                "tags": "openshiftClusterID=wj47ios111a-7xdqh"
              }
            }
          ]
        }
      ],
      "rootVolume": {
        "deviceType": "",
        "diskSize": 25,
        "sourceType": "image",
        "sourceUUID": "rhcos-47.83.202012030221-0",
        "volumeType": "tripleo"
      },
      "securityGroups": [
        {
          "filter": {},
          "name": "wj47ios111a-7xdqh-worker"
        }
      ],
      "serverMetadata": {
        "Name": "wj47ios111a-7xdqh-worker",
        "openshiftClusterID": "wj47ios111a-7xdqh"
      },
      "tags": [
        "openshiftClusterID=wj47ios111a-7xdqh"
      ],
      "trunk": true,
      "userDataSecret": {
        "name": "worker-user-data"
      }
    }
  }
}

$ oc get machineset -A
NAMESPACE               NAME                         DESIRED   CURRENT   READY   AVAILABLE   AGE
openshift-machine-api   wj47ios111a-7xdqh-worker-0   2         2         2       2           26m
$ oc get machine -A -o wide
NAMESPACE               NAME                               PHASE     TYPE        REGION      ZONE   AGE   NODE                               PROVIDERID   STATE
openshift-machine-api   wj47ios111a-7xdqh-master-0         Running   m1.xlarge   regionOne   nova   26m   wj47ios111a-7xdqh-master-0                      ACTIVE
openshift-machine-api   wj47ios111a-7xdqh-master-1         Running   m1.xlarge   regionOne   nova   26m   wj47ios111a-7xdqh-master-1                      ACTIVE
openshift-machine-api   wj47ios111a-7xdqh-master-2         Running   m1.xlarge   regionOne   nova   26m   wj47ios111a-7xdqh-master-2                      ACTIVE
openshift-machine-api   wj47ios111a-7xdqh-worker-0-mddh4   Running   m1.large    regionOne   nova   24m   wj47ios111a-7xdqh-worker-0-mddh4                ACTIVE
openshift-machine-api   wj47ios111a-7xdqh-worker-0-rmmkf   Running   m1.large    regionOne   nova   23m   wj47ios111a-7xdqh-worker-0-rmmkf                ACTIVE

# openstack server list --name wj4                                                                                                                                                                                                                  
+--------------------------------------+----------------------------------+--------+-------------------------------------------+----------------------------+-----------+
| ID                                   | Name                             | Status | Networks                                  | Image                      | Flavor    |
+--------------------------------------+----------------------------------+--------+-------------------------------------------+----------------------------+-----------+
| cc66c68c-82c4-4686-8ae6-4b3358ca842e | wj47ios111a-7xdqh-worker-0-rmmkf | ACTIVE | wj47ios111a-7xdqh-openshift=192.168.0.236 |                            | m1.large  |
| c59569a3-ce4d-4f18-8177-7e4670f39bce | wj47ios111a-7xdqh-worker-0-mddh4 | ACTIVE | wj47ios111a-7xdqh-openshift=192.168.3.213 |                            | m1.large  |
| cef21ccc-0c61-43ef-869c-33f38ae8bc63 | wj47ios111a-7xdqh-master-2       | ACTIVE | wj47ios111a-7xdqh-openshift=192.168.0.21  | rhcos-47.83.202012030221-0 | m1.xlarge |
| ed0c8910-abcb-4e42-bf6a-7d6eb659e2e8 | wj47ios111a-7xdqh-master-1       | ACTIVE | wj47ios111a-7xdqh-openshift=192.168.3.217 | rhcos-47.83.202012030221-0 | m1.xlarge |
| 10fe715e-47d7-4662-aaf0-6aa6f11ceff5 | wj47ios111a-7xdqh-master-0       | ACTIVE | wj47ios111a-7xdqh-openshift=192.168.3.66  | rhcos-47.83.202012030221-0 | m1.xlarge |
+--------------------------------------+----------------------------------+--------+-------------------------------------------+----------------------------+-----------+
# openstack server show wj47ios111a-7xdqh-worker-0-mddh4
+-----------------------------+-------------------------------------------------------------------------+
| Field                       | Value                                                                   |
+-----------------------------+-------------------------------------------------------------------------+
| OS-DCF:diskConfig           | MANUAL                                                                  |
| OS-EXT-AZ:availability_zone | nova                                                                    |
| OS-EXT-STS:power_state      | Running                                                                 |
| OS-EXT-STS:task_state       | None                                                                    |
| OS-EXT-STS:vm_state         | active                                                                  |
| OS-SRV-USG:launched_at      | 2021-01-11T09:18:09.000000                                              |
| OS-SRV-USG:terminated_at    | None                                                                    |
| accessIPv4                  |                                                                         |
| accessIPv6                  |                                                                         |
| addresses                   | wj47ios111a-7xdqh-openshift=192.168.3.213                               |
| config_drive                |                                                                         |
| created                     | 2021-01-11T09:17:57Z                                                    |
| flavor                      | m1.large (a9acc2de-39d7-4148-8d16-413c3b696e9d)                         |
| hostId                      | f2782b204ffcb5fd98b337f5fa4a7481010706a4904c0e37da37b002                |
| id                          | c59569a3-ce4d-4f18-8177-7e4670f39bce                                    |
| image                       |                                                                         |
| key_name                    | None                                                                    |
| name                        | wj47ios111a-7xdqh-worker-0-mddh4                                        |
| progress                    | 0                                                                       |
| project_id                  | 542c6ebd48bf40fa857fc245c7572e30                                        |
| properties                  | Name='wj47ios111a-7xdqh-worker', openshiftClusterID='wj47ios111a-7xdqh' |
| security_groups             | name='wj47ios111a-7xdqh-worker'                                         |
| status                      | ACTIVE                                                                  |
| updated                     | 2021-01-11T09:18:09Z                                                    |
| user_id                     | d57c957cb3c7010f14a049b72bbfd93d22c1cba255c3c2aa34be30ce5f83da7e        |
| volumes_attached            | id='bc35de6f-02b8-4ece-bf9e-300a56f3f9a2'                               |
+-----------------------------+-------------------------------------------------------------------------+
# openstack volume show bc35de6f-02b8-4ece-bf9e-300a56f3f9a2 -f yaml
attachments:
- attached_at: '2021-01-11T09:17:59.000000'
  attachment_id: 4769ed26-2520-428a-8f68-1ed156eafc09
  device: /dev/sda
  host_name: compute-d-023.localdomain
  id: bc35de6f-02b8-4ece-bf9e-300a56f3f9a2
  server_id: c59569a3-ce4d-4f18-8177-7e4670f39bce
  volume_id: bc35de6f-02b8-4ece-bf9e-300a56f3f9a2
availability_zone: nova
bootable: 'true'
consistencygroup_id: null
created_at: '2021-01-11T09:17:03.000000'
description: null
encrypted: false
id: bc35de6f-02b8-4ece-bf9e-300a56f3f9a2
multiattach: false
name: wj47ios111a-7xdqh-worker-0-mddh4
os-vol-tenant-attr:tenant_id: 542c6ebd48bf40fa857fc245c7572e30
properties: ''
replication_status: null
size: 25
snapshot_id: null
source_volid: null
status: in-use
type: tripleo
updated_at: '2021-01-11T09:18:00.000000'
user_id: d57c957cb3c7010f14a049b72bbfd93d22c1cba255c3c2aa34be30ce5f83da7e
volume_image_metadata:
  checksum: 3cea44acbeaf314c9399d109cf93f744
  container_format: bare
  disk_format: qcow2
  hw_disk_bus: scsi
  hw_qemu_guest_agent: 'yes'
  hw_scsi_model: virtio-scsi
  image_id: 6aaac14e-2758-4537-bd98-fa0e3c027bf2
  image_name: rhcos-47.83.202012030221-0
  min_disk: '0'
  min_ram: '0'
  size: '2328952832'

Comment 5 errata-xmlrpc 2021-02-24 15:50:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.