Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2054701

Summary:	[MAPO] Events are not created for MAPO machines
Product:	OpenShift Container Platform	Reporter:	Michał Dulko <mdulko>
Component:	Cloud Compute	Assignee:	Matthew Booth <mbooth>
Cloud Compute sub component:	OpenStack Provider	QA Contact:	Itay Matza <imatza>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	medium	CC:	aos-bugs, imatza, m.andre, mbooth, mfedosin, pprinett
Version:	4.10	Keywords:	Triaged
Target Milestone:	---
Target Release:	4.11.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-08-10 10:49:51 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Michał Dulko 2022-02-15 14:23:51 UTC

Description of problem:
Seems like Events doesn't work for OpenShift machines. This is most likely because we're attempting to create them for OpenStack machines, and I guess errors are silently ignored in case resources doesn't exist.

Version-Release number of selected component (if applicable):
Seen this in 4.11

How reproducible:
Always

Steps to Reproduce:
1. Install with MAPO 
2. Check for Events in Machines, e.g. SuccessfulCreateVolume is missing for machines with root volume.

Actual results:
Events from CAPO missing.

Expected results:
We have same events as upstream CAPO.

Additional info:

Comment 6 Itay Matza 2022-04-19 11:04:46 UTC

Verified with OCP 4.11.0-0.nightly-2022-04-16-163450 on top of RHOS-16.1-RHEL-8-20220329.n.1 with Kuryr.
(MAPO is the default for OpenStack deployments on this version)

Verification steps:
1) Install OCP 4.11 with MAPO and eCCM on top of OSP.

2) Check the openshift-machine-api events during the working with the environment:
```
[stack@undercloud-0 ~]$ oc get events -n openshift-machine-api
LAST SEEN   TYPE      REASON                              OBJECT                                              MESSAGE
133m        Normal    LeaderElection                      configmap/cluster-api-provider-healthcheck-leader   machine-api-controllers-5f8889656d-nvtfd_af92429a-e557-4c23-8603-baf5b642c90d became leader
133m        Normal    LeaderElection                      lease/cluster-api-provider-healthcheck-leader       machine-api-controllers-5f8889656d-nvtfd_af92429a-e557-4c23-8603-baf5b642c90d became leader
133m        Normal    LeaderElection                      configmap/cluster-api-provider-machineset-leader    machine-api-controllers-5f8889656d-nvtfd_e374a659-94d3-4acd-9ffe-0898a243168f became leader
133m        Normal    LeaderElection                      lease/cluster-api-provider-machineset-leader        machine-api-controllers-5f8889656d-nvtfd_e374a659-94d3-4acd-9ffe-0898a243168f became leader
133m        Normal    LeaderElection                      configmap/cluster-api-provider-nodelink-leader      machine-api-controllers-5f8889656d-nvtfd_650f7dea-7190-456d-8eba-df08384330c8 became leader
133m        Normal    LeaderElection                      lease/cluster-api-provider-nodelink-leader          machine-api-controllers-5f8889656d-nvtfd_650f7dea-7190-456d-8eba-df08384330c8 became leader
133m        Normal    LeaderElection                      configmap/cluster-api-provider-openstack-leader     machine-api-controllers-5f8889656d-nvtfd_fc5882bb-484c-407d-970b-0e945c2f2d8c became leader
133m        Normal    LeaderElection                      lease/cluster-api-provider-openstack-leader         machine-api-controllers-5f8889656d-nvtfd_fc5882bb-484c-407d-970b-0e945c2f2d8c became leader
144m        Warning   FailedScheduling                    pod/cluster-autoscaler-operator-7d6477b989-srr9r    no nodes available to schedule pods
142m        Warning   FailedScheduling                    pod/cluster-autoscaler-operator-7d6477b989-srr9r    no nodes available to schedule pods
141m        Warning   FailedScheduling                    pod/cluster-autoscaler-operator-7d6477b989-srr9r    0/3 nodes are available: 3 node(s) had taint {node.cloudprovider.kubernetes.io/uninitialized: true}, that the pod didn't tolerate.
139m        Warning   FailedScheduling                    pod/cluster-autoscaler-operator-7d6477b989-srr9r    0/3 nodes are available: 3 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
138m        Normal    Scheduled                           pod/cluster-autoscaler-operator-7d6477b989-srr9r    Successfully assigned openshift-machine-api/cluster-autoscaler-operator-7d6477b989-srr9r to ostest-hfkk5-master-2
136m        Warning   FailedMount                         pod/cluster-autoscaler-operator-7d6477b989-srr9r    MountVolume.SetUp failed for volume "cert" : secret "cluster-autoscaler-operator-cert" not found
136m        Warning   FailedMount                         pod/cluster-autoscaler-operator-7d6477b989-srr9r    Unable to attach or mount volumes: unmounted volumes=[cert], unattached volumes=[kube-api-access-hxhv2 auth-proxy-config cert]: timed out waiting for the condition
134m        Normal    AddedInterface                      pod/cluster-autoscaler-operator-7d6477b989-srr9r    Add eth0 [10.128.41.149/23] from kuryr
134m        Normal    Pulled                              pod/cluster-autoscaler-operator-7d6477b989-srr9r    Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d1e62c74ac62d2c718f2c7eeec8ea5c80f05ef35d70fe3a22b3f2e4c5a143a7" already present on machine
134m        Normal    Created                             pod/cluster-autoscaler-operator-7d6477b989-srr9r    Created container kube-rbac-proxy
134m        Normal    Started                             pod/cluster-autoscaler-operator-7d6477b989-srr9r    Started container kube-rbac-proxy
134m        Normal    Pulling                             pod/cluster-autoscaler-operator-7d6477b989-srr9r    Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aebc5b0fa8025a822a5bd868a970d038b67992ae51d601ead87e72a133214a2d"
134m        Normal    Pulled                              pod/cluster-autoscaler-operator-7d6477b989-srr9r    Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aebc5b0fa8025a822a5bd868a970d038b67992ae51d601ead87e72a133214a2d" in 16.412941901s
134m        Normal    Created                             pod/cluster-autoscaler-operator-7d6477b989-srr9r    Created container cluster-autoscaler-operator
134m        Normal    Started                             pod/cluster-autoscaler-operator-7d6477b989-srr9r    Started container cluster-autoscaler-operator
134m        Normal    CNIWaitingForVIFs                   pod/cluster-autoscaler-operator-7d6477b989-srr9r    Waiting for Neutron ports of openshift-machine-api/cluster-autoscaler-operator-7d6477b989-srr9r to become ACTIVE after binding.
144m        Normal    SuccessfulCreate                    replicaset/cluster-autoscaler-operator-7d6477b989   Created pod: cluster-autoscaler-operator-7d6477b989-srr9r
133m        Normal    LeaderElection                      configmap/cluster-autoscaler-operator-leader        cluster-autoscaler-operator-7d6477b989-srr9r_d494c806-9fae-4bd1-8096-372d152492b3 became leader
133m        Normal    LeaderElection                      lease/cluster-autoscaler-operator-leader            cluster-autoscaler-operator-7d6477b989-srr9r_d494c806-9fae-4bd1-8096-372d152492b3 became leader
144m        Normal    ScalingReplicaSet                   deployment/cluster-autoscaler-operator              Scaled up replica set cluster-autoscaler-operator-7d6477b989 to 1
138m        Normal    KuryrServiceSkipped                 service/cluster-autoscaler-operator                 Skipping Service openshift-machine-api/cluster-autoscaler-operator without Endpoints
136m        Normal    KuryrServiceSkipped                 service/cluster-autoscaler-operator                 Skipping Service openshift-machine-api/cluster-autoscaler-operator without Endpoints
134m        Normal    KuryrServiceSkipped                 service/cluster-autoscaler-operator                 Skipping Service openshift-machine-api/cluster-autoscaler-operator without Endpoints
134m        Normal    KuryrEnsureLB                       service/cluster-autoscaler-operator                 Provisioning a load balancer
132m        Normal    KuryrEnsuredLB                      service/cluster-autoscaler-operator                 Load balancer provisioned
144m        Warning   FailedScheduling                    pod/cluster-baremetal-operator-64868dcbf-n4sjf      no nodes available to schedule pods
142m        Warning   FailedScheduling                    pod/cluster-baremetal-operator-64868dcbf-n4sjf      no nodes available to schedule pods
141m        Warning   FailedScheduling                    pod/cluster-baremetal-operator-64868dcbf-n4sjf      0/3 nodes are available: 3 node(s) had taint {node.cloudprovider.kubernetes.io/uninitialized: true}, that the pod didn't tolerate.
140m        Warning   FailedScheduling                    pod/cluster-baremetal-operator-64868dcbf-n4sjf      0/3 nodes are available: 3 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
139m        Warning   FailedScheduling                    pod/cluster-baremetal-operator-64868dcbf-n4sjf      0/3 nodes are available: 3 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
138m        Normal    Scheduled                           pod/cluster-baremetal-operator-64868dcbf-n4sjf      Successfully assigned openshift-machine-api/cluster-baremetal-operator-64868dcbf-n4sjf to ostest-hfkk5-master-2
138m        Warning   FailedMount                         pod/cluster-baremetal-operator-64868dcbf-n4sjf      MountVolume.SetUp failed for volume "images" : failed to sync configmap cache: timed out waiting for the condition
138m        Warning   FailedMount                         pod/cluster-baremetal-operator-64868dcbf-n4sjf      MountVolume.SetUp failed for volume "cluster-baremetal-operator-tls" : failed to sync secret cache: timed out waiting for the condition
138m        Warning   FailedMount                         pod/cluster-baremetal-operator-64868dcbf-n4sjf      MountVolume.SetUp failed for volume "cert" : failed to sync secret cache: timed out waiting for the condition
136m        Warning   FailedMount                         pod/cluster-baremetal-operator-64868dcbf-n4sjf      MountVolume.SetUp failed for volume "cert" : secret "cluster-baremetal-webhook-server-cert" not found
136m        Warning   FailedMount                         pod/cluster-baremetal-operator-64868dcbf-n4sjf      MountVolume.SetUp failed for volume "cluster-baremetal-operator-tls" : secret "cluster-baremetal-operator-tls" not found
136m        Warning   FailedMount                         pod/cluster-baremetal-operator-64868dcbf-n4sjf      Unable to attach or mount volumes: unmounted volumes=[cluster-baremetal-operator-tls cert], unattached volumes=[cluster-baremetal-operator-tls cert images kube-api-access-rgj9z config]: timed out waiting for the condition
134m        Normal    AddedInterface                      pod/cluster-baremetal-operator-64868dcbf-n4sjf      Add eth0 [10.128.40.182/23] from kuryr
134m        Normal    Pulling                             pod/cluster-baremetal-operator-64868dcbf-n4sjf      Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6ad54bbca704345671f6705f2e816e76958b896349c1ed9deab19602b87830a9"
134m        Normal    Pulled                              pod/cluster-baremetal-operator-64868dcbf-n4sjf      Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6ad54bbca704345671f6705f2e816e76958b896349c1ed9deab19602b87830a9" in 16.726522564s
134m        Normal    Created                             pod/cluster-baremetal-operator-64868dcbf-n4sjf      Created container cluster-baremetal-operator
134m        Normal    Started                             pod/cluster-baremetal-operator-64868dcbf-n4sjf      Started container cluster-baremetal-operator
134m        Normal    Pulled                              pod/cluster-baremetal-operator-64868dcbf-n4sjf      Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d1e62c74ac62d2c718f2c7eeec8ea5c80f05ef35d70fe3a22b3f2e4c5a143a7" already present on machine
134m        Normal    CNIWaitingForVIFs                   pod/cluster-baremetal-operator-64868dcbf-n4sjf      Waiting for Neutron ports of openshift-machine-api/cluster-baremetal-operator-64868dcbf-n4sjf to become ACTIVE after binding.
144m        Normal    SuccessfulCreate                    replicaset/cluster-baremetal-operator-64868dcbf     Created pod: cluster-baremetal-operator-64868dcbf-n4sjf
138m        Normal    KuryrServiceSkipped                 service/cluster-baremetal-operator-service          Skipping Service openshift-machine-api/cluster-baremetal-operator-service without Endpoints
136m        Normal    KuryrServiceSkipped                 service/cluster-baremetal-operator-service          Skipping Service openshift-machine-api/cluster-baremetal-operator-service without Endpoints
134m        Normal    KuryrServiceSkipped                 service/cluster-baremetal-operator-service          Skipping Service openshift-machine-api/cluster-baremetal-operator-service without Endpoints
134m        Normal    KuryrEnsureLB                       service/cluster-baremetal-operator-service          Provisioning a load balancer
133m        Normal    KuryrEnsuredLB                      service/cluster-baremetal-operator-service          Load balancer provisioned
144m        Normal    ScalingReplicaSet                   deployment/cluster-baremetal-operator               Scaled up replica set cluster-baremetal-operator-64868dcbf to 1
138m        Normal    KuryrServiceSkipped                 service/cluster-baremetal-webhook-service           Skipping Service openshift-machine-api/cluster-baremetal-webhook-service without Endpoints
136m        Normal    KuryrServiceSkipped                 service/cluster-baremetal-webhook-service           Skipping Service openshift-machine-api/cluster-baremetal-webhook-service without Endpoints
134m        Normal    KuryrServiceSkipped                 service/cluster-baremetal-webhook-service           Skipping Service openshift-machine-api/cluster-baremetal-webhook-service without Endpoints
134m        Normal    KuryrEnsureLB                       service/cluster-baremetal-webhook-service           Provisioning a load balancer
133m        Normal    KuryrEnsuredLB                      service/cluster-baremetal-webhook-service           Load balancer provisioned
133m        Normal    KuryrEnsuredLB                      service/cluster-baremetal-webhook-service           Load balancer provisioned
134m        Normal    Scheduled                           pod/machine-api-controllers-5f8889656d-nvtfd        Successfully assigned openshift-machine-api/machine-api-controllers-5f8889656d-nvtfd to ostest-hfkk5-master-1
133m        Normal    AddedInterface                      pod/machine-api-controllers-5f8889656d-nvtfd        Add eth0 [10.128.40.103/23] from kuryr
133m        Normal    Pulling                             pod/machine-api-controllers-5f8889656d-nvtfd        Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f51b6e11a40527609d51e2cf96d744a5d1310e00c3e5fb12d997af194a13f6c"
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f51b6e11a40527609d51e2cf96d744a5d1310e00c3e5fb12d997af194a13f6c" in 9.975831074s
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container machineset-controller
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container machineset-controller
133m        Normal    Pulling                             pod/machine-api-controllers-5f8889656d-nvtfd        Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f3b10478ba516fc2ce64718ebd8890357c5f7a88fad962768cda1d4529749e03"
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f3b10478ba516fc2ce64718ebd8890357c5f7a88fad962768cda1d4529749e03" in 22.199515356s
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container machine-controller
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container machine-controller
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f51b6e11a40527609d51e2cf96d744a5d1310e00c3e5fb12d997af194a13f6c" already present on machine
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container nodelink-controller
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container nodelink-controller
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f51b6e11a40527609d51e2cf96d744a5d1310e00c3e5fb12d997af194a13f6c" already present on machine
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container machine-healthcheck-controller
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container machine-healthcheck-controller
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d1e62c74ac62d2c718f2c7eeec8ea5c80f05ef35d70fe3a22b3f2e4c5a143a7" already present on machine
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container kube-rbac-proxy-machineset-mtrc
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container kube-rbac-proxy-machineset-mtrc
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d1e62c74ac62d2c718f2c7eeec8ea5c80f05ef35d70fe3a22b3f2e4c5a143a7" already present on machine
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container kube-rbac-proxy-machine-mtrc
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container kube-rbac-proxy-machine-mtrc
133m        Normal    Pulled                              pod/machine-api-controllers-5f8889656d-nvtfd        Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d1e62c74ac62d2c718f2c7eeec8ea5c80f05ef35d70fe3a22b3f2e4c5a143a7" already present on machine
133m        Normal    Created                             pod/machine-api-controllers-5f8889656d-nvtfd        Created container kube-rbac-proxy-mhc-mtrc
133m        Normal    Started                             pod/machine-api-controllers-5f8889656d-nvtfd        Started container kube-rbac-proxy-mhc-mtrc
133m        Warning   ProbeError                          pod/machine-api-controllers-5f8889656d-nvtfd        Liveness probe error: Get "http://10.128.40.103:9442/readyz": dial tcp 10.128.40.103:9442: connect: connection refused...
133m        Warning   Unhealthy                           pod/machine-api-controllers-5f8889656d-nvtfd        Liveness probe failed: Get "http://10.128.40.103:9442/readyz": dial tcp 10.128.40.103:9442: connect: connection refused
133m        Normal    CNIWaitingForVIFs                   pod/machine-api-controllers-5f8889656d-nvtfd        Waiting for Neutron ports of openshift-machine-api/machine-api-controllers-5f8889656d-nvtfd to become ACTIVE after binding.
134m        Normal    SuccessfulCreate                    replicaset/machine-api-controllers-5f8889656d       Created pod: machine-api-controllers-5f8889656d-nvtfd
134m        Normal    ScalingReplicaSet                   deployment/machine-api-controllers                  Scaled up replica set machine-api-controllers-5f8889656d to 1
138m        Normal    KuryrServiceSkipped                 service/machine-api-controllers                     Skipping Service openshift-machine-api/machine-api-controllers without Endpoints
136m        Normal    KuryrServiceSkipped                 service/machine-api-controllers                     Skipping Service openshift-machine-api/machine-api-controllers without Endpoints
134m        Normal    KuryrServiceSkipped                 service/machine-api-controllers                     Skipping Service openshift-machine-api/machine-api-controllers without Endpoints
133m        Normal    KuryrEnsureLB                       service/machine-api-controllers                     Provisioning a load balancer
132m        Normal    KuryrEnsureLB                       service/machine-api-controllers                     Provisioning a load balancer
132m        Normal    KuryrEnsuredLB                      service/machine-api-controllers                     Load balancer provisioned
144m        Warning   FailedScheduling                    pod/machine-api-operator-7b5cc4cdb6-c2kvx           no nodes available to schedule pods
142m        Warning   FailedScheduling                    pod/machine-api-operator-7b5cc4cdb6-c2kvx           no nodes available to schedule pods
141m        Warning   FailedScheduling                    pod/machine-api-operator-7b5cc4cdb6-c2kvx           0/3 nodes are available: 3 node(s) had taint {node.cloudprovider.kubernetes.io/uninitialized: true}, that the pod didn't tolerate.
139m        Warning   FailedScheduling                    pod/machine-api-operator-7b5cc4cdb6-c2kvx           0/3 nodes are available: 3 node(s) had taint {node.kubernetes.io/not-ready: }, that the pod didn't tolerate.
138m        Normal    Scheduled                           pod/machine-api-operator-7b5cc4cdb6-c2kvx           Successfully assigned openshift-machine-api/machine-api-operator-7b5cc4cdb6-c2kvx to ostest-hfkk5-master-2
136m        Warning   FailedMount                         pod/machine-api-operator-7b5cc4cdb6-c2kvx           MountVolume.SetUp failed for volume "machine-api-operator-tls" : secret "machine-api-operator-tls" not found
136m        Warning   FailedMount                         pod/machine-api-operator-7b5cc4cdb6-c2kvx           Unable to attach or mount volumes: unmounted volumes=[machine-api-operator-tls], unattached volumes=[config machine-api-operator-tls kube-api-access-nh4p8 images]: timed out waiting for the condition
134m        Normal    AddedInterface                      pod/machine-api-operator-7b5cc4cdb6-c2kvx           Add eth0 [10.128.40.169/23] from kuryr
134m        Normal    Pulled                              pod/machine-api-operator-7b5cc4cdb6-c2kvx           Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7d1e62c74ac62d2c718f2c7eeec8ea5c80f05ef35d70fe3a22b3f2e4c5a143a7" already present on machine
134m        Normal    Created                             pod/machine-api-operator-7b5cc4cdb6-c2kvx           Created container kube-rbac-proxy
134m        Normal    Started                             pod/machine-api-operator-7b5cc4cdb6-c2kvx           Started container kube-rbac-proxy
134m        Normal    Pulling                             pod/machine-api-operator-7b5cc4cdb6-c2kvx           Pulling image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f51b6e11a40527609d51e2cf96d744a5d1310e00c3e5fb12d997af194a13f6c"
134m        Normal    Pulled                              pod/machine-api-operator-7b5cc4cdb6-c2kvx           Successfully pulled image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9f51b6e11a40527609d51e2cf96d744a5d1310e00c3e5fb12d997af194a13f6c" in 16.522546815s
134m        Normal    Created                             pod/machine-api-operator-7b5cc4cdb6-c2kvx           Created container machine-api-operator
134m        Normal    Started                             pod/machine-api-operator-7b5cc4cdb6-c2kvx           Started container machine-api-operator
134m        Normal    CNIWaitingForVIFs                   pod/machine-api-operator-7b5cc4cdb6-c2kvx           Waiting for Neutron ports of openshift-machine-api/machine-api-operator-7b5cc4cdb6-c2kvx to become ACTIVE after binding.
144m        Normal    SuccessfulCreate                    replicaset/machine-api-operator-7b5cc4cdb6          Created pod: machine-api-operator-7b5cc4cdb6-c2kvx
138m        Normal    KuryrServiceSkipped                 service/machine-api-operator-webhook                Skipping Service openshift-machine-api/machine-api-operator-webhook without Endpoints
136m        Normal    KuryrServiceSkipped                 service/machine-api-operator-webhook                Skipping Service openshift-machine-api/machine-api-operator-webhook without Endpoints
134m        Normal    KuryrServiceSkipped                 service/machine-api-operator-webhook                Skipping Service openshift-machine-api/machine-api-operator-webhook without Endpoints
133m        Normal    KuryrEnsureLB                       service/machine-api-operator-webhook                Provisioning a load balancer
132m        Normal    KuryrEnsureLB                       service/machine-api-operator-webhook                Provisioning a load balancer
132m        Normal    KuryrEnsuredLB                      service/machine-api-operator-webhook                Load balancer provisioned
144m        Normal    ScalingReplicaSet                   deployment/machine-api-operator                     Scaled up replica set machine-api-operator-7b5cc4cdb6 to 1
138m        Normal    KuryrServiceSkipped                 service/machine-api-operator                        Skipping Service openshift-machine-api/machine-api-operator without Endpoints
136m        Normal    KuryrServiceSkipped                 service/machine-api-operator                        Skipping Service openshift-machine-api/machine-api-operator without Endpoints
134m        Normal    KuryrServiceSkipped                 service/machine-api-operator                        Skipping Service openshift-machine-api/machine-api-operator without Endpoints
134m        Normal    KuryrEnsureLB                       service/machine-api-operator                        Provisioning a load balancer
133m        Normal    KuryrEnsuredLB                      service/machine-api-operator                        Load balancer provisioned
138m        Normal    CreateNetworkSucceed                kuryrnetwork/openshift-machine-api                  Neutron network 556441b9-6741-4fc1-8db0-632d1a852fcd for namespace
138m        Normal    CreateSubnetSucceed                 kuryrnetwork/openshift-machine-api                  Neutron subnet 7f6c93e5-aecf-4498-910a-2e6b970f57e6 for network 556441b9-6741-4fc1-8db0-632d1a852fcd
138m        Normal    AddingSubnetToRouterSucceed         kuryrnetwork/openshift-machine-api                  Neutron subnet 7f6c93e5-aecf-4498-910a-2e6b970f57e6 added to router 8195e838-674a-4a96-9c40-a2c9938f2287
138m        Normal    SGUpdateTriggered                   kuryrnetwork/openshift-machine-api                  Neutron security groups update has been triggered
134m        Normal    SGUpdateTriggered                   kuryrnetwork/openshift-machine-api                  Neutron security groups update has been triggered
123m        Normal    Reconciled                          machine/ostest-hfkk5-master-0                       Reconciled machine ostest-hfkk5-master-0
123m        Normal    Reconciled                          machine/ostest-hfkk5-master-1                       Reconciled machine ostest-hfkk5-master-1
123m        Normal    Reconciled                          machine/ostest-hfkk5-master-2                       Reconciled machine ostest-hfkk5-master-2
131m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-l8m7c                 Replaced all attributestags for b368d559-0be6-4235-a65d-eb5434fb08a0 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-l8m7c                 Created port ostest-hfkk5-worker-0-l8m7c-0 with id b368d559-0be6-4235-a65d-eb5434fb08a0
131m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-l8m7c                 Created trunk ostest-hfkk5-worker-0-l8m7c-0 with id 8517429c-9038-450d-af51-2e50cddc3ba6
131m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-l8m7c                 Replaced all attributestags for 8517429c-9038-450d-af51-2e50cddc3ba6 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-l8m7c                 Replaced all attributestags for 0837a9a1-6320-49d9-b287-b0003d0dde65 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-l8m7c                 Created port ostest-hfkk5-worker-0-l8m7c-1 with id 0837a9a1-6320-49d9-b287-b0003d0dde65
131m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-l8m7c                 Created trunk ostest-hfkk5-worker-0-l8m7c-1 with id b756e6ae-21bd-4ea9-8a82-ff77aec44236
131m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-l8m7c                 Replaced all attributestags for b756e6ae-21bd-4ea9-8a82-ff77aec44236 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-l8m7c                 Replaced all attributestags for 11248a61-2be2-4503-a26b-5e59281f13a2 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-l8m7c                 Created port ostest-hfkk5-worker-0-l8m7c-2 with id 11248a61-2be2-4503-a26b-5e59281f13a2
131m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-l8m7c                 Created trunk ostest-hfkk5-worker-0-l8m7c-2 with id c9a12895-b01a-4296-afd7-cff820588bf5
131m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-l8m7c                 Replaced all attributestags for c9a12895-b01a-4296-afd7-cff820588bf5 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulcreateserver              machine/ostest-hfkk5-worker-0-l8m7c                 Created server ostest-hfkk5-worker-0-l8m7c with id 4560fa0c-8ba0-4c98-8c39-4df603b36236
131m        Normal    Created                             machine/ostest-hfkk5-worker-0-l8m7c                 Created OpenStack instance 4560fa0c-8ba0-4c98-8c39-4df603b36236
126m        Normal    Reconciled                          machine/ostest-hfkk5-worker-0-l8m7c                 Reconciled machine ostest-hfkk5-worker-0-l8m7c
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-qpr5l                 Replaced all attributestags for e5ac8aae-e177-4b12-b621-fe4e94f63a48 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-qpr5l                 Created port ostest-hfkk5-worker-0-qpr5l-0 with id e5ac8aae-e177-4b12-b621-fe4e94f63a48
132m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-qpr5l                 Created trunk ostest-hfkk5-worker-0-qpr5l-0 with id 69f44146-581c-4e59-97b6-0e6ef58b57f5
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-qpr5l                 Replaced all attributestags for 69f44146-581c-4e59-97b6-0e6ef58b57f5 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-qpr5l                 Replaced all attributestags for 5ad16528-77c9-4a25-b837-9485962d378c with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-qpr5l                 Created port ostest-hfkk5-worker-0-qpr5l-1 with id 5ad16528-77c9-4a25-b837-9485962d378c
132m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-qpr5l                 Created trunk ostest-hfkk5-worker-0-qpr5l-1 with id 08023ad6-6684-495e-8268-6c4d5a3e34c8
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-qpr5l                 Replaced all attributestags for 08023ad6-6684-495e-8268-6c4d5a3e34c8 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-qpr5l                 Replaced all attributestags for aeaca7e8-9ff6-46de-869f-691730413b3a with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-qpr5l                 Created port ostest-hfkk5-worker-0-qpr5l-2 with id aeaca7e8-9ff6-46de-869f-691730413b3a
132m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-qpr5l                 Created trunk ostest-hfkk5-worker-0-qpr5l-2 with id 63bfc85c-3a9c-46c9-97a7-82bd361e6276
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-qpr5l                 Replaced all attributestags for 63bfc85c-3a9c-46c9-97a7-82bd361e6276 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
131m        Normal    Successfulcreateserver              machine/ostest-hfkk5-worker-0-qpr5l                 Created server ostest-hfkk5-worker-0-qpr5l with id 0009d9ad-0243-4bfd-bdf6-1071dd900910
131m        Normal    Created                             machine/ostest-hfkk5-worker-0-qpr5l                 Created OpenStack instance 0009d9ad-0243-4bfd-bdf6-1071dd900910
126m        Normal    Reconciled                          machine/ostest-hfkk5-worker-0-qpr5l                 Reconciled machine ostest-hfkk5-worker-0-qpr5l
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-wwzv6                 Replaced all attributestags for f03b6ee8-727d-4c5e-a17a-2e80961e41c5 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-wwzv6                 Created port ostest-hfkk5-worker-0-wwzv6-0 with id f03b6ee8-727d-4c5e-a17a-2e80961e41c5
132m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-wwzv6                 Created trunk ostest-hfkk5-worker-0-wwzv6-0 with id 122e2f15-f9cc-46d7-b744-4cbc7a4f36b1
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-wwzv6                 Replaced all attributestags for 122e2f15-f9cc-46d7-b744-4cbc7a4f36b1 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-wwzv6                 Replaced all attributestags for 3ac223c0-2094-4e5f-b25e-a4b3e60caa13 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-wwzv6                 Created port ostest-hfkk5-worker-0-wwzv6-1 with id 3ac223c0-2094-4e5f-b25e-a4b3e60caa13
132m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-wwzv6                 Created trunk ostest-hfkk5-worker-0-wwzv6-1 with id a19652dd-bada-43f0-acf9-ee4b0995efc7
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-wwzv6                 Replaced all attributestags for a19652dd-bada-43f0-acf9-ee4b0995efc7 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-wwzv6                 Replaced all attributestags for 02f605dc-fbcd-41e8-a97b-113cf3d74acf with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateport                machine/ostest-hfkk5-worker-0-wwzv6                 Created port ostest-hfkk5-worker-0-wwzv6-2 with id 02f605dc-fbcd-41e8-a97b-113cf3d74acf
132m        Normal    Successfulcreatetrunk               machine/ostest-hfkk5-worker-0-wwzv6                 Created trunk ostest-hfkk5-worker-0-wwzv6-2 with id 871b10e4-1a68-4038-8240-c8adb4d76df9
132m        Normal    Successfulreplaceallattributetags   machine/ostest-hfkk5-worker-0-wwzv6                 Replaced all attributestags for 871b10e4-1a68-4038-8240-c8adb4d76df9 with tags [cluster-api-provider-openstack openshift-machine-api-ostest-hfkk5 openshiftClusterID=ostest-hfkk5]
132m        Normal    Successfulcreateserver              machine/ostest-hfkk5-worker-0-wwzv6                 Created server ostest-hfkk5-worker-0-wwzv6 with id f9e241c1-7a84-46e5-94f0-856096dc09ea
132m        Normal    Created                             machine/ostest-hfkk5-worker-0-wwzv6                 Created OpenStack instance f9e241c1-7a84-46e5-94f0-856096dc09ea
126m        Normal    Reconciled                          machine/ostest-hfkk5-worker-0-wwzv6                 Reconciled machine ostest-hfkk5-worker-0-wwzv6
133m        Warning   ReconcileError                      machineset/ostest-hfkk5-worker-0                    failed to update machine set status: Operation cannot be fulfilled on machinesets.machine.openshift.io "ostest-hfkk5-worker-0": the object has been modified; please apply your changes to the latest version and try again
``` 

3) Events are created for MAPO machines, for example:
```
132m        Normal    Successfulcreateserver              machine/ostest-hfkk5-worker-0-wwzv6                 Created server ostest-hfkk5-worker-0-wwzv6 with id f9e241c1-7a84-46e5-94f0-856096dc09ea
```
Which came from upstream CAPO: https://github.com/kubernetes-sigs/cluster-api-provider-openstack/blob/af6f62379d2f3248605771cf4fdb2c9399797665/pkg/cloud/services/compute/instance.go#L269

Comment 8 errata-xmlrpc 2022-08-10 10:49:51 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069