Description of problem: Create a machine with invalid subnet, logs should suggest subnet is invalid Version-Release number of selected component (if applicable): 4.2.0-0.nightly-2019-08-22-153337 How reproducible: always Steps to Reproduce: 1. Create a new machine with invlid subnet apiVersion: machine.openshift.io/v1beta1 kind: Machine metadata: labels: machine.openshift.io/cluster-api-cluster: shared-jiazha-8ck2c machine.openshift.io/cluster-api-machine-role: worker machine.openshift.io/cluster-api-machine-type: worker machine.openshift.io/instance-type: m1.large machine.openshift.io/region: regionOne machine.openshift.io/zone: nova name: shared-jiazha-8ck2c-worker-invalidsubnet namespace: openshift-machine-api spec: metadata: creationTimestamp: null providerSpec: value: apiVersion: openstackproviderconfig.openshift.io/v1alpha1 cloudName: openstack cloudsSecret: name: openstack-cloud-credentials namespace: openshift-machine-api flavor: m1.large image: rhcos-42.80.20190815.3 kind: OpenstackProviderSpec metadata: creationTimestamp: null networks: - filter: {} subnets: - filter: name: shared-jiazha-8ck2c-nodes tags: openshiftClusterID=shared-jiazha-8ck2c-invalid securityGroups: - filter: {} name: shared-jiazha-8ck2c-worker serverMetadata: Name: shared-jiazha-8ck2c-worker openshiftClusterID: shared-jiazha-8ck2c tags: - openshiftClusterID=shared-jiazha-8ck2c trunk: true userDataSecret: name: worker-user-data 2. Check machine-controller logs Actual results: I0827 06:54:30.506560 1 controller.go:129] Reconciling Machine "shared-jiazha-8ck2c-worker-invalidsubnet" I0827 06:54:30.507042 1 controller.go:298] Machine "shared-jiazha-8ck2c-worker-invalidsubnet" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster I0827 06:54:30.624759 1 controller.go:129] Reconciling Machine "shared-jiazha-8ck2c-worker-invalidsubnet" I0827 06:54:30.624795 1 controller.go:298] Machine "shared-jiazha-8ck2c-worker-invalidsubnet" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster I0827 06:54:31.166576 1 controller.go:247] Reconciling machine object shared-jiazha-8ck2c-worker-invalidsubnet triggers idempotent create. E0827 06:54:39.784986 1 actuator.go:414] Machine error shared-jiazha-8ck2c-worker-invalidsubnet: error creating Openstack instance: Create new server err: Bad request with: [POST https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1/servers], error message: {"badRequest": {"message": "Invalid input for field/attribute server. Value: {u'name': u'shared-jiazha-8ck2c-worker-invalidsubnet', u'imageRef': u'8fe77b79-2a62-4b90-9f95-67c7bcc88070', u'tags': [u'cluster-api-provider-openstack', u'openshift-machine-api-shared-jiazha-8ck2c', u'openshiftClusterID=shared-jiazha-8ck2c'], u'flavorRef': u'a9acc2de-39d7-4148-8d16-413c3b696e9d', u'user_data': u'eyJpZ25pdGlvbiI6eyJjb25maWciOnsiYXBwZW5kIjpbeyJzb3VyY2UiOiJodHRwczovLzE5Mi4xNjguMC41OjIyNjIzL2NvbmZpZy93b3JrZXIiLCJ2ZXJpZmljYXRpb24iOnt9fV19LCJzZWN1cml0eSI6eyJ0bHMiOnsiY2VydGlmaWNhdGVBdXRob3JpdGllcyI6W3sic291cmNlIjoiZGF0YTp0ZXh0L3BsYWluO2NoYXJzZXQ9dXRmLTg7YmFzZTY0LExTMHRMUzFDUlVkSlRpQkRSVkpVU1VaSlEwRlVSUzB0TFMwdENrMUpTVVJGUkVORFFXWnBaMEYzU1VKQlowbEpZbnBsVms5UVJpOXRhMDEzUkZGWlNrdHZXa2xvZG1OT1FWRkZURUpSUVhkS2FrVlRUVUpCUjBFeFZVVUtRM2hOU21JelFteGliazV2WVZkYU1FMVNRWGRFWjFsRVZsRlJSRVYzWkhsaU1qa3dURmRPYUUxQ05GaEVWRVUxVFVSbmVVNXFSWGROYWxWNlRWWnZXQXBFVkVrMVRVUm5lVTE2UlhkTmFsVjZUVlp2ZDBwcVJWTk5Ra0ZIUVRGVlJVTjRUVXBpTTBKc1ltNU9iMkZYV2pCTlVrRjNSR2RaUkZaUlVVUkZkMlI1Q21JeU9UQk1WMDVvVFVsSlFrbHFRVTVDWjJ0eGFHdHBSemwzTUVKQlVVVkdRVUZQUTBGUk9FRk5TVWxDUTJkTFEwRlJSVUYyUVRaa0szTjJOelY0ZW1zS1QxQmFNazhySzBKTFdWazBVMmxNY0V0TlNXTmlWM0V3UzA4NFlsZHZUUzl5VlROSVRETktla0V6VG5WUll6QkpjRU4yVW1sb05raHdaMGxrY2tGS1N3cEhhRTQ0UkhnelYwNVdXVWhoYTA1bFdUQkRLME40VXpGRmVFbFJiMDl3VlZoaVZGTkpSMU16TkZWVE9UWjRhVFEyYjB0eFNIbFNRbVpNYUU0eVdsWjRDblZMVXk5RWNHTTFXRUl5VlZSWVQwTTJiM0ZFWkZGRGQwUnJVRzlGTUV0TlNXVlhabWw0T0hsT1VpdFdjblJPVDNkRlRHRndVMkp0T0c1S1NuTktOMHdLYXpneVUweDFUVmwyVVZrelowUkNibXh6VG10dVQyWkRVR3hsYWpWU1lrbDBLM0JCVjFjelVuVlpOa1F6Ym05dFRtdEhOa3NyYUdsMFVsaGpaRVpzYWdwR1MwbHZVRGg1TlhoeVNXVmFkMjFRU1VJMFVESlJkSGhTZVZFNWJUTnVlRVJXYURZdllUVjZOMFJDUWs5NWMwTktSRVZUUWlzN VUyTnRSamRZYkVKWENuSm1XR3RpU0djeFRYZEpSRUZSUVVKdk1FbDNVVVJCVDBKblRsWklVVGhDUVdZNFJVSkJUVU5CY1ZGM1JIZFpSRlpTTUZSQlVVZ3ZRa0ZWZDBGM1JVSUtMM3BCWkVKblRsWklVVFJGUm1kUlZUSXdSa1ZTT0dsSmFYRlVWVTlVZFVob1l5OXdhV1l3VDNWSk5IZEVVVmxLUzI5YVNXaDJZMDVCVVVWTVFsRkJSQXBuWjBWQ1FVVm9XVkEyTlcxb2MxbGtaalZqVUhKdWR6TTRjRUozY0ZOeWN6SkJMMVp3U0hwMlozRlRiVEI1ZGl0YWN6ZHdOR0ZvYmtkV2FFSlJObVk1Q210TU1FazFhVzh3TlVSTlRqTlBSVkZ3ZFdVMVREZG9iRVJwU2xkSFdVWlZVbXBJTlhCbFFYVlNiRTA0ZERKU1RVbDRWbkp4ZEdWb0sxRlNNa0o0Y1cwS1IwNUNTekIyUXpkRk4waGlTbTR4TTA0MlZ6QXlVVzl1WnlzMmMweFhUMVI0V1ZWSVUzRTJZbEF6ZFVFNWFHeDZNV3QzZVVNNGRIbGpWMnhEVUZGb01RcFJVMFpHWVVWRVJtdG1TMDgwVTB0WFlVUnhORkJVV0ZkUWFTOXNNVmt4Y1RJek9GUTRXRGx4TVZsSFl6aFRkR3BZTUV0WFVHWXlWVXhYTlRVMFExZHhDaXRDTlZCNkwwaENZWEJoYzFkNVNFa3pNWEJUWWs4MVJEZGtja2s1Y21WWFNrcEdTRVJ4ZDNkYVIyaElZbVZGZVdFNVYxTlJXa2hOVG1OYVVtUnRNM0FLTlM5eFZuTlJZMHB5Tld4VGJFdEhaM0ZPVERoWVNuazBibVZuUFFvdExTMHRMVVZPUkNCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2c9PSIsInZlcmlmaWNhdGlvbiI6e319XX19LCJ0aW1lb3V0cyI6e30sInZlcnNpb24iOiIyLjIuMCJ9LCJuZXR3b3JrZCI6e30sInBhc3N3ZCI6e30sInN0b3JhZ2UiOnt9LCJzeXN0ZW1kIjp7fX0=', u'security_groups': [{u'name': u'842bf4b1-0f72-4e8d-b9ad-13cc838d3475'}], u'metadata': {u'openshiftClusterID': u'shared-jiazha-8ck2c', u'Name': u'shared-jiazha-8ck2c-worker'}}. 'networks' is a required property", "code": 400}} W0827 06:54:39.785044 1 controller.go:249] Failed to create machine "shared-jiazha-8ck2c-worker-invalidsubnet": error creating Openstack instance: Create new server err: Bad request with: [POST https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1/servers], error message: {"badRequest": {"message": "Invalid input for field/attribute server. Value: {u'name': u'shared-jiazha-8ck2c-worker-invalidsubnet', u'imageRef': u'8fe77b79-2a62-4b90-9f95-67c7bcc88070', u'tags': [u'cluster-api-provider-openstack', u'openshift-machine-api-shared-jiazha-8ck2c', u'openshiftClusterID=shared-jiazha-8ck2c'], u'flavorRef': u'a9acc2de-39d7-4148-8d16-413c3b696e9d', u'user_data': u'eyJpZ25pdGlvbiI6eyJjb25maWciOnsiYXBwZW5kIjpbeyJzb3VyY2UiOiJodHRwczovLzE5Mi4xNjguMC41OjIyNjIzL2NvbmZpZy93b3JrZXIiLCJ2ZXJpZmljYXRpb24iOnt9fV19LCJzZWN1cml0eSI6eyJ0bHMiOnsiY2VydGlmaWNhdGVBdXRob3JpdGllcyI6W3sic291cmNlIjoiZGF0YTp0ZXh0L3BsYWluO2NoYXJzZXQ9dXRmLTg7YmFzZTY0LExTMHRMUzFDUlVkSlRpQkRSVkpVU1VaSlEwRlVSUzB0TFMwdENrMUpTVVJGUkVORFFXWnBaMEYzU1VKQlowbEpZbnBsVms5UVJpOXRhMDEzUkZGWlNrdHZXa2xvZG1OT1FWRkZURUpSUVhkS2FrVlRUVUpCUjBFeFZVVUtRM2hOU21JelFteGliazV2WVZkYU1FMVNRWGRFWjFsRVZsRlJSRVYzWkhsaU1qa3dURmRPYUUxQ05GaEVWRVUxVFVSbmVVNXFSWGROYWxWNlRWWnZXQXBFVkVrMVRVUm5lVTE2UlhkTmFsVjZUVlp2ZDBwcVJWTk5Ra0ZIUVRGVlJVTjRUVXBpTTBKc1ltNU9iMkZYV2pCTlVrRjNSR2RaUkZaUlVVUkZkMlI1Q21JeU9UQk1WMDVvVFVsSlFrbHFRVTVDWjJ0eGFHdHBSemwzTUVKQlVVVkdRVUZQUTBGUk9FRk5TVWxDUTJkTFEwRlJSVUYyUVRaa0szTjJOelY0ZW1zS1QxQmFNazhySzBKTFdWazBVMmxNY0V0TlNXTmlWM0V3UzA4NFlsZHZUUzl5VlROSVRETktla0V6VG5WUll6QkpjRU4yVW1sb05raHdaMGxrY2tGS1N3cEhhRTQ0UkhnelYwNVdXVWhoYTA1bFdUQkRLME40VXpGRmVFbFJiMDl3VlZoaVZGTkpSMU16TkZWVE9UWjRhVFEyYjB0eFNIbFNRbVpNYUU0eVdsWjRDblZMVXk5RWNHTTFXRUl5VlZSWVQwTTJiM0ZFWkZGRGQwUnJVRzlGTUV0TlNXVlhabWw0T0hsT1VpdFdjblJPVDNkRlRHRndVMkp0T0c1S1NuTktOMHdLYXpneVUweDFUVmwyVVZrelowUkNibXh6VG10dVQyWkRVR3hsYWpWU1lrbDBLM0JCVjFjelVuVlpOa1F6Ym05dFRtdEhOa3NyYUdsMFVsaGpaRVpzYWdwR1MwbHZVRGg1TlhoeVNXVmFkMjFRU1VJMFVESlJkSGhTZVZFNWJUTnVlRVJXYURZdllUVjZOMFJDUWs5NW MwTktSRVZUUWlzNVUyTnRSamRZYkVKWENuSm1XR3RpU0djeFRYZEpSRUZSUVVKdk1FbDNVVVJCVDBKblRsWklVVGhDUVdZNFJVSkJUVU5CY1ZGM1JIZFpSRlpTTUZSQlVVZ3ZRa0ZWZDBGM1JVSUtMM3BCWkVKblRsWklVVFJGUm1kUlZUSXdSa1ZTT0dsSmFYRlVWVTlVZFVob1l5OXdhV1l3VDNWSk5IZEVVVmxLUzI5YVNXaDJZMDVCVVVWTVFsRkJSQXBuWjBWQ1FVVm9XVkEyTlcxb2MxbGtaalZqVUhKdWR6TTRjRUozY0ZOeWN6SkJMMVp3U0hwMlozRlRiVEI1ZGl0YWN6ZHdOR0ZvYmtkV2FFSlJObVk1Q210TU1FazFhVzh3TlVSTlRqTlBSVkZ3ZFdVMVREZG9iRVJwU2xkSFdVWlZVbXBJTlhCbFFYVlNiRTA0ZERKU1RVbDRWbkp4ZEdWb0sxRlNNa0o0Y1cwS1IwNUNTekIyUXpkRk4waGlTbTR4TTA0MlZ6QXlVVzl1WnlzMmMweFhUMVI0V1ZWSVUzRTJZbEF6ZFVFNWFHeDZNV3QzZVVNNGRIbGpWMnhEVUZGb01RcFJVMFpHWVVWRVJtdG1TMDgwVTB0WFlVUnhORkJVV0ZkUWFTOXNNVmt4Y1RJek9GUTRXRGx4TVZsSFl6aFRkR3BZTUV0WFVHWXlWVXhYTlRVMFExZHhDaXRDTlZCNkwwaENZWEJoYzFkNVNFa3pNWEJUWWs4MVJEZGtja2s1Y21WWFNrcEdTRVJ4ZDNkYVIyaElZbVZGZVdFNVYxTlJXa2hOVG1OYVVtUnRNM0FLTlM5eFZuTlJZMHB5Tld4VGJFdEhaM0ZPVERoWVNuazBibVZuUFFvdExTMHRMVVZPUkNCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2c9PSIsInZlcmlmaWNhdGlvbiI6e319XX19LCJ0aW1lb3V0cyI6e30sInZlcnNpb24iOiIyLjIuMCJ9LCJuZXR3b3JrZCI6e30sInBhc3N3ZCI6e30sInN0b3JhZ2UiOnt9LCJzeXN0ZW1kIjp7fX0=', u'security_groups': [{u'name': u'842bf4b1-0f72-4e8d-b9ad-13cc838d3475'}], u'metadata': {u'openshiftClusterID': u'shared-jiazha-8ck2c', u'Name': u'shared-jiazha-8ck2c-worker'}}. 'networks' is a required property", "code": 400}} Expected results: Logs should suggest subnet is invalid. Additional info:
The log output is really ugly, but doesn't it say that the subnet was invalid right here: controller.go:249] Failed to create machine "shared-jiazha-8ck2c-worker-invalidsubnet": error creating Openstack instance: Create new server err: Bad request with: [POST https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1/servers], error message: {"badRequest": {"message": "Invalid input for field/attribute server. Value: {u'name': u'shared-jiazha-8ck2c-worker-invalidsubnet', u'imageRef': u'8fe77b79-2a62-4b90-9f95-67c7bcc88070', u'tags': [u'cluster-api-provider-openstack', u'openshift-machine-api-shared-jiazha-8ck2c', u'openshiftClusterID=shared-jiazha-8ck2c'], u'flavorRef': u'a9acc2de-39d7-4148-8d16-413c3b696e9d', u'user_data': ....
The error message from OpenStack mentions that the 'networks' property is missing from the Instance definition. And indeed, CAPO's machine-service compiles the Instance definition by inserting any real NetworkID found after querying OpenStack with the Machine's Network property filters. If no valid NetworkID is found, the Instance's Network section is left empty.
Verifed. clusterversion: 4.2.0-0.nightly-2019-09-15-052022 I0916 06:11:04.704983 1 controller.go:129] Reconciling Machine "share-0916c-8vp8z-worker-aaa" I0916 06:11:04.705022 1 controller.go:298] Machine "share-0916c-8vp8z-worker-aaa" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster I0916 06:11:04.711614 1 controller.go:129] Reconciling Machine "share-0916c-8vp8z-worker-aaa" I0916 06:11:04.711640 1 controller.go:298] Machine "share-0916c-8vp8z-worker-aaa" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster I0916 06:11:05.005775 1 controller.go:247] Reconciling machine object share-0916c-8vp8z-worker-aaa triggers idempotent create. E0916 06:11:11.340611 1 actuator.go:470] Machine error share-0916c-8vp8z-worker-aaa: error creating Openstack instance: No network was found or provided. Please check your machine configuration and try again W0916 06:11:11.340640 1 controller.go:249] Failed to create machine "share-0916c-8vp8z-worker-aaa": error creating Openstack instance: No network was found or provided. Please check your machine configuration and try again
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922