Bug 1746021

Summary: [osp] Create a machine with invalid subnet, logs should output subnet is invalid
Product: OpenShift Container Platform Reporter: sunzhaohua <zhsun>
Component: Cloud ComputeAssignee: Pierre Prinetti <pprinett>
Status: CLOSED ERRATA QA Contact: sunzhaohua <zhsun>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.2.0CC: agarcial, calfonso, eduen, egarcia, jchaloup
Target Milestone: ---   
Target Release: 4.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: osp
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-16 06:37:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description sunzhaohua 2019-08-27 13:32:30 UTC
Description of problem:
Create a machine with invalid subnet, logs should suggest subnet is invalid 

Version-Release number of selected component (if applicable):
4.2.0-0.nightly-2019-08-22-153337

How reproducible:
always

Steps to Reproduce:
1. Create a new machine with invlid subnet

apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
  labels:
    machine.openshift.io/cluster-api-cluster: shared-jiazha-8ck2c
    machine.openshift.io/cluster-api-machine-role: worker
    machine.openshift.io/cluster-api-machine-type: worker
    machine.openshift.io/instance-type: m1.large
    machine.openshift.io/region: regionOne
    machine.openshift.io/zone: nova
  name: shared-jiazha-8ck2c-worker-invalidsubnet
  namespace: openshift-machine-api
spec:
  metadata:
    creationTimestamp: null
  providerSpec:
    value:
      apiVersion: openstackproviderconfig.openshift.io/v1alpha1
      cloudName: openstack
      cloudsSecret:
        name: openstack-cloud-credentials
        namespace: openshift-machine-api
      flavor: m1.large
      image: rhcos-42.80.20190815.3
      kind: OpenstackProviderSpec
      metadata:
        creationTimestamp: null
      networks:
      - filter: {}
        subnets:
        - filter:
            name: shared-jiazha-8ck2c-nodes
            tags: openshiftClusterID=shared-jiazha-8ck2c-invalid
      securityGroups:
      - filter: {}
        name: shared-jiazha-8ck2c-worker
      serverMetadata:
        Name: shared-jiazha-8ck2c-worker
        openshiftClusterID: shared-jiazha-8ck2c
      tags:
      - openshiftClusterID=shared-jiazha-8ck2c
      trunk: true
      userDataSecret:
        name: worker-user-data
        
2. Check machine-controller logs


Actual results:
I0827 06:54:30.506560       1 controller.go:129] Reconciling Machine "shared-jiazha-8ck2c-worker-invalidsubnet"
I0827 06:54:30.507042       1 controller.go:298] Machine "shared-jiazha-8ck2c-worker-invalidsubnet" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster
I0827 06:54:30.624759       1 controller.go:129] Reconciling Machine "shared-jiazha-8ck2c-worker-invalidsubnet"
I0827 06:54:30.624795       1 controller.go:298] Machine "shared-jiazha-8ck2c-worker-invalidsubnet" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster
I0827 06:54:31.166576       1 controller.go:247] Reconciling machine object shared-jiazha-8ck2c-worker-invalidsubnet triggers idempotent create.
E0827 06:54:39.784986       1 actuator.go:414] Machine error shared-jiazha-8ck2c-worker-invalidsubnet: error creating Openstack instance: Create new server err: Bad request with: [POST https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1/servers], error message: {"badRequest": {"message": "Invalid input for field/attribute server. Value: {u'name': u'shared-jiazha-8ck2c-worker-invalidsubnet', u'imageRef': u'8fe77b79-2a62-4b90-9f95-67c7bcc88070', u'tags': [u'cluster-api-provider-openstack', u'openshift-machine-api-shared-jiazha-8ck2c', u'openshiftClusterID=shared-jiazha-8ck2c'], u'flavorRef': u'a9acc2de-39d7-4148-8d16-413c3b696e9d', u'user_data': u'eyJpZ25pdGlvbiI6eyJjb25maWciOnsiYXBwZW5kIjpbeyJzb3VyY2UiOiJodHRwczovLzE5Mi4xNjguMC41OjIyNjIzL2NvbmZpZy93b3JrZXIiLCJ2ZXJpZmljYXRpb24iOnt9fV19LCJzZWN1cml0eSI6eyJ0bHMiOnsiY2VydGlmaWNhdGVBdXRob3JpdGllcyI6W3sic291cmNlIjoiZGF0YTp0ZXh0L3BsYWluO2NoYXJzZXQ9dXRmLTg7YmFzZTY0LExTMHRMUzFDUlVkSlRpQkRSVkpVU1VaSlEwRlVSUzB0TFMwdENrMUpTVVJGUkVORFFXWnBaMEYzU1VKQlowbEpZbnBsVms5UVJpOXRhMDEzUkZGWlNrdHZXa2xvZG1OT1FWRkZURUpSUVhkS2FrVlRUVUpCUjBFeFZVVUtRM2hOU21JelFteGliazV2WVZkYU1FMVNRWGRFWjFsRVZsRlJSRVYzWkhsaU1qa3dURmRPYUUxQ05GaEVWRVUxVFVSbmVVNXFSWGROYWxWNlRWWnZXQXBFVkVrMVRVUm5lVTE2UlhkTmFsVjZUVlp2ZDBwcVJWTk5Ra0ZIUVRGVlJVTjRUVXBpTTBKc1ltNU9iMkZYV2pCTlVrRjNSR2RaUkZaUlVVUkZkMlI1Q21JeU9UQk1WMDVvVFVsSlFrbHFRVTVDWjJ0eGFHdHBSemwzTUVKQlVVVkdRVUZQUTBGUk9FRk5TVWxDUTJkTFEwRlJSVUYyUVRaa0szTjJOelY0ZW1zS1QxQmFNazhySzBKTFdWazBVMmxNY0V0TlNXTmlWM0V3UzA4NFlsZHZUUzl5VlROSVRETktla0V6VG5WUll6QkpjRU4yVW1sb05raHdaMGxrY2tGS1N3cEhhRTQ0UkhnelYwNVdXVWhoYTA1bFdUQkRLME40VXpGRmVFbFJiMDl3VlZoaVZGTkpSMU16TkZWVE9UWjRhVFEyYjB0eFNIbFNRbVpNYUU0eVdsWjRDblZMVXk5RWNHTTFXRUl5VlZSWVQwTTJiM0ZFWkZGRGQwUnJVRzlGTUV0TlNXVlhabWw0T0hsT1VpdFdjblJPVDNkRlRHRndVMkp0T0c1S1NuTktOMHdLYXpneVUweDFUVmwyVVZrelowUkNibXh6VG10dVQyWkRVR3hsYWpWU1lrbDBLM0JCVjFjelVuVlpOa1F6Ym05dFRtdEhOa3NyYUdsMFVsaGpaRVpzYWdwR1MwbHZVRGg1TlhoeVNXVmFkMjFRU1VJMFVESlJkSGhTZVZFNWJUTnVlRVJXYURZdllUVjZOMFJDUWs5NWMwTktSRVZUUWlzN
VUyTnRSamRZYkVKWENuSm1XR3RpU0djeFRYZEpSRUZSUVVKdk1FbDNVVVJCVDBKblRsWklVVGhDUVdZNFJVSkJUVU5CY1ZGM1JIZFpSRlpTTUZSQlVVZ3ZRa0ZWZDBGM1JVSUtMM3BCWkVKblRsWklVVFJGUm1kUlZUSXdSa1ZTT0dsSmFYRlVWVTlVZFVob1l5OXdhV1l3VDNWSk5IZEVVVmxLUzI5YVNXaDJZMDVCVVVWTVFsRkJSQXBuWjBWQ1FVVm9XVkEyTlcxb2MxbGtaalZqVUhKdWR6TTRjRUozY0ZOeWN6SkJMMVp3U0hwMlozRlRiVEI1ZGl0YWN6ZHdOR0ZvYmtkV2FFSlJObVk1Q210TU1FazFhVzh3TlVSTlRqTlBSVkZ3ZFdVMVREZG9iRVJwU2xkSFdVWlZVbXBJTlhCbFFYVlNiRTA0ZERKU1RVbDRWbkp4ZEdWb0sxRlNNa0o0Y1cwS1IwNUNTekIyUXpkRk4waGlTbTR4TTA0MlZ6QXlVVzl1WnlzMmMweFhUMVI0V1ZWSVUzRTJZbEF6ZFVFNWFHeDZNV3QzZVVNNGRIbGpWMnhEVUZGb01RcFJVMFpHWVVWRVJtdG1TMDgwVTB0WFlVUnhORkJVV0ZkUWFTOXNNVmt4Y1RJek9GUTRXRGx4TVZsSFl6aFRkR3BZTUV0WFVHWXlWVXhYTlRVMFExZHhDaXRDTlZCNkwwaENZWEJoYzFkNVNFa3pNWEJUWWs4MVJEZGtja2s1Y21WWFNrcEdTRVJ4ZDNkYVIyaElZbVZGZVdFNVYxTlJXa2hOVG1OYVVtUnRNM0FLTlM5eFZuTlJZMHB5Tld4VGJFdEhaM0ZPVERoWVNuazBibVZuUFFvdExTMHRMVVZPUkNCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2c9PSIsInZlcmlmaWNhdGlvbiI6e319XX19LCJ0aW1lb3V0cyI6e30sInZlcnNpb24iOiIyLjIuMCJ9LCJuZXR3b3JrZCI6e30sInBhc3N3ZCI6e30sInN0b3JhZ2UiOnt9LCJzeXN0ZW1kIjp7fX0=', u'security_groups': [{u'name': u'842bf4b1-0f72-4e8d-b9ad-13cc838d3475'}], u'metadata': {u'openshiftClusterID': u'shared-jiazha-8ck2c', u'Name': u'shared-jiazha-8ck2c-worker'}}. 'networks' is a required property", "code": 400}}
W0827 06:54:39.785044       1 controller.go:249] Failed to create machine "shared-jiazha-8ck2c-worker-invalidsubnet": error creating Openstack instance: Create new server err: Bad request with: [POST https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1/servers], error message: {"badRequest": {"message": "Invalid input for field/attribute server. Value: {u'name': u'shared-jiazha-8ck2c-worker-invalidsubnet', u'imageRef': u'8fe77b79-2a62-4b90-9f95-67c7bcc88070', u'tags': [u'cluster-api-provider-openstack', u'openshift-machine-api-shared-jiazha-8ck2c', u'openshiftClusterID=shared-jiazha-8ck2c'], u'flavorRef': u'a9acc2de-39d7-4148-8d16-413c3b696e9d', u'user_data': u'eyJpZ25pdGlvbiI6eyJjb25maWciOnsiYXBwZW5kIjpbeyJzb3VyY2UiOiJodHRwczovLzE5Mi4xNjguMC41OjIyNjIzL2NvbmZpZy93b3JrZXIiLCJ2ZXJpZmljYXRpb24iOnt9fV19LCJzZWN1cml0eSI6eyJ0bHMiOnsiY2VydGlmaWNhdGVBdXRob3JpdGllcyI6W3sic291cmNlIjoiZGF0YTp0ZXh0L3BsYWluO2NoYXJzZXQ9dXRmLTg7YmFzZTY0LExTMHRMUzFDUlVkSlRpQkRSVkpVU1VaSlEwRlVSUzB0TFMwdENrMUpTVVJGUkVORFFXWnBaMEYzU1VKQlowbEpZbnBsVms5UVJpOXRhMDEzUkZGWlNrdHZXa2xvZG1OT1FWRkZURUpSUVhkS2FrVlRUVUpCUjBFeFZVVUtRM2hOU21JelFteGliazV2WVZkYU1FMVNRWGRFWjFsRVZsRlJSRVYzWkhsaU1qa3dURmRPYUUxQ05GaEVWRVUxVFVSbmVVNXFSWGROYWxWNlRWWnZXQXBFVkVrMVRVUm5lVTE2UlhkTmFsVjZUVlp2ZDBwcVJWTk5Ra0ZIUVRGVlJVTjRUVXBpTTBKc1ltNU9iMkZYV2pCTlVrRjNSR2RaUkZaUlVVUkZkMlI1Q21JeU9UQk1WMDVvVFVsSlFrbHFRVTVDWjJ0eGFHdHBSemwzTUVKQlVVVkdRVUZQUTBGUk9FRk5TVWxDUTJkTFEwRlJSVUYyUVRaa0szTjJOelY0ZW1zS1QxQmFNazhySzBKTFdWazBVMmxNY0V0TlNXTmlWM0V3UzA4NFlsZHZUUzl5VlROSVRETktla0V6VG5WUll6QkpjRU4yVW1sb05raHdaMGxrY2tGS1N3cEhhRTQ0UkhnelYwNVdXVWhoYTA1bFdUQkRLME40VXpGRmVFbFJiMDl3VlZoaVZGTkpSMU16TkZWVE9UWjRhVFEyYjB0eFNIbFNRbVpNYUU0eVdsWjRDblZMVXk5RWNHTTFXRUl5VlZSWVQwTTJiM0ZFWkZGRGQwUnJVRzlGTUV0TlNXVlhabWw0T0hsT1VpdFdjblJPVDNkRlRHRndVMkp0T0c1S1NuTktOMHdLYXpneVUweDFUVmwyVVZrelowUkNibXh6VG10dVQyWkRVR3hsYWpWU1lrbDBLM0JCVjFjelVuVlpOa1F6Ym05dFRtdEhOa3NyYUdsMFVsaGpaRVpzYWdwR1MwbHZVRGg1TlhoeVNXVmFkMjFRU1VJMFVESlJkSGhTZVZFNWJUTnVlRVJXYURZdllUVjZOMFJDUWs5NW
MwTktSRVZUUWlzNVUyTnRSamRZYkVKWENuSm1XR3RpU0djeFRYZEpSRUZSUVVKdk1FbDNVVVJCVDBKblRsWklVVGhDUVdZNFJVSkJUVU5CY1ZGM1JIZFpSRlpTTUZSQlVVZ3ZRa0ZWZDBGM1JVSUtMM3BCWkVKblRsWklVVFJGUm1kUlZUSXdSa1ZTT0dsSmFYRlVWVTlVZFVob1l5OXdhV1l3VDNWSk5IZEVVVmxLUzI5YVNXaDJZMDVCVVVWTVFsRkJSQXBuWjBWQ1FVVm9XVkEyTlcxb2MxbGtaalZqVUhKdWR6TTRjRUozY0ZOeWN6SkJMMVp3U0hwMlozRlRiVEI1ZGl0YWN6ZHdOR0ZvYmtkV2FFSlJObVk1Q210TU1FazFhVzh3TlVSTlRqTlBSVkZ3ZFdVMVREZG9iRVJwU2xkSFdVWlZVbXBJTlhCbFFYVlNiRTA0ZERKU1RVbDRWbkp4ZEdWb0sxRlNNa0o0Y1cwS1IwNUNTekIyUXpkRk4waGlTbTR4TTA0MlZ6QXlVVzl1WnlzMmMweFhUMVI0V1ZWSVUzRTJZbEF6ZFVFNWFHeDZNV3QzZVVNNGRIbGpWMnhEVUZGb01RcFJVMFpHWVVWRVJtdG1TMDgwVTB0WFlVUnhORkJVV0ZkUWFTOXNNVmt4Y1RJek9GUTRXRGx4TVZsSFl6aFRkR3BZTUV0WFVHWXlWVXhYTlRVMFExZHhDaXRDTlZCNkwwaENZWEJoYzFkNVNFa3pNWEJUWWs4MVJEZGtja2s1Y21WWFNrcEdTRVJ4ZDNkYVIyaElZbVZGZVdFNVYxTlJXa2hOVG1OYVVtUnRNM0FLTlM5eFZuTlJZMHB5Tld4VGJFdEhaM0ZPVERoWVNuazBibVZuUFFvdExTMHRMVVZPUkNCRFJWSlVTVVpKUTBGVVJTMHRMUzB0Q2c9PSIsInZlcmlmaWNhdGlvbiI6e319XX19LCJ0aW1lb3V0cyI6e30sInZlcnNpb24iOiIyLjIuMCJ9LCJuZXR3b3JrZCI6e30sInBhc3N3ZCI6e30sInN0b3JhZ2UiOnt9LCJzeXN0ZW1kIjp7fX0=', u'security_groups': [{u'name': u'842bf4b1-0f72-4e8d-b9ad-13cc838d3475'}], u'metadata': {u'openshiftClusterID': u'shared-jiazha-8ck2c', u'Name': u'shared-jiazha-8ck2c-worker'}}. 'networks' is a required property", "code": 400}}

Expected results:
Logs should suggest subnet is invalid.

Additional info:

Comment 1 egarcia 2019-09-10 16:04:51 UTC
The log output is really ugly, but doesn't it say that the subnet was invalid right here:


controller.go:249] Failed to create machine "shared-jiazha-8ck2c-worker-invalidsubnet": error creating Openstack instance: Create new server err: Bad request with: [POST https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13774/v2.1/servers],


error message: {"badRequest": {"message": "Invalid input for field/attribute server. Value: {u'name': u'shared-jiazha-8ck2c-worker-invalidsubnet', u'imageRef': u'8fe77b79-2a62-4b90-9f95-67c7bcc88070', u'tags': [u'cluster-api-provider-openstack', u'openshift-machine-api-shared-jiazha-8ck2c', u'openshiftClusterID=shared-jiazha-8ck2c'], u'flavorRef': u'a9acc2de-39d7-4148-8d16-413c3b696e9d', u'user_data': ....

Comment 2 Pierre Prinetti 2019-09-12 16:14:51 UTC
The error message from OpenStack mentions that the 'networks' property is missing from the Instance definition.

And indeed, CAPO's machine-service compiles the Instance definition by inserting any real NetworkID found after querying OpenStack with the Machine's Network property filters.
If no valid NetworkID is found, the Instance's Network section is left empty.

Comment 4 sunzhaohua 2019-09-16 06:13:07 UTC
Verifed.

clusterversion: 4.2.0-0.nightly-2019-09-15-052022


I0916 06:11:04.704983       1 controller.go:129] Reconciling Machine "share-0916c-8vp8z-worker-aaa"
I0916 06:11:04.705022       1 controller.go:298] Machine "share-0916c-8vp8z-worker-aaa" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster
I0916 06:11:04.711614       1 controller.go:129] Reconciling Machine "share-0916c-8vp8z-worker-aaa"
I0916 06:11:04.711640       1 controller.go:298] Machine "share-0916c-8vp8z-worker-aaa" in namespace "openshift-machine-api" doesn't specify "cluster.k8s.io/cluster-name" label, assuming nil cluster
I0916 06:11:05.005775       1 controller.go:247] Reconciling machine object share-0916c-8vp8z-worker-aaa triggers idempotent create.
E0916 06:11:11.340611       1 actuator.go:470] Machine error share-0916c-8vp8z-worker-aaa: error creating Openstack instance: No network was found or provided. Please check your machine configuration and try again
W0916 06:11:11.340640       1 controller.go:249] Failed to create machine "share-0916c-8vp8z-worker-aaa": error creating Openstack instance: No network was found or provided. Please check your machine configuration and try again

Comment 5 errata-xmlrpc 2019-10-16 06:37:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922