Hide Forgot
Description of problem: Support for kuryr pool driver selection is needed, i.e., neutron for baremetal deployments or nested for deployments on top of VMs, or even a multi driver for hybrid deployments. Version-Release number of the following components: rpm -q openshift-ansible openshift-ansible-roles-3.10.0-0.27.0.git.0.abed3b7.el7.noarch openshift-ansible-docs-3.10.0-0.27.0.git.0.abed3b7.el7.noarch openshift-ansible-3.10.0-0.27.0.git.0.abed3b7.el7.noarch openshift-ansible-playbooks-3.10.0-0.27.0.git.0.abed3b7.el7.noarch rpm -q ansible ansible-2.4.3.0-1.el7ae.noarch ansible --version ansible 2.4.3.0 config file = /etc/ansible/ansible.cfg configured module search path = [u'/home/cloud-user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Feb 9 2018, 09:51:13) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
Should be in openshift-ansible-3.11.0-0.15.0
Per OCP program call on 21-SEP-2018 we are deferring Kuryr-related bugs to 3.11.z
Verified on openshift-ansible-3.11.18-1.git.0.033596c.el7.noarch, with: . openstack-kuryr-controller 14.0-80 container image . openstack-kuryr-cni 14.0-75 container image Note it only works for OSP 14 Kuryr containers. Verification steps: 1. Install OSP 14, or OSP 13 but use OSP 14 kuryr containers 2. Install OCP 3.11, with the default config for kuryr_openstack_pool_driver (no need to specify it) 3. Check nested-vlan label is not present for all the nodes: $ oc get nodes --show-labels NAME STATUS ROLES AGE VERSION LABELS app-node-0.openshift.example.com Ready compute 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=f5cfa48f-4428-421d-8938-f22b40e66636,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=app-node-0.openshift.example.com,node-role.kubernetes.io/compute=true app-node-1.openshift.example.com Ready compute 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=f5cfa48f-4428-421d-8938-f22b40e66636,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=app-node-1.openshift.example.com,node-role.kubernetes.io/compute=true infra-node-0.openshift.example.com Ready infra 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=f5cfa48f-4428-421d-8938-f22b40e66636,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=infra-node-0.openshift.example.com,node-role.kubernetes.io/infra=true master-0.openshift.example.com Ready master 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=c4885424-715b-4f78-8849-2fb3287d753e,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=master-0.openshift.example.com,node-role.kubernetes.io/master=true 4. Check basic OCP operation 5. Uninstall OCP 6. Install OCP 3.11, enabling Kuryr multi driver in OSEv3.yml: # Kuryr label configuration kuryr_openstack_pool_driver: multi openshift_node_groups: - name: node-config-master labels: - 'node-role.kubernetes.io/master=true' - 'pod_vif=nested-vlan' edits: [] - name: node-config-infra labels: - 'node-role.kubernetes.io/infra=true' - 'pod_vif=nested-vlan' edits: [] - name: node-config-compute labels: - 'node-role.kubernetes.io/compute=true' - 'pod_vif=nested-vlan' edits: [] 7. Check nested-vlan label is present for all the nodes: $ oc get nodes --show-labels NAME STATUS ROLES AGE VERSION LABELS app-node-0.openshift.example.com Ready compute 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=f5cfa48f-4428-421d-8938-f22b40e66636,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=app-node-0.openshift.example.com,node-role.kubernetes.io/compute=true,pod_vif=nested-vlan app-node-1.openshift.example.com Ready compute 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=f5cfa48f-4428-421d-8938-f22b40e66636,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=app-node-1.openshift.example.com,node-role.kubernetes.io/compute=true,pod_vif=nested-vlan infra-node-0.openshift.example.com Ready infra 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=f5cfa48f-4428-421d-8938-f22b40e66636,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=infra-node-0.openshift.example.com,node-role.kubernetes.io/infra=true,pod_vif=nested-vlan master-0.openshift.example.com Ready master 3h v1.11.0+d4cacc0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=c4885424-715b-4f78-8849-2fb3287d753e,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=master-0.openshift.example.com,node-role.kubernetes.io/master=true,pod_vif=nested-vlan 8. Create and expose and application: $ oc new-project test $ oc run --image=kuryr/demo demo $ oc scale deploymentconfig demo --replicas=3 $ oc get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE demo-1-4wlbj 1/1 Running 0 1h 10.11.0.8 app-node-1.openshift.example.com <none> demo-1-c2px6 1/1 Running 0 1h 10.11.0.34 app-node-1.openshift.example.com <none> demo-1-qwmnv 1/1 Running 0 1h 10.11.0.22 app-node-0.openshift.example.com <none> $ oc expose deploymentconfig demo --port=80 --target-port=8080 --type=LoadBalancer $ oc get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE demo LoadBalancer 172.30.39.33 10.46.22.53 80:32638/TCP 2m 9. Check connectivity between pods: $ oc rsh demo-1-4wlbj ping 10.11.0.34 -c 1 PING 10.11.0.34 (10.11.0.34): 56 data bytes 64 bytes from 10.11.0.34: seq=0 ttl=64 time=0.675 ms --- 10.11.0.34 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 0.675/0.675/0.675 ms $ oc rsh demo-1-4wlbj ping 10.11.0.22 -c 1 PING 10.11.0.22 (10.11.0.22): 56 data bytes 64 bytes from 10.11.0.22: seq=0 ttl=64 time=0.835 ms --- 10.11.0.22 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 0.835/0.835/0.835 ms 10. Check pod ports belong to trunk ports: (shiftstack) [cloud-user@ansible-host-0 ~]$ openstack network trunk list +--------------------------------------+------------------------------------+--------------------------------------+-------------+ | ID | Name | Parent Port | Description | +--------------------------------------+------------------------------------+--------------------------------------+-------------+ | 2a2ade3b-6d1e-44ff-8a63-b441125b2978 | master-0.openshift.example.com | 1e3ac9a8-f212-4ed8-a0d1-b7005c1440f5 | | | 3c38db50-88b9-4027-9c82-30c983658a2e | app-node-1.openshift.example.com | bc59357b-902a-4739-8b43-ede9587e6e6e | | | b015057f-55cc-4e54-b34c-ae270a5b8d46 | app-node-0.openshift.example.com | 7095a643-38ad-4d80-aecd-fa91b7ae9d64 | | | c8dd02e5-c4e1-4abc-8b42-978cb100a3af | infra-node-0.openshift.example.com | 8ad0440b-d0e5-437d-a909-16d73d5a05f8 | | +--------------------------------------+------------------------------------+--------------------------------------+-------------+ (shiftstack) [cloud-user@ansible-host-0 ~]$ openstack network trunk show app-node-1.openshift.example.com +-----------------+--------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------+--------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | created_at | 2018-10-11T11:06:50Z | | description | | | id | 3c38db50-88b9-4027-9c82-30c983658a2e | | name | app-node-1.openshift.example.com | | port_id | bc59357b-902a-4739-8b43-ede9587e6e6e | | project_id | 0127ae37e08b4ef9972632e71f190b14 | | revision_number | 4 | | status | ACTIVE | | sub_ports | port_id='43ef4aa9-038b-4aa1-bb16-3d72f43c01fa', segmentation_id='260', segmentation_type='vlan' | | | port_id='a02dc715-f45e-472d-8867-65695c55ed8f', segmentation_id='1079', segmentation_type='vlan' | | | port_id='1fc61706-2eed-49aa-b343-a936e2b4b279', segmentation_id='2529', segmentation_type='vlan' | | | port_id='2218fa93-c66f-4c7c-b67b-698fa27d0785', segmentation_id='3441', segmentation_type='vlan' | | | port_id='c049c8af-75ac-4599-b0da-0889d7ce70ef', segmentation_id='3528', segmentation_type='vlan' | | tags | [] | | tenant_id | 0127ae37e08b4ef9972632e71f190b14 | | updated_at | 2018-10-11T13:03:22Z | +-----------------+--------------------------------------------------------------------------------------------------+ (shiftstack) [cloud-user@ansible-host-0 ~]$ openstack port list | grep 10.11.0.34 | c049c8af-75ac-4599-b0da-0889d7ce70ef | | fa:16:3e:2c:0a:89 | ip_address='10.11.0.34', subnet_id='bf8a0880-a81b-4698-a4cc-2be6bca67512' | ACTIVE | (shiftstack) [cloud-user@ansible-host-0 ~]$ openstack port list | grep 10.11.0.8 | a02dc715-f45e-472d-8867-65695c55ed8f | | fa:16:3e:43:71:be | ip_address='10.11.0.8', subnet_id='bf8a0880-a81b-4698-a4cc-2be6bca67512' | ACTIVE | (shiftstack) [cloud-user@ansible-host-0 ~]$ openstack network trunk show app-node-0.openshift.example.com +-----------------+--------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------+--------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | created_at | 2018-10-11T11:06:50Z | | description | | | id | b015057f-55cc-4e54-b34c-ae270a5b8d46 | | name | app-node-0.openshift.example.com | | port_id | 7095a643-38ad-4d80-aecd-fa91b7ae9d64 | | project_id | 0127ae37e08b4ef9972632e71f190b14 | | revision_number | 4 | | status | ACTIVE | | sub_ports | port_id='a8339996-e716-464e-9897-fb09bdb9b615', segmentation_id='794', segmentation_type='vlan' | | | port_id='91c37ebd-0682-4540-8497-247896ec2f99', segmentation_id='1030', segmentation_type='vlan' | | | port_id='439f3f86-cf30-4744-a020-2b74c60ebc0f', segmentation_id='1999', segmentation_type='vlan' | | | port_id='31cd2a4a-6540-4eb7-ad33-86568d85b988', segmentation_id='2368', segmentation_type='vlan' | | | port_id='0ff74019-3837-4a8a-b50f-46c73cf1d8f6', segmentation_id='2740', segmentation_type='vlan' | | tags | [] | | tenant_id | 0127ae37e08b4ef9972632e71f190b14 | | updated_at | 2018-10-11T13:04:02Z | +-----------------+--------------------------------------------------------------------------------------------------+ (shiftstack) [cloud-user@ansible-host-0 ~]$ openstack port list | grep 10.11.0.22 | 91c37ebd-0682-4540-8497-247896ec2f99 | | fa:16:3e:4d:67:34 | ip_address='10.11.0.22', subnet_id='bf8a0880-a81b-4698-a4cc-2be6bca67512' | ACTIVE | 11. Delete the test project: $ oc delete project test
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0024