Bug 1503975

Summary: [RFE] OpenStack Kuryr port pool support
Product: OpenShift Container Platform Reporter: Antoni Segura Puimedon <asegurap>
Component: RFEAssignee: Paul Weil <pweil>
Status: CLOSED CURRENTRELEASE QA Contact: Xiaoli Tian <xtian>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: aos-bugs, asegurap, jokerman, juriarte, ltomasbo, mmccomas, oblaut, smunilla, tsedovic, tzumainn
Target Milestone: ---Keywords: FutureFeature, Triaged
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-20 21:42:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1503243    
Bug Blocks: 1503972    

Description Antoni Segura Puimedon 2017-10-19 08:16:51 UTC 
Description of problem:
Port pooling is a way to isolate the user from the latency inherent to Neutron bulk creation and deletion operations. The way it works is allocating and/or pre-allocating Neutron ports and assign them to the N worker nodes in the cluster.

This patch is about:
a) Adding inventory options that will make the Kuryr configmaps be set and tuned for the controller (it should include the knobs that kuryr pool drivers expose).
b) Adds an option to pre-create ports for the pool and consequently creates the ports for the pool.


Steps to Reproduce:
1. Set the kuryr pooling options in the inventory file with a pre-created pool size of 200
2. ansible-playbook playbooks/byo/config.yml -vvv
3. oc -n kube-system get ConfigMap -o yaml

Actual results:
No Port Pooling options are set in the ConfigMap or just the default, not the ones set in the inventory.

Expected results:
The inventory set port pooling options are in the kuryr controller ConfigMap.
200 pool ports were created
The controller pod is ready, which means the pool was loaded.
Comment 7 Jon Uriarte 2018-07-19 09:49:09 UTC 
This RFE was already verified in openshift-ansible-3.10.0-0.67.0 over OSP 13 2018-05-23.1 puddle.

1. Leave kuryr configuration by default, but set openshift_kuryr_precreate_subports (in inventory/group_vars/all.yml):
  ## Ports pooling is enabled by default (with nested driver).
  ## To disable, uncomment the next:
  #kuryr_openstack_pool_driver: noop
  #
  ## You can also alter the port pooling defaults here
  #kuryr_openstack_pool_max: 0
  #kuryr_openstack_pool_min: 1
  #kuryr_openstack_pool_batch: 5
  #kuryr_openstack_pool_update_frequency: 20
  #
  ## You can pre-populate the pools with subports by uncommenting the next line
  ## and specifying the amount of subports you want to pre-create per
  ## origin-node, e.g.: openshift_kuryr_precreate_subports: 5
  openshift_kuryr_precreate_subports: 5
  #
  ## You can also change the default device owner for the precreated subports
  openshift_kuryr_device_owner: compute:kuryr

2. Deploy kuryr with OpenShift-Ansible
3. Check the ConfigMap contains port pool configuration
   $ oc -n openshift-infra get cm kuryr-config -o yaml
    ...
    [kubernetes]
    ...
    vif_pool_driver = nested
    ...

    [vif_pool]
              
    #
    # From kuryr_kubernetes
    #
    # Set a maximun amount of ports per pool. 0 to disable (integer value)
    ports_pool_max = 0                           
                                                                          
    # Set a target minimum size of the pool of ports (integer value)      
    ports_pool_min = 1                                                    
                                            
    # Number of ports to be created in a bulk request (integer value)     
    ports_pool_batch = 5                                                  
                                                  
    # Minimun interval (in seconds) between pool updates (integer value)  
    ports_pool_update_frequency = 20
    ...

4. Check precreated ports per trunk in openstack.

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk list
+--------------------------------------+------------------------------------+--------------------------------------+-------------+
| ID                                   | Name                               | Parent Port                          | Description |
+--------------------------------------+------------------------------------+--------------------------------------+-------------+
| 4c877f6f-1055-4149-a670-b38c4180bf6f | master-0.openshift.example.com     | a9829c12-dede-4e04-8b1b-05e515a750a1 |             |
| 531dba6e-2ef6-4434-97d6-12b5bfdb0329 | infra-node-0.openshift.example.com | 34269f29-4f35-4616-a2ff-a5e0f3a33984 |             |
| a70d7cdf-b607-43e0-bd16-3860083524ff | app-node-1.openshift.example.com   | a83cfad4-be92-418d-8798-2fe184515f6a |             |
| aa6bf2b4-bc93-4adb-b858-01d55a778b2a | app-node-0.openshift.example.com   | 93a7e85d-c410-4d34-ae04-7822223c74fc |             |
+--------------------------------------+------------------------------------+--------------------------------------+-------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show master-0.openshift.example.com
+-----------------+--------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                            |
+-----------------+--------------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                               |
| created_at      | 2018-05-08T10:00:02Z                                                                             |
| description     |                                                                                                  |
| id              | 4c877f6f-1055-4149-a670-b38c4180bf6f                                                             |
| name            | master-0.openshift.example.com                                                                   |
| port_id         | a9829c12-dede-4e04-8b1b-05e515a750a1                                                             |
| project_id      | 6c07532860e641989bacc5583275080a                                                                 |
| revision_number | 5                                                                                                |
| status          | ACTIVE                                                                                           |
| sub_ports       | port_id='026788d7-06db-4911-b4f8-6eee8cabeb06', segmentation_id='1', segmentation_type='vlan'    |
|                 | port_id='7a7ca31b-2178-45cd-bd3d-c581aa41eb10', segmentation_id='2', segmentation_type='vlan'    |
|                 | port_id='83cd5723-e2e3-4a59-8243-4e1aab704aa4', segmentation_id='3', segmentation_type='vlan'    |
|                 | port_id='660dd6b7-dfd9-491a-9db1-2115d6dbd4b3', segmentation_id='4', segmentation_type='vlan'    |
|                 | port_id='617846f3-c2d4-4b50-80ef-e6686119bb95', segmentation_id='5', segmentation_type='vlan'    |
|                 | port_id='8df19782-2539-403c-96cc-fc659abea238', segmentation_id='3444', segmentation_type='vlan' |
|                 | port_id='a0076398-5930-4ffb-ba51-335b6a5190dc', segmentation_id='3464', segmentation_type='vlan' |
|                 | port_id='4d2c4fbb-b078-4fae-ac31-76f04ba0839c', segmentation_id='3728', segmentation_type='vlan' |
|                 | port_id='e953854f-d1a0-430f-8cf9-ec81d37cf8b0', segmentation_id='3815', segmentation_type='vlan' |
|                 | port_id='e767d2d5-b79d-4ede-abab-f6bb9dafa149', segmentation_id='3948', segmentation_type='vlan' |
| tags            | []                                                                                               |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                                 |
| updated_at      | 2018-05-08T10:56:54Z                                                                             |
+-----------------+--------------------------------------------------------------------------------------------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show infra-node-0.openshift.example.com
+-----------------+--------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                            |
+-----------------+--------------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                               |
| created_at      | 2018-05-08T09:59:35Z                                                                             |
| description     |                                                                                                  |
| id              | 531dba6e-2ef6-4434-97d6-12b5bfdb0329                                                             |
| name            | infra-node-0.openshift.example.com                                                               |
| port_id         | 34269f29-4f35-4616-a2ff-a5e0f3a33984                                                             |
| project_id      | 6c07532860e641989bacc5583275080a                                                                 |
| revision_number | 5                                                                                                |
| status          | ACTIVE                                                                                           |
| sub_ports       | port_id='66aa7ce2-83c2-4b07-8fa5-bb81270cf445', segmentation_id='1', segmentation_type='vlan'    |
|                 | port_id='8ea29a23-7895-4e40-8441-9d521821de90', segmentation_id='2', segmentation_type='vlan'    |
|                 | port_id='8326327a-5e08-45c8-b448-f7e8c61feb0b', segmentation_id='3', segmentation_type='vlan'    |
|                 | port_id='a66883d5-210a-49c9-8924-58ed9512f976', segmentation_id='4', segmentation_type='vlan'    |
|                 | port_id='0a0d36f8-a55a-44dd-96a0-e93037caf6c8', segmentation_id='5', segmentation_type='vlan'    |
|                 | port_id='07f13bae-d44d-44d6-8b0a-47838a266bbd', segmentation_id='1231', segmentation_type='vlan' |
|                 | port_id='d4f377cf-1bb3-40c4-b1c6-968509489a30', segmentation_id='1818', segmentation_type='vlan' |
|                 | port_id='86d280f3-bf1d-4c73-991a-64086bd7a063', segmentation_id='1851', segmentation_type='vlan' |
|                 | port_id='7288719a-fdd4-43db-9f1a-df3cf712cf08', segmentation_id='1927', segmentation_type='vlan' |
|                 | port_id='dd16a816-fc6c-4a47-833f-9fe79ead806e', segmentation_id='3814', segmentation_type='vlan' |
| tags            | []                                                                                               |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                                 |
| updated_at      | 2018-05-08T10:56:57Z                                                                             |
+-----------------+--------------------------------------------------------------------------------------------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show app-node-0.openshift.example.com
+-----------------+-----------------------------------------------------------------------------------------------+
| Field           | Value                                                                                         |
+-----------------+-----------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                            |
| created_at      | 2018-05-08T09:59:34Z                                                                          |
| description     |                                                                                               |
| id              | aa6bf2b4-bc93-4adb-b858-01d55a778b2a                                                          |
| name            | app-node-0.openshift.example.com                                                              |
| port_id         | 93a7e85d-c410-4d34-ae04-7822223c74fc                                                          |
| project_id      | 6c07532860e641989bacc5583275080a                                                              |
| revision_number | 2                                                                                             |
| status          | ACTIVE                                                                                        |
| sub_ports       | port_id='1d76be7c-b711-4fe2-a941-1922318e3475', segmentation_id='1', segmentation_type='vlan' |
|                 | port_id='89caadf0-9ed4-4959-8638-4f90e3d90d6d', segmentation_id='2', segmentation_type='vlan' |
|                 | port_id='ada9a8ad-fbf3-4aac-81d2-b27a9e9910a6', segmentation_id='3', segmentation_type='vlan' |
|                 | port_id='8cabf3bf-d403-4a43-b3c6-7bd6833b7e6b', segmentation_id='4', segmentation_type='vlan' |
|                 | port_id='716bde48-2ec2-4aab-8ee4-d3e3c46ca372', segmentation_id='5', segmentation_type='vlan' |
| tags            | []                                                                                            |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                              |
| updated_at      | 2018-05-08T09:59:46Z                                                                          |
+-----------------+-----------------------------------------------------------------------------------------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show app-node-1.openshift.example.com
+-----------------+-----------------------------------------------------------------------------------------------+
| Field           | Value                                                                                         |
+-----------------+-----------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                            |
| created_at      | 2018-05-08T09:59:34Z                                                                          |
| description     |                                                                                               |
| id              | a70d7cdf-b607-43e0-bd16-3860083524ff                                                          |
| name            | app-node-1.openshift.example.com                                                              |
| port_id         | a83cfad4-be92-418d-8798-2fe184515f6a                                                          |
| project_id      | 6c07532860e641989bacc5583275080a                                                              |
| revision_number | 2                                                                                             |
| status          | ACTIVE                                                                                        |
| sub_ports       | port_id='ec61324d-e6e5-4abf-8f3c-ab399445b05c', segmentation_id='1', segmentation_type='vlan' |
|                 | port_id='e1da30ac-0a6b-4efc-8346-32e59eb79ca2', segmentation_id='2', segmentation_type='vlan' |
|                 | port_id='546cc02a-a33d-4a24-91eb-834f4cc03e83', segmentation_id='3', segmentation_type='vlan' |
|                 | port_id='7ea4a036-33c8-4966-b0b9-e11993a92bfb', segmentation_id='4', segmentation_type='vlan' |
|                 | port_id='46ec2829-5d1e-417f-adf2-9e071b6f7801', segmentation_id='5', segmentation_type='vlan' |
| tags            | []                                                                                            |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                              |
| updated_at      | 2018-05-08T09:59:52Z                                                                          |
+-----------------+-----------------------------------------------------------------------------------------------+

5 subports with vlan from 1 to 5 have been precreated in each openshift node.