Bug 1503975 - [RFE] OpenStack Kuryr port pool support
Summary: [RFE] OpenStack Kuryr port pool support
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RFE
Version: unspecified
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.10.0
Assignee: Paul Weil
QA Contact: Xiaoli Tian
URL:
Whiteboard:
Depends On: 1503243
Blocks: 1503972
TreeView+ depends on / blocked
 
Reported: 2017-10-19 08:16 UTC by Antoni Segura Puimedon
Modified: 2018-12-20 21:42 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-12-20 21:42:05 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift openshift-ansible pull 6021 None closed Initial Kuryr Ports Pool Support 2020-09-09 10:35:51 UTC
Github openshift openshift-ansible pull 6062 None closed Add readiness probe to kuryr controller pod 2020-09-09 10:35:51 UTC
Github openshift openshift-ansible pull 6248 None closed Add support to pre-create subports at each trunk 2020-09-09 10:35:51 UTC
Github openshift openshift-ansible pull 7590 None closed Make ports pool the default when deploying with kuryr 2020-09-09 10:35:51 UTC
OpenStack gerrit 530942 None MERGED Fix bypass list length check when all values are None 2020-09-09 10:35:51 UTC

Description Antoni Segura Puimedon 2017-10-19 08:16:51 UTC
Description of problem:
Port pooling is a way to isolate the user from the latency inherent to Neutron bulk creation and deletion operations. The way it works is allocating and/or pre-allocating Neutron ports and assign them to the N worker nodes in the cluster.

This patch is about:
a) Adding inventory options that will make the Kuryr configmaps be set and tuned for the controller (it should include the knobs that kuryr pool drivers expose).
b) Adds an option to pre-create ports for the pool and consequently creates the ports for the pool.


Steps to Reproduce:
1. Set the kuryr pooling options in the inventory file with a pre-created pool size of 200
2. ansible-playbook playbooks/byo/config.yml -vvv
3. oc -n kube-system get ConfigMap -o yaml

Actual results:
No Port Pooling options are set in the ConfigMap or just the default, not the ones set in the inventory.

Expected results:
The inventory set port pooling options are in the kuryr controller ConfigMap.
200 pool ports were created
The controller pod is ready, which means the pool was loaded.

Comment 7 Jon Uriarte 2018-07-19 09:49:09 UTC
This RFE was already verified in openshift-ansible-3.10.0-0.67.0 over OSP 13 2018-05-23.1 puddle.

1. Leave kuryr configuration by default, but set openshift_kuryr_precreate_subports (in inventory/group_vars/all.yml):
  ## Ports pooling is enabled by default (with nested driver).
  ## To disable, uncomment the next:
  #kuryr_openstack_pool_driver: noop
  #
  ## You can also alter the port pooling defaults here
  #kuryr_openstack_pool_max: 0
  #kuryr_openstack_pool_min: 1
  #kuryr_openstack_pool_batch: 5
  #kuryr_openstack_pool_update_frequency: 20
  #
  ## You can pre-populate the pools with subports by uncommenting the next line
  ## and specifying the amount of subports you want to pre-create per
  ## origin-node, e.g.: openshift_kuryr_precreate_subports: 5
  openshift_kuryr_precreate_subports: 5
  #
  ## You can also change the default device owner for the precreated subports
  openshift_kuryr_device_owner: compute:kuryr

2. Deploy kuryr with OpenShift-Ansible
3. Check the ConfigMap contains port pool configuration
   $ oc -n openshift-infra get cm kuryr-config -o yaml
    ...
    [kubernetes]
    ...
    vif_pool_driver = nested
    ...

    [vif_pool]
              
    #
    # From kuryr_kubernetes
    #
    # Set a maximun amount of ports per pool. 0 to disable (integer value)
    ports_pool_max = 0                           
                                                                          
    # Set a target minimum size of the pool of ports (integer value)      
    ports_pool_min = 1                                                    
                                            
    # Number of ports to be created in a bulk request (integer value)     
    ports_pool_batch = 5                                                  
                                                  
    # Minimun interval (in seconds) between pool updates (integer value)  
    ports_pool_update_frequency = 20
    ...

4. Check precreated ports per trunk in openstack.

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk list
+--------------------------------------+------------------------------------+--------------------------------------+-------------+
| ID                                   | Name                               | Parent Port                          | Description |
+--------------------------------------+------------------------------------+--------------------------------------+-------------+
| 4c877f6f-1055-4149-a670-b38c4180bf6f | master-0.openshift.example.com     | a9829c12-dede-4e04-8b1b-05e515a750a1 |             |
| 531dba6e-2ef6-4434-97d6-12b5bfdb0329 | infra-node-0.openshift.example.com | 34269f29-4f35-4616-a2ff-a5e0f3a33984 |             |
| a70d7cdf-b607-43e0-bd16-3860083524ff | app-node-1.openshift.example.com   | a83cfad4-be92-418d-8798-2fe184515f6a |             |
| aa6bf2b4-bc93-4adb-b858-01d55a778b2a | app-node-0.openshift.example.com   | 93a7e85d-c410-4d34-ae04-7822223c74fc |             |
+--------------------------------------+------------------------------------+--------------------------------------+-------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show master-0.openshift.example.com
+-----------------+--------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                            |
+-----------------+--------------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                               |
| created_at      | 2018-05-08T10:00:02Z                                                                             |
| description     |                                                                                                  |
| id              | 4c877f6f-1055-4149-a670-b38c4180bf6f                                                             |
| name            | master-0.openshift.example.com                                                                   |
| port_id         | a9829c12-dede-4e04-8b1b-05e515a750a1                                                             |
| project_id      | 6c07532860e641989bacc5583275080a                                                                 |
| revision_number | 5                                                                                                |
| status          | ACTIVE                                                                                           |
| sub_ports       | port_id='026788d7-06db-4911-b4f8-6eee8cabeb06', segmentation_id='1', segmentation_type='vlan'    |
|                 | port_id='7a7ca31b-2178-45cd-bd3d-c581aa41eb10', segmentation_id='2', segmentation_type='vlan'    |
|                 | port_id='83cd5723-e2e3-4a59-8243-4e1aab704aa4', segmentation_id='3', segmentation_type='vlan'    |
|                 | port_id='660dd6b7-dfd9-491a-9db1-2115d6dbd4b3', segmentation_id='4', segmentation_type='vlan'    |
|                 | port_id='617846f3-c2d4-4b50-80ef-e6686119bb95', segmentation_id='5', segmentation_type='vlan'    |
|                 | port_id='8df19782-2539-403c-96cc-fc659abea238', segmentation_id='3444', segmentation_type='vlan' |
|                 | port_id='a0076398-5930-4ffb-ba51-335b6a5190dc', segmentation_id='3464', segmentation_type='vlan' |
|                 | port_id='4d2c4fbb-b078-4fae-ac31-76f04ba0839c', segmentation_id='3728', segmentation_type='vlan' |
|                 | port_id='e953854f-d1a0-430f-8cf9-ec81d37cf8b0', segmentation_id='3815', segmentation_type='vlan' |
|                 | port_id='e767d2d5-b79d-4ede-abab-f6bb9dafa149', segmentation_id='3948', segmentation_type='vlan' |
| tags            | []                                                                                               |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                                 |
| updated_at      | 2018-05-08T10:56:54Z                                                                             |
+-----------------+--------------------------------------------------------------------------------------------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show infra-node-0.openshift.example.com
+-----------------+--------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                            |
+-----------------+--------------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                               |
| created_at      | 2018-05-08T09:59:35Z                                                                             |
| description     |                                                                                                  |
| id              | 531dba6e-2ef6-4434-97d6-12b5bfdb0329                                                             |
| name            | infra-node-0.openshift.example.com                                                               |
| port_id         | 34269f29-4f35-4616-a2ff-a5e0f3a33984                                                             |
| project_id      | 6c07532860e641989bacc5583275080a                                                                 |
| revision_number | 5                                                                                                |
| status          | ACTIVE                                                                                           |
| sub_ports       | port_id='66aa7ce2-83c2-4b07-8fa5-bb81270cf445', segmentation_id='1', segmentation_type='vlan'    |
|                 | port_id='8ea29a23-7895-4e40-8441-9d521821de90', segmentation_id='2', segmentation_type='vlan'    |
|                 | port_id='8326327a-5e08-45c8-b448-f7e8c61feb0b', segmentation_id='3', segmentation_type='vlan'    |
|                 | port_id='a66883d5-210a-49c9-8924-58ed9512f976', segmentation_id='4', segmentation_type='vlan'    |
|                 | port_id='0a0d36f8-a55a-44dd-96a0-e93037caf6c8', segmentation_id='5', segmentation_type='vlan'    |
|                 | port_id='07f13bae-d44d-44d6-8b0a-47838a266bbd', segmentation_id='1231', segmentation_type='vlan' |
|                 | port_id='d4f377cf-1bb3-40c4-b1c6-968509489a30', segmentation_id='1818', segmentation_type='vlan' |
|                 | port_id='86d280f3-bf1d-4c73-991a-64086bd7a063', segmentation_id='1851', segmentation_type='vlan' |
|                 | port_id='7288719a-fdd4-43db-9f1a-df3cf712cf08', segmentation_id='1927', segmentation_type='vlan' |
|                 | port_id='dd16a816-fc6c-4a47-833f-9fe79ead806e', segmentation_id='3814', segmentation_type='vlan' |
| tags            | []                                                                                               |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                                 |
| updated_at      | 2018-05-08T10:56:57Z                                                                             |
+-----------------+--------------------------------------------------------------------------------------------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show app-node-0.openshift.example.com
+-----------------+-----------------------------------------------------------------------------------------------+
| Field           | Value                                                                                         |
+-----------------+-----------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                            |
| created_at      | 2018-05-08T09:59:34Z                                                                          |
| description     |                                                                                               |
| id              | aa6bf2b4-bc93-4adb-b858-01d55a778b2a                                                          |
| name            | app-node-0.openshift.example.com                                                              |
| port_id         | 93a7e85d-c410-4d34-ae04-7822223c74fc                                                          |
| project_id      | 6c07532860e641989bacc5583275080a                                                              |
| revision_number | 2                                                                                             |
| status          | ACTIVE                                                                                        |
| sub_ports       | port_id='1d76be7c-b711-4fe2-a941-1922318e3475', segmentation_id='1', segmentation_type='vlan' |
|                 | port_id='89caadf0-9ed4-4959-8638-4f90e3d90d6d', segmentation_id='2', segmentation_type='vlan' |
|                 | port_id='ada9a8ad-fbf3-4aac-81d2-b27a9e9910a6', segmentation_id='3', segmentation_type='vlan' |
|                 | port_id='8cabf3bf-d403-4a43-b3c6-7bd6833b7e6b', segmentation_id='4', segmentation_type='vlan' |
|                 | port_id='716bde48-2ec2-4aab-8ee4-d3e3c46ca372', segmentation_id='5', segmentation_type='vlan' |
| tags            | []                                                                                            |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                              |
| updated_at      | 2018-05-08T09:59:46Z                                                                          |
+-----------------+-----------------------------------------------------------------------------------------------+

(overcloud) [cloud-user@ansible-host ~]$ openstack network trunk show app-node-1.openshift.example.com
+-----------------+-----------------------------------------------------------------------------------------------+
| Field           | Value                                                                                         |
+-----------------+-----------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                            |
| created_at      | 2018-05-08T09:59:34Z                                                                          |
| description     |                                                                                               |
| id              | a70d7cdf-b607-43e0-bd16-3860083524ff                                                          |
| name            | app-node-1.openshift.example.com                                                              |
| port_id         | a83cfad4-be92-418d-8798-2fe184515f6a                                                          |
| project_id      | 6c07532860e641989bacc5583275080a                                                              |
| revision_number | 2                                                                                             |
| status          | ACTIVE                                                                                        |
| sub_ports       | port_id='ec61324d-e6e5-4abf-8f3c-ab399445b05c', segmentation_id='1', segmentation_type='vlan' |
|                 | port_id='e1da30ac-0a6b-4efc-8346-32e59eb79ca2', segmentation_id='2', segmentation_type='vlan' |
|                 | port_id='546cc02a-a33d-4a24-91eb-834f4cc03e83', segmentation_id='3', segmentation_type='vlan' |
|                 | port_id='7ea4a036-33c8-4966-b0b9-e11993a92bfb', segmentation_id='4', segmentation_type='vlan' |
|                 | port_id='46ec2829-5d1e-417f-adf2-9e071b6f7801', segmentation_id='5', segmentation_type='vlan' |
| tags            | []                                                                                            |
| tenant_id       | 6c07532860e641989bacc5583275080a                                                              |
| updated_at      | 2018-05-08T09:59:52Z                                                                          |
+-----------------+-----------------------------------------------------------------------------------------------+

5 subports with vlan from 1 to 5 have been precreated in each openshift node.


Note You need to log in before you can comment on or make changes to this bug.