Bug 1504662 - [RFE] Fast bulk neutron port create/update operations
Summary: [RFE] Fast bulk neutron port create/update operations
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: z1
: 15.0 (Stein)
Assignee: Nate Johnston
QA Contact: Candido Campos
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-20 11:13 UTC by Antoni Segura Puimedon
Modified: 2019-10-03 08:42 UTC (History)
14 users (show)

Fixed In Version: openstack-neutron-14.0.3-0.20190913160457.0f4f48c.el8ost
Doc Type: Technology Preview
Doc Text:
Neutron bulk port creation (create multiple ports in a single request) has been optimized for speed and is now significantly faster. The benefits of this improvement include faster initialization of containers via Kuryr on neutron networks.
Clone Of:
Environment:
Last Closed: 2019-10-03 08:42:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 510830 0 None MERGED net_utils: Add a random mac generator 2020-09-18 13:22:56 UTC
OpenStack gerrit 584061 0 None MERGED Incorporate capability for multiple mac generation 2020-09-18 13:22:53 UTC
OpenStack gerrit 585028 0 None MERGED Add bulk port creation of DB objects 2020-09-18 13:22:53 UTC
OpenStack gerrit 592164 0 None MERGED Add bulk IP address assignment to ipam driver 2020-09-18 13:22:56 UTC
OpenStack gerrit 613149 0 None MERGED Fix random_mac_generator to make proper EUI64s 2020-09-18 13:22:53 UTC
OpenStack gerrit 624815 0 None MERGED Utilize bulk port creation ops in ml2 plugin 2020-09-18 13:22:52 UTC
OpenStack gerrit 665516 0 None MERGED Fix bulk port binding 2020-09-18 13:22:52 UTC
OpenStack gerrit 679852 0 None MERGED Fix bulk port functioning with requested security groups 2020-09-18 13:22:52 UTC
OpenStack gerrit 681585 0 None MERGED Fix bulk port functioning with requested security groups 2020-09-18 13:22:52 UTC
Red Hat Product Errata RHBA-2019:2957 0 None None None 2019-10-03 08:42:29 UTC

Description Antoni Segura Puimedon 2017-10-20 11:13:44 UTC
Description of problem:

Neutron added bulk creation operations for certain resources like ports and subnets some time ago. This operations allow you to create N resources with a single API request.

Unfortunately, internally Neutron processes each of the N resources separately multiplying by N the amount of works that needs to be done in large stretches of the creation flow. All in all it makes it common for a 2000 port creation operation where all the requested ports are equal to take close to one hour.

This feature requests the code to be optimized so that if all the requested ports are equal, they be processed together and perform just x database transactions instead of xN transactions.

Steps to Reproduce:
1. source openrc
2. Create the following script

#!/usr/bin/env python
import sys

import os_client_config
from pprint import pprint


neutron = os_client_config.make_client('network', cloud='envvars')
nova = os_client_config.make_client('compute', cloud='envvars')

hypervisors = nova.hypervisors.list()

bulk_port_req = {'ports': []}
for hyper in hypervisors:
    if 'compute-0' in hyper.hypervisor_hostname:
        continue
    bulk_port_req['ports'].extend([{
        'binding:host_id': hyper.hypervisor_hostname,
        'project_id': sys.argv[2],
        'network_id': sys.argv[3],
        'device_owner': 'compute:kuryr',
        'admin_state_up': True,
        'name': 'kaboom',
        'security_groups': sys.argv[4].split(',')} for _ in range(int(sys.argv[1]))])

ports = neutron.create_port(bulk_port_req).get('ports')
pprint(ports)

3. chmod +x port_creator
4. Create the ports by doing:
./port_creator 2000 project_uuid network_uuid security_groups_comma_separated_uuids


Actual results:
The ports are created in x*N transactions as visible in the log.

Expected results:
The ports are created in x transactions as visible in the log.

Comment 10 Nate Johnston 2019-05-13 13:30:29 UTC
Final change 624815 merged upstream March 21st.

Comment 35 errata-xmlrpc 2019-10-03 08:42:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2957


Note You need to log in before you can comment on or make changes to this bug.