Bug 1926145

+++ This bug was initially created as a clone of Bug #1920318 +++

Description of problem:

- The neutron call back hook is making sure to create a default security group rule every time during creation of a network even if enable_security_group is set False.
- so if OSP is deployed with enable_security_group = False, default security group rule is never created, and network creation will fail forever

Please check the error and logic below.
This log is printed while creating a network.
Below is the test result.

Workaround is
- manually populating database with deploying osp with "enable_security_group=true", and creating and deleting a network.
- And then disable 'enable_security_group'

- config --
[securitygroup]
enable_security_group = False

- command --
openstack network create mytestnet123

- resp --
RESP BODY: {"NeutronError": {"type": "QuotaResourceUnknown", "message": "Unknown quota resources ['security_group_rule'].", "detail": ""}}

- log of  Unknown quota resources ['security_group_rule'] --
2021-01-24 19:19:38.187 27 DEBUG neutron_lib.callbacks.manager [req-16cf2bb3-f5f2-4e42-9acf-065a58e6ace4 990d542b08b74375b699aab524349af5 5a8645766a4541449c521ab52bfd290e - default default] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler-1179307'] for network, before_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2021-01-24 19:19:38.202 27 DEBUG neutron_lib.callbacks.manager [req-16cf2bb3-f5f2-4e42-9acf-065a58e6ace4 990d542b08b74375b699aab524349af5 5a8645766a4541449c521ab52bfd290e - default default] Notify callbacks [] for security_group, before_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193
2021-01-24 19:19:38.242 27 DEBUG neutron_lib.callbacks.manager [req-16cf2bb3-f5f2-4e42-9acf-065a58e6ace4 990d542b08b74375b699aab524349af5 5a8645766a4541449c521ab52bfd290e - default default] Callback neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler-1179307 raised Unknown quota resources ['security_group_rule']. _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:210

- logic --
    @registry.receives(resources.NETWORK, [events.BEFORE_CREATE])
    def _ensure_default_security_group_handler(self, resource, event, trigger,
                                               context, **kwargs):
        if event == events.BEFORE_UPDATE:
            tenant_id = kwargs['original_' + resource]['tenant_id']
        else:
            tenant_id = kwargs[resource]['tenant_id']
        if tenant_id:
            self._ensure_default_security_group(context, tenant_id)

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Donghwi Cha on 2021-01-26 03:43:00 UTC ---

Hi, Sadique. 

Thanks for creating a report on behalf of me. 

For the workaround that I provided above,
 
it is not exactly a kind of a workaround as in solution 
because the default security group is bound its project(tenant).
 
So whenever there is new project created, the same issue will happen, 
and switching on/off the flag by restarting service to fill the database would not be practical way of workaround. 

If you have OSP deployed already, you can reproduce the issue by this step
- create a new project
- make sure there is no default security group rule 
- disable the port security on ml2 ini file (enable_security_group = False)
- create a network (tenant network is fine for this testing)
- error -> RESP BODY: {"NeutronError": {"type": "QuotaResourceUnknown", "message": "Unknown quota resources ['security_group_rule'].", "detail": ""}}

--- Additional comment from Slawek Kaplonski on 2021-01-26 13:03:42 UTC ---

Thx for the bug report. I was able to reproduce it with ML2/OVN backend.
There is no such issue when e.g. OVS is used.

--- Additional comment from Slawek Kaplonski on 2021-01-26 13:54:29 UTC ---

Small correction: the same issue is for ml2/ovs also

--- Additional comment from Donghwi Cha on 2021-01-26 16:20:09 UTC ---

Hi Slawek.

Yes, this issue is effective for both ovs and ovn as far as I can see on the Neutron server logic. 

I just had a quick view on the middleware logic of Neutron,
hope the note below can be helpful. 

So the train version Neutron is relying on its own custom callback library, neutron_lib.callbacks

When the neutron pecan app is instantiated, 
list of hooks will be included to its makeapp call. 

pecan_wsgi/app.py

        hooks.UserFilterHook(),  # priority 90
        hooks.ContextHook(),  # priority 95
        hooks.ExceptionTranslationHook(),  # priority 100
        hooks.BodyValidationHook(),  # priority 120
        hooks.OwnershipValidationHook(),  # priority 125
        hooks.QuotaEnforcementHook(),  # priority 130
        hooks.NotifierHook(),  # priority 135
        hooks.QueryParametersHook(),  # priority 139
        hooks.PolicyHook(),  # priority 140

and the NotifierHook above has registry.publish on one of its function "def after", 
where the message for callback is registered by neutron_lib.callbacks's method, registry.publish. 

So whenever there is api request on the resource of network(create) or port(create, update), 
the handler function will be called in the background as callback, by registry.receives,
and that is when is why function ensure_default_security_group_handler is called every time on the creation of the network. 

Currently enable_security_group is only used by neutron agent on its rpc call, 
IMO, probably we can load this config on neutron server side, 
and add one more flag to the handler to skip default sg creation.