Bug 1431370 - [RFE] Ability to select OpenStack External external network during the instance provisioning
Summary: [RFE] Ability to select OpenStack External external network during the instan...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: GA
: 5.9.0
Assignee: Ladislav Smola
QA Contact: Ola Pavlenko
URL:
Whiteboard: openstack
: 1507509 (view as bug list)
Depends On:
Blocks: 1551709 1552233
TreeView+ depends on / blocked
 
Reported: 2017-03-11 17:29 UTC by Saif Ali
Modified: 2018-03-08 16:22 UTC (History)
13 users (show)

Fixed In Version: 5.9.0.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1552233 (view as bug list)
Environment:
Last Closed: 2018-03-01 13:10:28 UTC
Category: Feature
Cloudforms Team: Openstack


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:0380 normal SHIPPED_LIVE Moderate: Red Hat CloudForms security, bug fix, and enhancement update 2018-03-01 18:37:12 UTC

Description Saif Ali 2017-03-11 17:29:30 UTC
Description of problem:
Ability to select OpenStack External external network during the instance provisioning

Version-Release number of selected component (if applicable):
4.2

How reproducible:
1: Create external shared network on the admin tenant in OSP
2: Add OSP provider using different tenant e.g CFME tenant. Enable tenant mapping.
 
Actual results:
try to provision instance in CFME tenant
the result no option to select the external shared network.


Expected results:
I should be able to select the external shared network from the network list

Additional info:

Comment 3 Ladislav Smola 2017-03-13 16:09:24 UTC
Hello,

network with a flag "router:external" should be used only with a router. 

Look at the docs for the provider networks, if you want a public network that you can attach directly to the VMs, e.g:
https://docs.openstack.org/liberty/networking-guide/scenario-provider-ovs.html#create-initial-networks
https://docs.openstack.org/mitaka/install-guide-ubuntu/launch-instance-networks-provider.html#create-the-provider-network

notice the "router:external           | False"

Then you should be able to attach a provider network from the CFME without problems.

The Horizon implementation that allows attaching router:external network directly, is a bug in my opinion. Since if you attach a router:external=True network directly to the Vm, it will not work (when it's not a provider network of course). So Horizon should not allow plugging of an external network that will not give you the external connectivity.

The router:external network only usage should be, that it holds the floating IPs. Then in order to attach the floating IP to an interface, the interface needs to be connected to a subnet, which is connected to a router(router interfaces) which is connected to an external network(router gateway). You can assign floating IPs only from the connected external networks, otherwise the NAT will not work.

Kind Regards,
Ladislav

Comment 11 Andreas Karis 2017-04-26 17:17:18 UTC
Hello,

In OpenStack, it is perfectly valid to attach a VM to a network with 'router:external' set.

It is perfectly valid to create a public network with 'router:external' set, then attach a router to it, and have a private network attached to that router. 

Then, you can boot instances on the private network and get them floating IPs, and on the public network as well without floating IPs. 

If cloudforms does not allow this, then this is a bug in cloudforms.

~~~
[stack@undercloud-6 ~]$ neutron net-list
+--------------------------------------+-----------+----------------------------------------------------------+
| id                                   | name      | subnets                                                  |
+--------------------------------------+-----------+----------------------------------------------------------+
| e41479c0-946d-4ff1-a26a-ca6b0248a60c | provider1 | 3c19285b-d21a-48c4-bd52-4a0d608dedf5 2000:10::/64        |
|                                      |           | 2ac48199-86e3-4b89-85c0-3d9e4f1548d6 10.0.0.0/24         |
| 9bf6120b-03d0-4907-a03a-68bad4770da8 | private   | 5aaa44a4-d2d3-4555-a5f9-0ed3dcdf1191 2000:192:168:1::/64 |
|                                      |           | db2b1070-d173-43ed-9e5a-93f872740435 192.168.0.0/24      |
+--------------------------------------+-----------+----------------------------------------------------------+
[stack@undercloud-6 ~]$ neutron net-show e41479c0-946d-4ff1-a26a-ca6b0248a60c
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2017-04-26T16:51:49                  |
| description               |                                      |
| id                        | e41479c0-946d-4ff1-a26a-ca6b0248a60c |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 1500                                 |
| name                      | provider1                            |
| port_security_enabled     | True                                 |
| provider:network_type     | vlan                                 |
| provider:physical_network | datacentre                           |
| provider:segmentation_id  | 905                                  |
| qos_policy_id             |                                      |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | 3c19285b-d21a-48c4-bd52-4a0d608dedf5 |
|                           | 2ac48199-86e3-4b89-85c0-3d9e4f1548d6 |
| tags                      |                                      |
| tenant_id                 | cb0d5805d72a4f2a89ff260b15629799     |
| updated_at                | 2017-04-26T16:51:49                  |
+---------------------------+--------------------------------------+
[stack@undercloud-6 ~]$ neutron net-show 9bf6120b-03d0-4907-a03a-68bad4770da8
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2017-04-26T16:51:46                  |
| description               |                                      |
| id                        | 9bf6120b-03d0-4907-a03a-68bad4770da8 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| mtu                       | 1450                                 |
| name                      | private                              |
| port_security_enabled     | True                                 |
| provider:network_type     | vxlan                                |
| provider:physical_network |                                      |
| provider:segmentation_id  | 51                                   |
| qos_policy_id             |                                      |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   | db2b1070-d173-43ed-9e5a-93f872740435 |
|                           | 5aaa44a4-d2d3-4555-a5f9-0ed3dcdf1191 |
| tags                      |                                      |
| tenant_id                 | cb0d5805d72a4f2a89ff260b15629799     |
| updated_at                | 2017-04-26T16:51:46                  |
+---------------------------+--------------------------------------+
[stack@undercloud-6 ~]$ nova list
+--------------------------------------+--------------+--------+------------+-------------+---------------------------------------------------------------------+
| ID                                   | Name         | Status | Task State | Power State | Networks                                                            |
+--------------------------------------+--------------+--------+------------+-------------+---------------------------------------------------------------------+
| b5ae2558-506f-4cce-9432-4d0cbc37dd84 | cirros-test1 | ACTIVE | -          | Running     | private=2000:192:168:1:f816:3eff:fe39:ce73, 192.168.0.3, 10.0.0.103 |
| 2de1324f-a91d-4eff-b68c-11c8ae578533 | cirros-test2 | ACTIVE | -          | Running     | provider1=2000:10::f816:3eff:fe90:fa39, 10.0.0.105                  |
+--------------------------------------+--------------+--------+------------+-------------+---------------------------------------------------------------------+
[stack@undercloud-6 ~]$ ping 10.0.0.103
PING 10.0.0.103 (10.0.0.103) 56(84) bytes of data.
64 bytes from 10.0.0.103: icmp_seq=1 ttl=63 time=2.09 ms
^C
--- 10.0.0.103 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.092/2.092/2.092/0.000 ms
[stack@undercloud-6 ~]$ ping 10.0.0.105
PING 10.0.0.105 (10.0.0.105) 56(84) bytes of data.
64 bytes from 10.0.0.105: icmp_seq=1 ttl=64 time=0.770 ms
^C
--- 10.0.0.105 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.770/0.770/0.770/0.000 ms
[stack@undercloud-6 ~]$ 
[stack@undercloud-6 ~]$ nova show b5ae2558-506f-4cce-9432-4d0cbc37dd84 | grep network
nova show | private network                      | 2000:192:168:1:f816:3eff:fe39:ce73, 192.168.0.3, 10.0.0.103 |
[stack@undercloud-6 ~]$ nova show 2de1324f-a91d-4eff-b68c-11c8ae578533 | grep network
| provider1 network                    | 2000:10::f816:3eff:fe90:fa39, 10.0.0.105                 |
~~~

Why would this even be invalid?

FWIW, I tested changing the network type:
~~~
[stack@undercloud-6 ~]$ neutron net-update --router:external=false e41479c0-946d-4ff1-a26a-ca6b0248a60c
Updated network: e41479c0-946d-4ff1-a26a-ca6b0248a60c
[stack@undercloud-6 ~]$ neutron net-show e41479c0-946d-4ff1-a26a-ca6b0248a60c
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2017-04-26T16:51:49                  |
| description               |                                      |
| id                        | e41479c0-946d-4ff1-a26a-ca6b0248a60c |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| mtu                       | 1500                                 |
| name                      | provider1                            |
| port_security_enabled     | True                                 |
| provider:network_type     | vlan                                 |
| provider:physical_network | datacentre                           |
| provider:segmentation_id  | 905                                  |
| qos_policy_id             |                                      |
| router:external           | False                                |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | 3c19285b-d21a-48c4-bd52-4a0d608dedf5 |
|                           | 2ac48199-86e3-4b89-85c0-3d9e4f1548d6 |
| tags                      |                                      |
| tenant_id                 | cb0d5805d72a4f2a89ff260b15629799     |
| updated_at                | 2017-04-26T16:51:49                  |
+---------------------------+--------------------------------------+
[stack@undercloud-6 ~]$ neutron net-list | grep e41479c0-946d-4ff1-a26a-ca6b0248a60c -A1
| e41479c0-946d-4ff1-a26a-ca6b0248a60c | provider1 | 3c19285b-d21a-48c4-bd52-4a0d608dedf5 2000:10::/64        |
|                                      |           | 2ac48199-86e3-4b89-85c0-3d9e4f1548d6 10.0.0.0/24         |
~~~

At the same time, I ran a ping to an instance on that network (from the directly attached gateway, 10.0.0.1!) - note that I had no packet loss:
~~~
[stack@undercloud-6 ~]$ ping 10.0.0.105 -i 0.2
PING 10.0.0.105 (10.0.0.105) 56(84) bytes of data.
64 bytes from 10.0.0.105: icmp_seq=1 ttl=64 time=1.08 ms
64 bytes from 10.0.0.105: icmp_seq=2 ttl=64 time=0.893 ms
64 bytes from 10.0.0.105: icmp_seq=3 ttl=64 time=0.843 ms
64 bytes from 10.0.0.105: icmp_seq=4 ttl=64 time=1.12 ms
64 bytes from 10.0.0.105: icmp_seq=5 ttl=64 time=0.880 ms
64 bytes from 10.0.0.105: icmp_seq=6 ttl=64 time=0.814 ms
64 bytes from 10.0.0.105: icmp_seq=7 ttl=64 time=0.915 ms
64 bytes from 10.0.0.105: icmp_seq=8 ttl=64 time=0.703 ms
64 bytes from 10.0.0.105: icmp_seq=9 ttl=64 time=0.807 ms
64 bytes from 10.0.0.105: icmp_seq=10 ttl=64 time=0.732 ms
64 bytes from 10.0.0.105: icmp_seq=11 ttl=64 time=0.666 ms
64 bytes from 10.0.0.105: icmp_seq=12 ttl=64 time=0.683 ms
64 bytes from 10.0.0.105: icmp_seq=13 ttl=64 time=0.635 ms
64 bytes from 10.0.0.105: icmp_seq=14 ttl=64 time=0.608 ms
64 bytes from 10.0.0.105: icmp_seq=15 ttl=64 time=0.925 ms
64 bytes from 10.0.0.105: icmp_seq=16 ttl=64 time=0.690 ms
64 bytes from 10.0.0.105: icmp_seq=17 ttl=64 time=0.582 ms
64 bytes from 10.0.0.105: icmp_seq=18 ttl=64 time=0.662 ms
64 bytes from 10.0.0.105: icmp_seq=19 ttl=64 time=0.579 ms
64 bytes from 10.0.0.105: icmp_seq=20 ttl=64 time=0.556 ms
64 bytes from 10.0.0.105: icmp_seq=21 ttl=64 time=0.613 ms
64 bytes from 10.0.0.105: icmp_seq=22 ttl=64 time=0.602 ms
64 bytes from 10.0.0.105: icmp_seq=23 ttl=64 time=0.695 ms
64 bytes from 10.0.0.105: icmp_seq=24 ttl=64 time=0.708 ms
64 bytes from 10.0.0.105: icmp_seq=25 ttl=64 time=0.759 ms
64 bytes from 10.0.0.105: icmp_seq=26 ttl=64 time=0.734 ms
64 bytes from 10.0.0.105: icmp_seq=27 ttl=64 time=0.706 ms
64 bytes from 10.0.0.105: icmp_seq=28 ttl=64 time=0.727 ms
64 bytes from 10.0.0.105: icmp_seq=29 ttl=64 time=0.568 ms
64 bytes from 10.0.0.105: icmp_seq=30 ttl=64 time=0.681 ms
64 bytes from 10.0.0.105: icmp_seq=31 ttl=64 time=0.570 ms
64 bytes from 10.0.0.105: icmp_seq=32 ttl=64 time=0.821 ms
64 bytes from 10.0.0.105: icmp_seq=33 ttl=64 time=0.633 ms
^C
--- 10.0.0.105 ping statistics ---
33 packets transmitted, 33 received, 0% packet loss, time 6406ms
rtt min/avg/max/mdev = 0.556/0.733/1.129/0.143 ms
[stack@undercloud-6 ~]$ ip r g 10.0.0.105
10.0.0.105 dev eth1.905  src 10.0.0.1 
    cache 
~~~

I didn't test this any further, but this should only be a flag in the database, AFAICT.

Comment 12 Andreas Karis 2017-04-26 17:25:07 UTC
"Since if you attach a router:external=True network directly to the Vm, it will not work (when it's not a provider network of course). So Horizon should not allow plugging of an external network that will not give you the external connectivity."

==> in neutron, there is no such differentiation between a tenant network and a provider network. I know that people like to use the terms, but the differentiation is only happening in people's head for logical reasons -the only values that an admin (or privileged user) can modify are:
~~~
 provider:network_type     | vlan                                 |
| provider:physical_network | datacentre                           |
| provider:segmentation_id  | 905    
~~~
Note that they all contain the term 'provider:'.
The parameter 'tenant_network_types' (and its name) only stem from the fact that these are the allowed network types for non-admin tenant users.

"The router:external network only usage should be, that it holds the floating IPs. Then in order to attach the floating IP to an interface, the interface needs to be connected to a subnet, which is connected to a router(router interfaces) which is connected to an external network(router gateway). You can assign floating IPs only from the connected external networks, otherwise the NAT will not work."
No. The router:external network is one that can be used as a gateway of a router. This does mean that it can hold floating IPs, but it does not need to (not everything is NATted, especially not in IPv6). Also, as shown above, there are use cases where a subnet can be router:external, because a router is attached to it, and still VMs need to be booted directly on this network.

Comment 13 Ladislav Smola 2017-04-27 10:00:11 UTC
You are right that a router:external is just a flag, but it has an exact meaning, please check the documentation: https://developer.openstack.org/api-ref/networking/v2/?expanded=show-network-details-detail#show-network-details

"router:external": "Indicates whether this network can provide floating IPs via a router."

So it should not be used in a case, where there is no router in play. From what I see, this tag is being wrongly used for marking any external network. But seems like this practice is so spread, it almost makes the documentation sound wrong.

----

That being said, we did an internal test just now and it's possible to use provider network with router:external=True. Testing connectivity through router and directly attached. So it seems we should indeed allow router:external networks to be attached directly. (Going through OpenStack forums, this doesn't sound like a good practice also because tenants should be allowed to use only tenant assigned Floating IPs, this way, there is no limitation)

For IPv6, there is no NAT and Floating IP, but the "router:external" can be relevant when using dual stack IPv6/4 network router:external network.

Comment 18 Andreas Karis 2017-04-27 14:29:22 UTC
You are right that a router:external is just a flag, but it has an exact meaning, please check the documentation: https://developer.openstack.org/api-ref/networking/v2/?expanded=show-network-details-detail#show-network-details

"router:external": "Indicates whether this network can provide floating IPs via a router."

> You are confusing the words `can` and `must`. I laid out an example above where a network indeed does provide floating IPs, and also has instances running on them. So the exact meaning of `router:external` is that it **can** have floating-ips ( but it does not need to do so exclusively).

===

"this doesn't sound like a good practice also because tenants should be allowed to use only tenant assigned Floating IP"  

> OSP has the `shared` flag to allow or deny exactly this behavior: " 	Indicates whether this network is shared across all tenants. By default, only administrative users can change this value." 

Another question is of course if cloudforms wants to support what's available in OSP, or if it additionally wants to be a watchdog for good practices :-)

Comment 19 Ladislav Smola 2017-04-27 15:04:10 UTC
> You are confusing the words `can` and `must`. I laid out an example above where a network indeed does provide floating IPs, and also has instances running on them. So the exact meaning of `router:external` is that it **can** have floating-ips ( but it does not need to do so exclusively).

Well, that is debatable. :-) If there is a Neutron router and the network is set as a gateway, then I would say it **can** provide a Floating IP via a Router (your example doesn't show a router). Otherwise the **can** would apply to any network(except IPv6), since you can reconfigure any network to do anything, so it wouldn't make much sense to even have that tag. :-)

====

> Another question is of course if cloudforms wants to support what's available in OSP, or if it additionally wants to be a watchdog for good practices

Well we are trying to make the default provisioning simple, so adding extra validation, that tries to ensure provisioning doesn't fail is a part of that. This one was done because with the default OS setup (one big shared public network) ~1.5 years ago, the VM didn't get the connectivity, when connecting directly and OpenStack Neutron response out there was mainly, that this doesn't make any sense. With time, things change, so we change the defaults. :-)

But since this is CF, you can create a Vm provisioning service, that will offer you networks (or anything) you need, with custom validations, matching the needed use case. So it's important to say, CF supports what's available in OSP at any time, just the defaults can vary. :-)

Comment 20 CFME Bot 2017-05-04 12:46:31 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/70edcc40e631a9ef7ed2aaf79d09baf21e4b5726

commit 70edcc40e631a9ef7ed2aaf79d09baf21e4b5726
Author:     Ladislav Smola <lsmola@redhat.com>
AuthorDate: Thu Apr 27 12:43:22 2017 +0200
Commit:     Ladislav Smola <lsmola@redhat.com>
CommitDate: Thu Apr 27 12:45:39 2017 +0200

    Move public/external network method into base class
    
    Move public/external network method into base class, since
    external network can be attached directly to a Vm, under some
    circumstances.
    
    Fixes BZ:
    https://bugzilla.redhat.com/show_bug.cgi?id=1431370

 app/models/cloud_network.rb                    | 4 ++++
 app/models/mixins/has_network_manager_mixin.rb | 1 -
 2 files changed, 4 insertions(+), 1 deletion(-)

Comment 21 CFME Bot 2017-05-04 13:44:44 UTC
New commit detected on ManageIQ/manageiq-providers-openstack/master:
https://github.com/ManageIQ/manageiq-providers-openstack/commit/57114d7cab118b38d32f44e35d529210edba88e1

commit 57114d7cab118b38d32f44e35d529210edba88e1
Author:     Ladislav Smola <lsmola@redhat.com>
AuthorDate: Thu Apr 27 12:47:08 2017 +0200
Commit:     Ladislav Smola <lsmola@redhat.com>
CommitDate: Thu Apr 27 15:36:44 2017 +0200

    Allow all networks to be attached directly to a VM
    
    Allow all networks to be attached directly to a VM
    
    Fixes BZ:
    https://bugzilla.redhat.com/show_bug.cgi?id=1431370

 .../providers/openstack/cloud_manager/cloud_tenant.rb |  4 ----
 .../openstack/cloud_manager/provision_workflow.rb     |  9 ---------
 .../manageiq/providers/openstack/network_manager.rb   |  1 -
 .../openstack/network_manager/cloud_network/public.rb |  3 ---
 .../cloud_manager/provision_workflow_spec.rb          | 18 +++++++++++-------
 .../openstack/cloud_manager/refresh_spec_common.rb    | 19 ++++++++++++-------
 6 files changed, 23 insertions(+), 31 deletions(-)

Comment 24 errata-xmlrpc 2018-03-01 13:10:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0380

Comment 26 Satoe Imaishi 2018-03-08 16:22:59 UTC
*** Bug 1507509 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.