Bug 1899853 - additionalSecurityGroupIDs not working for master nodes
Summary: additionalSecurityGroupIDs not working for master nodes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.6
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.7.0
Assignee: Pierre Prinetti
QA Contact: weiwei jiang
URL:
Whiteboard:
Depends On:
Blocks: 1901736
TreeView+ depends on / blocked
 
Reported: 2020-11-20 09:01 UTC by Robert Heinzmann
Modified: 2021-02-24 15:35 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Control Plane ports were not assigned the additional user-defined Security groups Consequence: Additional user-defined Security group rules were not properly applied to Control plane nodes Fix: The additional user-defined Security groups are now assigned to the Control plane nodes ports Result: Additional user-defined security groups now correctly apply to Control plane nodes.
Clone Of:
Environment:
Last Closed: 2021-02-24 15:35:02 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4411 0 None closed Bug 1899853: openstack: CP nodes port to use addtional SGs 2021-01-13 10:55:41 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:35:31 UTC

Description Robert Heinzmann 2020-11-20 09:01:26 UTC
Version:

$ openshift-install version

[stack@osp16 test-additional-sg]$ ../openshift-install version
../openshift-install 4.6.3
built from commit a4f0869e0d2a5b2d645f0f28ef9e4b100fa8f779
release image 192.168.100.98:443/ocp4/openshift4@sha256:14986d2b9c112ca955aaa03f7157beadda0bd3c089e5e1d56f28020d2dd55c52

Platform:

OpenStack 16.1

Please specify:

IPI (automated install with `openshift-install`. If you don't know, then it's IPI)


What happened?

Configuring additionalSecurityGroupIDs for the masters does not work. Cluster is bootstrapped, however additional security groups are missing. 

~~~
controlPlane:
  hyperthreading: Enabled
  architecture: amd64
  name: master
  platform:
    openstack:
      type: openshift.master
      additionalSecurityGroupIDs:
      - 2959554a-8cca-4260-82bf-e0fcbb87f40c
  replicas: 3
~~~

The additional security group can not be found on the resulting servers

~~~
[stack@osp16 ocp-test1]$ openstack server show ocp-99l7h-master-0
+-----------------------------+----------------------------------------------------------+
| Field                       | Value                                                    |
+-----------------------------+----------------------------------------------------------+
| id                          | d6c5466a-b43a-47c2-84a4-41afe574d3f2                     |
| name                        | ocp-99l7h-master-0                                       |
| security_groups             | name='ocp-99l7h-master'                                  
~~~

I would suspect that https://github.com/openshift/installer/blob/release-4.6/data/data/openstack/topology/private-network.tf#L46 is missing the additional ID's that are configured here https://github.com/openshift/installer/blob/release-4.6/data/data/openstack/masters/main.tf#L49

Reproducer: http://pastebin.test.redhat.com/919822
Fixed Reproducer: http://pastebin.test.redhat.com/919824

It seems that port security group settings overrule machine settings.

What did you expect to happen?

Masters get additional security groups

How to reproduce it (as minimally and precisely as possible)?

Create additional security group
Configure additionalSecurityGroupIDs in the installconfig
Install IPI
verify the security groups

Comment 1 Pierre Prinetti 2020-11-25 11:24:29 UTC
A question from Martin on Github[1]:

> The patch fixes exactly the issue that is reported in the BZ so I do not see any reason not to approve.
> It would be nice to know if we also need to apply the additional security groups to the VIP ports. Otherwise
> what is the point of adding security groups at all to api_port and ingress_port?


@robert: do we cover the use case by just setting the additional SGs on the master ports, or should we go beyond that to deliver value for the customer?


[1]: https://github.com/openshift/installer/pull/4411#pullrequestreview-538216817

Comment 2 Maria Emmanuelli 2020-11-25 13:14:01 UTC
@pierre The SG should be set only to the master ports and not to the VIP ports. It should follow the same behaviour as with the worker nodes.

Comment 3 Pierre Prinetti 2020-11-25 13:27:45 UTC
Perfect, thanks.

Comment 5 weiwei jiang 2020-11-26 07:12:39 UTC
Checked with latest
./openshift-install 4.7.0-0.nightly-2020-11-25-114114
built from commit a9e6c4d8fa0e7d5edb9cf95330689a65261ff09c
release image registry.svc.ci.openshift.org/ocp/release@sha256:bf37e13af0e254d0b744b62ace0dcf5560230374d7877a8fde16cf9134ec7862

---
apiVersion: v1
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  platform:
    openstack:
      additionalSecurityGroupIDs: &1
      - 8794f45c-4f54-40a4-aadb-38d6c32e286e
  replicas: 3
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform:
    openstack:
      additionalSecurityGroupIDs: *1
      type: m1.large
  replicas: 3
metadata:
  name: wj47ios1126y
platform:
  openstack:
    cloud: openstack
    computeFlavor: m1.xlarge
    region: regionOne
    trunkSupport: '1'
    octaviaSupport: '0'
    lbFloatingIP: 10.0.102.125
    ingressFloatingIP: 10.0.103.227
    externalNetwork: provider_net_cci_8
pullSecret: HIDDEN
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  serviceNetwork:
  - 172.30.0.0/16
  machineNetwork:
  - cidr: 192.168.0.0/18
  networkType: OpenShiftSDN
publish: External
baseDomain: 1126-7gp.qe.rhcloud.com
sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa37P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D
  openshift-qe@redhat.com

and the control-plane still not take affect, need wait new payload to have a try.

# openstack server show wj47ios1126y-xb8px-master-2
+-----------------------------+---------------------------------------------------------------------------+
| Field                       | Value                                                                     |
+-----------------------------+---------------------------------------------------------------------------+
| OS-DCF:diskConfig           | MANUAL                                                                    |
| OS-EXT-AZ:availability_zone | nova                                                                      |
| OS-EXT-STS:power_state      | Running                                                                   |
| OS-EXT-STS:task_state       | None                                                                      |
| OS-EXT-STS:vm_state         | active                                                                    |
| OS-SRV-USG:launched_at      | 2020-11-26T06:24:13.000000                                                |
| OS-SRV-USG:terminated_at    | None                                                                      |
| accessIPv4                  |                                                                           |
| accessIPv6                  |                                                                           |
| addresses                   | wj47ios1126y-xb8px-openshift=192.168.2.137                                |
| config_drive                |                                                                           |
| created                     | 2020-11-26T06:23:21Z                                                      |
| flavor                      | m1.xlarge (3f183920-6cba-4bfb-ab3a-599559cf0f97)                          |
| hostId                      | fa75360746cc026112b3802a27c06948313b684d2f736724e3482ad4                  |
| id                          | 4ed158a9-cd4f-440a-85ba-5c55cf0a40d9                                      |
| image                       | wj47ios1126y-xb8px-rhcos (64a046b8-0f3b-44d9-b51d-83dcbe9cb928)           |
| key_name                    | None                                                                      |
| name                        | wj47ios1126y-xb8px-master-2                                               |
| progress                    | 0                                                                         |
| project_id                  | 542c6ebd48bf40fa857fc245c7572e30                                          |
| properties                  | Name='wj47ios1126y-xb8px-master', openshiftClusterID='wj47ios1126y-xb8px' |
| security_groups             | name='wj47ios1126y-xb8px-master'                                          |
| status                      | ACTIVE                                                                    |
| updated                     | 2020-11-26T06:24:13Z                                                      |
| user_id                     | b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1          |
| volumes_attached            |                                                                           |
+-----------------------------+---------------------------------------------------------------------------+
[root@wjiang-bind-bastion ~]# openstack server show wj47ios1126y-xb8px-worker-0-dmvtk
+-----------------------------+---------------------------------------------------------------------------+
| Field                       | Value                                                                     |
+-----------------------------+---------------------------------------------------------------------------+
| OS-DCF:diskConfig           | MANUAL                                                                    |
| OS-EXT-AZ:availability_zone | nova                                                                      |
| OS-EXT-STS:power_state      | Running                                                                   |
| OS-EXT-STS:task_state       | None                                                                      |
| OS-EXT-STS:vm_state         | active                                                                    |
| OS-SRV-USG:launched_at      | 2020-11-26T06:50:50.000000                                                |
| OS-SRV-USG:terminated_at    | None                                                                      |
| accessIPv4                  |                                                                           |
| accessIPv6                  |                                                                           |
| addresses                   | wj47ios1126y-xb8px-openshift=192.168.3.154                                |
| config_drive                |                                                                           |
| created                     | 2020-11-26T06:50:26Z                                                      |
| flavor                      | m1.large (a9acc2de-39d7-4148-8d16-413c3b696e9d)                           |
| hostId                      | fa75360746cc026112b3802a27c06948313b684d2f736724e3482ad4                  |
| id                          | c5949b79-a8a9-4cd0-a0e0-6fe37f567270                                      |
| image                       | wj47ios1126y-xb8px-rhcos (64a046b8-0f3b-44d9-b51d-83dcbe9cb928)           |
| key_name                    | None                                                                      |
| name                        | wj47ios1126y-xb8px-worker-0-dmvtk                                         |
| progress                    | 0                                                                         |
| project_id                  | 542c6ebd48bf40fa857fc245c7572e30                                          |
| properties                  | Name='wj47ios1126y-xb8px-worker', openshiftClusterID='wj47ios1126y-xb8px' |
| security_groups             | name='wj47ios1126y-xb8px-worker'                                          |
|                             | name='default'                                                            |
| status                      | ACTIVE                                                                    |
| updated                     | 2020-11-26T06:50:50Z                                                      |
| user_id                     | b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1          |
| volumes_attached            |                                                                           |
+-----------------------------+---------------------------------------------------------------------------+

Comment 6 weiwei jiang 2020-11-27 02:22:46 UTC
Checked with latest payload
./openshift-install 4.7.0-0.nightly-2020-11-26-221840
built from commit 64ec239bc596635b50dd82485c9932cdf10c861e
release image registry.svc.ci.openshift.org/ocp/release@sha256:542e9447623e5e5f0ba96be505d695b81b7b0b088452a19d66b0c4f1e0f6654b

with install-config.yaml:
---
apiVersion: v1
controlPlane:
  architecture: amd64
  hyperthreading: Enabled
  name: master
  platform:
    openstack:
      additionalSecurityGroupIDs: &1
      - 8794f45c-4f54-40a4-aadb-38d6c32e286e
  replicas: 3
compute:
- architecture: amd64
  hyperthreading: Enabled
  name: worker
  platform:
    openstack:
      additionalSecurityGroupIDs: *1
      type: m1.large
  replicas: 3
metadata:
  name: wj47ios1127z
platform:
  openstack:
    cloud: openstack
    computeFlavor: m1.xlarge
    region: regionOne
    trunkSupport: '1'
    octaviaSupport: '0'
    lbFloatingIP: 10.0.103.36
    ingressFloatingIP: 10.0.103.31
    externalNetwork: provider_net_cci_8
pullSecret: HIDDEN
networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  serviceNetwork:
  - 172.30.0.0/16
  machineNetwork:
  - cidr: 192.168.0.0/18
  networkType: OpenShiftSDN
publish: External
baseDomain: 1127-ggc.qe.rhcloud.com
sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa37P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D
  openshift-qe@redhat.com


# openstack server list  --name wj47ios1127z-9mdcj -f value -c ID | xargs -n 1 openstack server show -f json|jq -r '"===========\n"+"Server: "+ .name, "Security Groups:\n" + .security_groups'                                                    
===========
Server: wj47ios1127z-9mdcj-worker-0-l8n4w
Security Groups:
name='wj47ios1127z-9mdcj-worker'
name='default'
===========
Server: wj47ios1127z-9mdcj-worker-0-kb2wj
Security Groups:
name='wj47ios1127z-9mdcj-worker'
name='default'
===========
Server: wj47ios1127z-9mdcj-worker-0-dx2v9
Security Groups:
name='wj47ios1127z-9mdcj-worker'
name='default'
===========
Server: wj47ios1127z-9mdcj-master-2
Security Groups:
name='default'
name='wj47ios1127z-9mdcj-master'
===========
Server: wj47ios1127z-9mdcj-master-1
Security Groups:
name='default'
name='wj47ios1127z-9mdcj-master'
===========
Server: wj47ios1127z-9mdcj-master-0
Security Groups:
name='default'
name='wj47ios1127z-9mdcj-master'

Comment 8 milti leonard 2020-12-07 18:23:43 UTC
is there anything further that needs to be done on this? also, can this be back-ported to 4.5/4.6?

thnx, m

Comment 9 Pierre Prinetti 2020-12-07 20:08:42 UTC
(In reply to milti leonard from comment #8)
> is there anything further that needs to be done on this? also, can this be
> back-ported to 4.5/4.6?
> 
> thnx, m

The 4.6 patch for is waiting for the release-manager's approval https://github.com/openshift/installer/pull/4420

Comment 14 errata-xmlrpc 2021-02-24 15:35:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.