Bug 1899853
| Summary: | additionalSecurityGroupIDs not working for master nodes | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Robert Heinzmann <rheinzma> |
| Component: | Installer | Assignee: | Pierre Prinetti <pprinett> |
| Installer sub component: | OpenShift on OpenStack | QA Contact: | weiwei jiang <wjiang> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | CC: | maemmanu, mleonard, pprinett |
| Version: | 4.6 | Keywords: | UpcomingSprint |
| Target Milestone: | --- | ||
| Target Release: | 4.7.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Control Plane ports were not assigned the additional user-defined Security groups
Consequence: Additional user-defined Security group rules were not properly applied to Control plane nodes
Fix: The additional user-defined Security groups are now assigned to the Control plane nodes ports
Result: Additional user-defined security groups now correctly apply to Control plane nodes.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-02-24 15:35:02 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1901736 | ||
A question from Martin on Github[1]: > The patch fixes exactly the issue that is reported in the BZ so I do not see any reason not to approve. > It would be nice to know if we also need to apply the additional security groups to the VIP ports. Otherwise > what is the point of adding security groups at all to api_port and ingress_port? @robert: do we cover the use case by just setting the additional SGs on the master ports, or should we go beyond that to deliver value for the customer? [1]: https://github.com/openshift/installer/pull/4411#pullrequestreview-538216817 @pierre The SG should be set only to the master ports and not to the VIP ports. It should follow the same behaviour as with the worker nodes. Perfect, thanks. Checked with latest
./openshift-install 4.7.0-0.nightly-2020-11-25-114114
built from commit a9e6c4d8fa0e7d5edb9cf95330689a65261ff09c
release image registry.svc.ci.openshift.org/ocp/release@sha256:bf37e13af0e254d0b744b62ace0dcf5560230374d7877a8fde16cf9134ec7862
---
apiVersion: v1
controlPlane:
architecture: amd64
hyperthreading: Enabled
name: master
platform:
openstack:
additionalSecurityGroupIDs: &1
- 8794f45c-4f54-40a4-aadb-38d6c32e286e
replicas: 3
compute:
- architecture: amd64
hyperthreading: Enabled
name: worker
platform:
openstack:
additionalSecurityGroupIDs: *1
type: m1.large
replicas: 3
metadata:
name: wj47ios1126y
platform:
openstack:
cloud: openstack
computeFlavor: m1.xlarge
region: regionOne
trunkSupport: '1'
octaviaSupport: '0'
lbFloatingIP: 10.0.102.125
ingressFloatingIP: 10.0.103.227
externalNetwork: provider_net_cci_8
pullSecret: HIDDEN
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
serviceNetwork:
- 172.30.0.0/16
machineNetwork:
- cidr: 192.168.0.0/18
networkType: OpenShiftSDN
publish: External
baseDomain: 1126-7gp.qe.rhcloud.com
sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa37P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D
openshift-qe
and the control-plane still not take affect, need wait new payload to have a try.
# openstack server show wj47ios1126y-xb8px-master-2
+-----------------------------+---------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+---------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2020-11-26T06:24:13.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | wj47ios1126y-xb8px-openshift=192.168.2.137 |
| config_drive | |
| created | 2020-11-26T06:23:21Z |
| flavor | m1.xlarge (3f183920-6cba-4bfb-ab3a-599559cf0f97) |
| hostId | fa75360746cc026112b3802a27c06948313b684d2f736724e3482ad4 |
| id | 4ed158a9-cd4f-440a-85ba-5c55cf0a40d9 |
| image | wj47ios1126y-xb8px-rhcos (64a046b8-0f3b-44d9-b51d-83dcbe9cb928) |
| key_name | None |
| name | wj47ios1126y-xb8px-master-2 |
| progress | 0 |
| project_id | 542c6ebd48bf40fa857fc245c7572e30 |
| properties | Name='wj47ios1126y-xb8px-master', openshiftClusterID='wj47ios1126y-xb8px' |
| security_groups | name='wj47ios1126y-xb8px-master' |
| status | ACTIVE |
| updated | 2020-11-26T06:24:13Z |
| user_id | b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1 |
| volumes_attached | |
+-----------------------------+---------------------------------------------------------------------------+
[root@wjiang-bind-bastion ~]# openstack server show wj47ios1126y-xb8px-worker-0-dmvtk
+-----------------------------+---------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+---------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2020-11-26T06:50:50.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | wj47ios1126y-xb8px-openshift=192.168.3.154 |
| config_drive | |
| created | 2020-11-26T06:50:26Z |
| flavor | m1.large (a9acc2de-39d7-4148-8d16-413c3b696e9d) |
| hostId | fa75360746cc026112b3802a27c06948313b684d2f736724e3482ad4 |
| id | c5949b79-a8a9-4cd0-a0e0-6fe37f567270 |
| image | wj47ios1126y-xb8px-rhcos (64a046b8-0f3b-44d9-b51d-83dcbe9cb928) |
| key_name | None |
| name | wj47ios1126y-xb8px-worker-0-dmvtk |
| progress | 0 |
| project_id | 542c6ebd48bf40fa857fc245c7572e30 |
| properties | Name='wj47ios1126y-xb8px-worker', openshiftClusterID='wj47ios1126y-xb8px' |
| security_groups | name='wj47ios1126y-xb8px-worker' |
| | name='default' |
| status | ACTIVE |
| updated | 2020-11-26T06:50:50Z |
| user_id | b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1 |
| volumes_attached | |
+-----------------------------+---------------------------------------------------------------------------+
Checked with latest payload
./openshift-install 4.7.0-0.nightly-2020-11-26-221840
built from commit 64ec239bc596635b50dd82485c9932cdf10c861e
release image registry.svc.ci.openshift.org/ocp/release@sha256:542e9447623e5e5f0ba96be505d695b81b7b0b088452a19d66b0c4f1e0f6654b
with install-config.yaml:
---
apiVersion: v1
controlPlane:
architecture: amd64
hyperthreading: Enabled
name: master
platform:
openstack:
additionalSecurityGroupIDs: &1
- 8794f45c-4f54-40a4-aadb-38d6c32e286e
replicas: 3
compute:
- architecture: amd64
hyperthreading: Enabled
name: worker
platform:
openstack:
additionalSecurityGroupIDs: *1
type: m1.large
replicas: 3
metadata:
name: wj47ios1127z
platform:
openstack:
cloud: openstack
computeFlavor: m1.xlarge
region: regionOne
trunkSupport: '1'
octaviaSupport: '0'
lbFloatingIP: 10.0.103.36
ingressFloatingIP: 10.0.103.31
externalNetwork: provider_net_cci_8
pullSecret: HIDDEN
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
serviceNetwork:
- 172.30.0.0/16
machineNetwork:
- cidr: 192.168.0.0/18
networkType: OpenShiftSDN
publish: External
baseDomain: 1127-ggc.qe.rhcloud.com
sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa37P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D
openshift-qe
# openstack server list --name wj47ios1127z-9mdcj -f value -c ID | xargs -n 1 openstack server show -f json|jq -r '"===========\n"+"Server: "+ .name, "Security Groups:\n" + .security_groups'
===========
Server: wj47ios1127z-9mdcj-worker-0-l8n4w
Security Groups:
name='wj47ios1127z-9mdcj-worker'
name='default'
===========
Server: wj47ios1127z-9mdcj-worker-0-kb2wj
Security Groups:
name='wj47ios1127z-9mdcj-worker'
name='default'
===========
Server: wj47ios1127z-9mdcj-worker-0-dx2v9
Security Groups:
name='wj47ios1127z-9mdcj-worker'
name='default'
===========
Server: wj47ios1127z-9mdcj-master-2
Security Groups:
name='default'
name='wj47ios1127z-9mdcj-master'
===========
Server: wj47ios1127z-9mdcj-master-1
Security Groups:
name='default'
name='wj47ios1127z-9mdcj-master'
===========
Server: wj47ios1127z-9mdcj-master-0
Security Groups:
name='default'
name='wj47ios1127z-9mdcj-master'
is there anything further that needs to be done on this? also, can this be back-ported to 4.5/4.6? thnx, m (In reply to milti leonard from comment #8) > is there anything further that needs to be done on this? also, can this be > back-ported to 4.5/4.6? > > thnx, m The 4.6 patch for is waiting for the release-manager's approval https://github.com/openshift/installer/pull/4420 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633 |
Version: $ openshift-install version [stack@osp16 test-additional-sg]$ ../openshift-install version ../openshift-install 4.6.3 built from commit a4f0869e0d2a5b2d645f0f28ef9e4b100fa8f779 release image 192.168.100.98:443/ocp4/openshift4@sha256:14986d2b9c112ca955aaa03f7157beadda0bd3c089e5e1d56f28020d2dd55c52 Platform: OpenStack 16.1 Please specify: IPI (automated install with `openshift-install`. If you don't know, then it's IPI) What happened? Configuring additionalSecurityGroupIDs for the masters does not work. Cluster is bootstrapped, however additional security groups are missing. ~~~ controlPlane: hyperthreading: Enabled architecture: amd64 name: master platform: openstack: type: openshift.master additionalSecurityGroupIDs: - 2959554a-8cca-4260-82bf-e0fcbb87f40c replicas: 3 ~~~ The additional security group can not be found on the resulting servers ~~~ [stack@osp16 ocp-test1]$ openstack server show ocp-99l7h-master-0 +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | id | d6c5466a-b43a-47c2-84a4-41afe574d3f2 | | name | ocp-99l7h-master-0 | | security_groups | name='ocp-99l7h-master' ~~~ I would suspect that https://github.com/openshift/installer/blob/release-4.6/data/data/openstack/topology/private-network.tf#L46 is missing the additional ID's that are configured here https://github.com/openshift/installer/blob/release-4.6/data/data/openstack/masters/main.tf#L49 Reproducer: http://pastebin.test.redhat.com/919822 Fixed Reproducer: http://pastebin.test.redhat.com/919824 It seems that port security group settings overrule machine settings. What did you expect to happen? Masters get additional security groups How to reproduce it (as minimally and precisely as possible)? Create additional security group Configure additionalSecurityGroupIDs in the installconfig Install IPI verify the security groups