Version: $ openshift-install version [stack@osp16 test-additional-sg]$ ../openshift-install version ../openshift-install 4.6.3 built from commit a4f0869e0d2a5b2d645f0f28ef9e4b100fa8f779 release image 192.168.100.98:443/ocp4/openshift4@sha256:14986d2b9c112ca955aaa03f7157beadda0bd3c089e5e1d56f28020d2dd55c52 Platform: OpenStack 16.1 Please specify: IPI (automated install with `openshift-install`. If you don't know, then it's IPI) What happened? Configuring additionalSecurityGroupIDs for the masters does not work. Cluster is bootstrapped, however additional security groups are missing. ~~~ controlPlane: hyperthreading: Enabled architecture: amd64 name: master platform: openstack: type: openshift.master additionalSecurityGroupIDs: - 2959554a-8cca-4260-82bf-e0fcbb87f40c replicas: 3 ~~~ The additional security group can not be found on the resulting servers ~~~ [stack@osp16 ocp-test1]$ openstack server show ocp-99l7h-master-0 +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | id | d6c5466a-b43a-47c2-84a4-41afe574d3f2 | | name | ocp-99l7h-master-0 | | security_groups | name='ocp-99l7h-master' ~~~ I would suspect that https://github.com/openshift/installer/blob/release-4.6/data/data/openstack/topology/private-network.tf#L46 is missing the additional ID's that are configured here https://github.com/openshift/installer/blob/release-4.6/data/data/openstack/masters/main.tf#L49 Reproducer: http://pastebin.test.redhat.com/919822 Fixed Reproducer: http://pastebin.test.redhat.com/919824 It seems that port security group settings overrule machine settings. What did you expect to happen? Masters get additional security groups How to reproduce it (as minimally and precisely as possible)? Create additional security group Configure additionalSecurityGroupIDs in the installconfig Install IPI verify the security groups
A question from Martin on Github[1]: > The patch fixes exactly the issue that is reported in the BZ so I do not see any reason not to approve. > It would be nice to know if we also need to apply the additional security groups to the VIP ports. Otherwise > what is the point of adding security groups at all to api_port and ingress_port? @robert: do we cover the use case by just setting the additional SGs on the master ports, or should we go beyond that to deliver value for the customer? [1]: https://github.com/openshift/installer/pull/4411#pullrequestreview-538216817
@pierre The SG should be set only to the master ports and not to the VIP ports. It should follow the same behaviour as with the worker nodes.
Perfect, thanks.
Checked with latest ./openshift-install 4.7.0-0.nightly-2020-11-25-114114 built from commit a9e6c4d8fa0e7d5edb9cf95330689a65261ff09c release image registry.svc.ci.openshift.org/ocp/release@sha256:bf37e13af0e254d0b744b62ace0dcf5560230374d7877a8fde16cf9134ec7862 --- apiVersion: v1 controlPlane: architecture: amd64 hyperthreading: Enabled name: master platform: openstack: additionalSecurityGroupIDs: &1 - 8794f45c-4f54-40a4-aadb-38d6c32e286e replicas: 3 compute: - architecture: amd64 hyperthreading: Enabled name: worker platform: openstack: additionalSecurityGroupIDs: *1 type: m1.large replicas: 3 metadata: name: wj47ios1126y platform: openstack: cloud: openstack computeFlavor: m1.xlarge region: regionOne trunkSupport: '1' octaviaSupport: '0' lbFloatingIP: 10.0.102.125 ingressFloatingIP: 10.0.103.227 externalNetwork: provider_net_cci_8 pullSecret: HIDDEN networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 serviceNetwork: - 172.30.0.0/16 machineNetwork: - cidr: 192.168.0.0/18 networkType: OpenShiftSDN publish: External baseDomain: 1126-7gp.qe.rhcloud.com sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa37P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D openshift-qe and the control-plane still not take affect, need wait new payload to have a try. # openstack server show wj47ios1126y-xb8px-master-2 +-----------------------------+---------------------------------------------------------------------------+ | Field | Value | +-----------------------------+---------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2020-11-26T06:24:13.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | wj47ios1126y-xb8px-openshift=192.168.2.137 | | config_drive | | | created | 2020-11-26T06:23:21Z | | flavor | m1.xlarge (3f183920-6cba-4bfb-ab3a-599559cf0f97) | | hostId | fa75360746cc026112b3802a27c06948313b684d2f736724e3482ad4 | | id | 4ed158a9-cd4f-440a-85ba-5c55cf0a40d9 | | image | wj47ios1126y-xb8px-rhcos (64a046b8-0f3b-44d9-b51d-83dcbe9cb928) | | key_name | None | | name | wj47ios1126y-xb8px-master-2 | | progress | 0 | | project_id | 542c6ebd48bf40fa857fc245c7572e30 | | properties | Name='wj47ios1126y-xb8px-master', openshiftClusterID='wj47ios1126y-xb8px' | | security_groups | name='wj47ios1126y-xb8px-master' | | status | ACTIVE | | updated | 2020-11-26T06:24:13Z | | user_id | b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1 | | volumes_attached | | +-----------------------------+---------------------------------------------------------------------------+ [root@wjiang-bind-bastion ~]# openstack server show wj47ios1126y-xb8px-worker-0-dmvtk +-----------------------------+---------------------------------------------------------------------------+ | Field | Value | +-----------------------------+---------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2020-11-26T06:50:50.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | wj47ios1126y-xb8px-openshift=192.168.3.154 | | config_drive | | | created | 2020-11-26T06:50:26Z | | flavor | m1.large (a9acc2de-39d7-4148-8d16-413c3b696e9d) | | hostId | fa75360746cc026112b3802a27c06948313b684d2f736724e3482ad4 | | id | c5949b79-a8a9-4cd0-a0e0-6fe37f567270 | | image | wj47ios1126y-xb8px-rhcos (64a046b8-0f3b-44d9-b51d-83dcbe9cb928) | | key_name | None | | name | wj47ios1126y-xb8px-worker-0-dmvtk | | progress | 0 | | project_id | 542c6ebd48bf40fa857fc245c7572e30 | | properties | Name='wj47ios1126y-xb8px-worker', openshiftClusterID='wj47ios1126y-xb8px' | | security_groups | name='wj47ios1126y-xb8px-worker' | | | name='default' | | status | ACTIVE | | updated | 2020-11-26T06:50:50Z | | user_id | b414646065ab99780ef1bbcba52c07d2033a6f99fd0b10a3b1b12fcb5e5275e1 | | volumes_attached | | +-----------------------------+---------------------------------------------------------------------------+
Checked with latest payload ./openshift-install 4.7.0-0.nightly-2020-11-26-221840 built from commit 64ec239bc596635b50dd82485c9932cdf10c861e release image registry.svc.ci.openshift.org/ocp/release@sha256:542e9447623e5e5f0ba96be505d695b81b7b0b088452a19d66b0c4f1e0f6654b with install-config.yaml: --- apiVersion: v1 controlPlane: architecture: amd64 hyperthreading: Enabled name: master platform: openstack: additionalSecurityGroupIDs: &1 - 8794f45c-4f54-40a4-aadb-38d6c32e286e replicas: 3 compute: - architecture: amd64 hyperthreading: Enabled name: worker platform: openstack: additionalSecurityGroupIDs: *1 type: m1.large replicas: 3 metadata: name: wj47ios1127z platform: openstack: cloud: openstack computeFlavor: m1.xlarge region: regionOne trunkSupport: '1' octaviaSupport: '0' lbFloatingIP: 10.0.103.36 ingressFloatingIP: 10.0.103.31 externalNetwork: provider_net_cci_8 pullSecret: HIDDEN networking: clusterNetwork: - cidr: 10.128.0.0/14 hostPrefix: 23 serviceNetwork: - 172.30.0.0/16 machineNetwork: - cidr: 192.168.0.0/18 networkType: OpenShiftSDN publish: External baseDomain: 1127-ggc.qe.rhcloud.com sshKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCWkwurd8TNAi+D7ffvyDdhGBSQtJx3/Yedlwvvha0q772vLlOAGlKCw4dajKy6qty1/GGQDgTJ17h3C9TEArI8ZqILnyydeY56DL+ELN3dtGBVof/N2qtW0+SmEnd1Mi7Qy5Tx4e/GVmB3NgX9szwNOVXhebzgBsXc9x+RtCVLPLC8J+qqSdTUZ0UfJsh2ptlQLGHmmTpF//QlJ1tngvAFeCOxJUhrLAa37P9MtFsiNk31EfKyBk3eIdZljTERmqFaoJCohsFFEdO7tVgU6p5NwniAyBGZVjZBzjELoI1aZ+/g9yReIScxl1R6PWqEzcU6lGo2hInnb6nuZFGb+90D openshift-qe # openstack server list --name wj47ios1127z-9mdcj -f value -c ID | xargs -n 1 openstack server show -f json|jq -r '"===========\n"+"Server: "+ .name, "Security Groups:\n" + .security_groups' =========== Server: wj47ios1127z-9mdcj-worker-0-l8n4w Security Groups: name='wj47ios1127z-9mdcj-worker' name='default' =========== Server: wj47ios1127z-9mdcj-worker-0-kb2wj Security Groups: name='wj47ios1127z-9mdcj-worker' name='default' =========== Server: wj47ios1127z-9mdcj-worker-0-dx2v9 Security Groups: name='wj47ios1127z-9mdcj-worker' name='default' =========== Server: wj47ios1127z-9mdcj-master-2 Security Groups: name='default' name='wj47ios1127z-9mdcj-master' =========== Server: wj47ios1127z-9mdcj-master-1 Security Groups: name='default' name='wj47ios1127z-9mdcj-master' =========== Server: wj47ios1127z-9mdcj-master-0 Security Groups: name='default' name='wj47ios1127z-9mdcj-master'
is there anything further that needs to be done on this? also, can this be back-ported to 4.5/4.6? thnx, m
(In reply to milti leonard from comment #8) > is there anything further that needs to be done on this? also, can this be > back-ported to 4.5/4.6? > > thnx, m The 4.6 patch for is waiting for the release-manager's approval https://github.com/openshift/installer/pull/4420
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633