Created attachment 1750840 [details] ipsec pod log Description of problem: After installing Submariner on AWS and OSP clusters, the Active gateway shows that not all connections were established (2 out of 9 IPSec tunnels) Version-Release number of selected component (if applicable): Submariner 0.8.0 How reproducible: Sometimes Steps to Reproduce: https://qe-jenkins-csb-skynet.cloud.paas.psi.redhat.com/job/Submariner-OSP-AWS-No-Overlapping/160/Test-Report Actual results: $ subctl show all Showing information for cluster "pkomarov-cluster-a": Discovered network details: Network plugin: OpenShiftSDN Service CIDRs: [172.31.0.0/16] Cluster CIDRs: [10.132.0.0/14] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE pkomarov-cluster-a 10.1.64.160 18.225.31.220 libreswan local default-cl2 10.2.0.206 66.187.232.129 libreswan remote GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS default-cl2-mr8gk-worker-kskb2 default-cl2 10.2.0.206 libreswan 172.32.0.0/16, 10.136.0.0/14 connected NODE HA STATUS SUMMARY ip-10-1-64-160 active All connections (1) are established COMPONENT REPOSITORY VERSION submariner registry.redhat.io/rhacm2-tech-preview v0.8.0 submariner-operator registry.redhat.io/rhacm2-tech-preview/submariner-rhe v0.8.0 service-discovery registry.redhat.io/rhacm2-tech-preview v0.8.0 Showing information for cluster "default-cl2": Discovered network details: Network plugin: OpenShiftSDN Service CIDRs: [172.32.0.0/16] Cluster CIDRs: [10.136.0.0/14] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE default-cl2 10.2.0.206 66.187.232.129 libreswan local pkomarov-cluster-a 10.1.64.160 18.225.31.220 libreswan remote GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS ip-10-1-64-160 pkomarov-cluster-a 10.1.64.160 libreswan 172.31.0.0/16, 10.132.0.0/14 connected NODE HA STATUS SUMMARY default-cl2-mr8gk-worker-kskb2 active All connections (1) are established COMPONENT REPOSITORY VERSION submariner registry.redhat.io/rhacm2-tech-preview v0.8.0 submariner-operator registry.redhat.io/rhacm2-tech-preview/submariner-rhe v0.8.0 service-discovery registry.redhat.io/rhacm2-tech-preview v0.8.0 # However, looking at the active gateway, I see that not all IPSec connections were established: 000 Total IPsec connections: loaded 9, active 2 000 000 State Information: DDoS cookies not required, Accepting new IKE connections 000 IKE SAs: total(2), half-open(1), open(0), authenticated(1), anonymous(0) 000 IPsec SAs: total(2), authenticated(2), anonymous(0) 000 000 #5: "submariner-cable-default-cl2-10-2-0-206-0-0":53058 STATE_V2_ESTABLISHED_CHILD_SA (IPsec SA established); EVENT_SA_REKEY in 28518s; newest IPSEC; eroute owner; isakmp#4; idle; 000 #5: "submariner-cable-default-cl2-10-2-0-206-0-0" esp.eda0e601.232.129 esp.7f2df5b4.64.160 tun.0.232.129 tun.0.64.160 Traffic: ESPin=0B ESPout=0B! ESPmax=0B 000 #7: "submariner-cable-default-cl2-10-2-0-206-0-0":4501 STATE_PARENT_I1 (sent IKE_SA_INIT request); EVENT_RETRANSMIT in 0s; idle; 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #7: pending CHILD SA for "submariner-cable-default-cl2-10-2-0-206-0-0" 000 #4: "submariner-cable-default-cl2-10-2-0-206-2-2":53058 STATE_V2_ESTABLISHED_IKE_SA (established IKE SA); EVENT_SA_REKEY in 3318s; newest ISAKMP; idle; 000 #6: "submariner-cable-default-cl2-10-2-0-206-2-2":53058 STATE_V2_ESTABLISHED_CHILD_SA (IPsec SA established); EVENT_SA_REKEY in 28519s; newest IPSEC; eroute owner; isakmp#4; idle; 000 #6: "submariner-cable-default-cl2-10-2-0-206-2-2" esp.16339c47.232.129 esp.5c3520a.64.160 tun.0.232.129 tun.0.64.160 Traffic: ESPin=924B ESPout=924B! ESPmax=0B Expected results: The number of loaded ipsec connections, should be the same number of the active ones. Additional info: Full gateway pod log attached
G2Bsync 768992593 comment nyechiel Thu, 28 Jan 2021 11:33:29 UTC G2Bsync This seems like a Libreswan/IPsec issues which is being investigated here: https://github.com/submariner-io/submariner/issues/1081
Created attachment 1754799 [details] Here's how the ipsec connection 9/9 active (bug fixed)
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (RHEA: Submariner 0.8 - bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:0728