Bug 1732171

Summary: The journal gateway public port TCP 19531 on the bootstrap machine using ipi-on-azure is closed
Product: OpenShift Container Platform Reporter: Etienne Simard <esimard>
Component: InstallerAssignee: Abhinav Dahiya <adahiya>
Installer sub component: openshift-installer QA Contact: Johnny Liu <jialiu>
Status: CLOSED NOTABUG Docs Contact:
Severity: medium    
Priority: unspecified    
Version: 4.2.0   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-22 21:17:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Etienne Simard 2019-07-22 21:15:16 UTC
Description of problem:

The port TCP 19531 Inbound is not currently allowed by the `${CLUSTER_NAME}*controlplane-nsg` Security Group in azure.

Version-Release number of the following components:

Cluster ID
c2410ca0-8988-4419-ba18-fde6175421af
Provider
Azure
OpenShift Version
4.2.0-0.okd-2019-07-22-195548

How reproducible:
Easily

Steps to Reproduce:
1. Create Cluster with ipi-on-azure
2. Before the end of the install, confirm that you can ssh to the ${BOOTSTRAP_IP} and then `curl --insecure --cert ${INSTALL_DIR}/tls/journal-gatewayd.crt --key ${INSTALL_DIR}/tls/journal-gatewayd.key 'https://${BOOTSTRAP_IP}:19531/entries?follow&_SYSTEMD_UNIT=bootkube.service'`


Actual results:

$  curl -vvv --insecure --cert ./tls/journal-gatewayd.crt --key ./tls/journal-gatewayd.key 'https://${BOOTSTRAP_IP}:19531/entries?follow&_SYSTEMD_UNIT=bootkube.service'
* Trying ${BOOTSTRAP_IP}...
* TCP_NODELAY set
* connect to ${BOOTSTRAP_IP} port 19531 failed: Connection timed out
* Failed to connect to ${BOOTSTRAP_IP} port 19531: Connection timed out
* Closing connection 0
curl: (7) Failed to connect to ${BOOTSTRAP_IP} port 19531: Connection timed out

$ telnet ${BOOTSTRAP_IP} 19531
Trying ${BOOTSTRAP_IP}...
^C

Check the `${CLUSTER_NAME}*controlplane-nsg` Security Group in azure and you will see that we allow TCP 22 inbound but not 19531


Expected results:

TCP 19531 inbound should exist in `${CLUSTER_NAME}*controlplane-nsg` Security Group

$ telnet ${BOOTSTRAP_IP} 19531
Trying ${BOOTSTRAP_IP}...
Connected to ${BOOTSTRAP_IP}.
Escape character is '^]'.
^CConnection closed by foreign host.

$ curl -vvv --insecure --cert ./tls/journal-gatewayd.crt --key ./tls/journal-gatewayd.key 'https://${BOOTSTRAP_IP}:19531/entries?follow&_SYSTEMD_UNIT=bootkube.service'
* Trying ${BOOTSTRAP_IP}...
* TCP_NODELAY set
* Connected to ${BOOTSTRAP_IP} (${BOOTSTRAP_IP}) port 19531 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* ignoring certificate verify locations due to disabled peer verification
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: O=OpenShift Bootstrap; CN=journal-gatewayd
* start date: Jul 22 20:18:43 2019 GMT
* expire date: Jul 19 20:18:50 2029 GMT
* issuer: OU=openshift; CN=root-ca
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET /entries?follow&_SYSTEMD_UNIT=bootkube.service HTTP/1.1
> Host: ${BOOTSTRAP_IP}:19531
> User-Agent: curl/7.59.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Connection: Keep-Alive
< Transfer-Encoding: chunked
< Content-Type: text/plain
< Date: Mon, 22 Jul 2019 20:41:27 GMT
< 
Jul 22 20:25:47 qe-esimard-24414-p5cmb-bootstrap bootkube.sh[1502]: Pulling release image...


Additional info:

We should add the following rule (I tested it to get the Expected results)

bootstrap_journal_gateway_in PORT 19531/TCP Source Any, Allow

similar to the current Inbound rule for SSH:

bootstrap_ssh_in  PORT 22/TCP source Any, Allow

Comment 1 Abhinav Dahiya 2019-07-22 21:17:18 UTC
We have moved away from requiring the journald-gateway port to be publicly accessible. in favour of a SSH based debug collector. So for 4.2 platform this port is not required to be public.