Bug 1915563
| Summary: | Unable to connect to mysql server after upgrading to 4.6.0-202012161211.p0 with driver not found | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | tflannag | |
| Component: | Metering Operator | Assignee: | tflannag | |
| Status: | CLOSED ERRATA | QA Contact: | Peter Ruan <pruan> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 4.7 | CC: | aos-bugs, btofel, iwatson, ocasalsa, pruan, scuppett, sd-operator-metering, tflannag | |
| Target Milestone: | --- | |||
| Target Release: | 4.7.z | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 1913800 | |||
| : | 1931534 (view as bug list) | Environment: | ||
| Last Closed: | 2021-03-25 01:45:10 UTC | Type: | --- | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1931534 | |||
| Bug Blocks: | 1913800 | |||
|
Comment 3
Peter Ruan
2021-02-18 17:44:29 UTC
metastore pod failed during upgrade Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:334) at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:164) at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1342) at com.mysql.cj.NativeSession.connect(NativeSession.java:157) at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956) at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826) ... 63 more Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate) at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171) at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:98) at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:387) at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:336) at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:188) at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:99) at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:325) ... 68 more verified by doing the following:
1. spin up a OCP >= 4.7
2. install custom catalogsource with 4.5 support (this is needed because 4.5 channel is not visible in a OCP4.7 installation)
$> oc create -f catalogsrc.yaml
### catalogsrc.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
name: metering-custom-rhel
namespace: openshift-marketplace
spec:
sourceType: grpc
image: quay.io/tflannag/index:4.5-rhel
displayName: "Metering Custom 4.5 Image"
publisher: "Metering Dev"
3. Install metering operator from Operatorhub using the 4.5 catalogsource
4. import custom mysql5.7 image into cluster
`oc import-image --confirm mysql:5.7 --from=quay.io/tflannag/mysql:5.7`
5. run the following bash script to install mysql
#! /bin/bash
set -eou pipefail
NAMESPACE=${1:?}
SECRET_NAME=${2:-"${NAMESPACE}-mysql-secret"}
MYSQL_IMAGE_STREAM=${MYSQL_IMAGE_STREAM:=mysql:5.7}
CREATE_OUTPUT_FILE=${CREATE_OUTPUT_FILE:=false}
if ! oc get ns ${NAMESPACE} > /dev/null 2>&1; then
echo "Creating the namespace"
oc create ns ${NAMESPACE}
fi
# TODO: this shouldn't flush output to stderr
echo "Creating the mysql instance"
oc -n ${NAMESPACE} new-app \
--image-stream ${MYSQL_IMAGE_STREAM} \
MYSQL_USER=testuser \
MYSQL_PASSWORD=testpass \
MYSQL_DATABASE=metastore \
-l db=mysql > /dev/null 2>&1
echo "Creating the secret name containing the username and password"
oc -n ${NAMESPACE} create secret generic ${SECRET_NAME} \
--from-literal=username=testuser \
--from-literal=password=testpass 2>/dev/null
# TODO: not entirely sure why this was needed. Should be able to use the service DNS path instead
# of manually referencing the spec.ClusterIP of the service that gets created from `oc new-app`.
service=$(kubectl --namespace ${NAMESPACE} get svc -l db=mysql --no-headers | awk '{ print $1 }')
export CLUSTER_IP=$(kubectl --namespace ${NAMESPACE} get svc ${service} -o jsonpath='{.spec.clusterIP}')
while [[ $? != 0 ]]; do
echo "Waiting for the 'mysql' Service to have a populated spec.ClusterIP"
export CLUSTER_IP=$(kubectl --namespace ${NAMESPACE} get svc ${service} --jsonpath='{.spec.clusterIP}')
done
echo "Grabbed the MySQL Service ClusterIP: ${CLUSTER_IP}"
6. install the following meteringconfig (replace the mysql exposed IP). Please note the `url` line ==> url: "jdbc:mysql://172.30.18.153:3306/metastore?enabledTLSProtocols=TLSv1.2"
apiVersion: metering.openshift.io/v1
kind: MeteringConfig
metadata:
name: operator-metering
namespace: openshift-metering
spec:
storage:
hive:
type: "s3"
s3:
bucket: "qe-metering-pruan-mac"
region: "us-west-1"
secretName: "my-aws-secret"
# Set to false if you want to provide an existing bucket, instead of
# having Metering create the bucket on your behalf.
createBucket: true
type: hive
reporting-operator:
spec:
resources:
limits:
cpu: 500m
memory: 500Mi
requests:
cpu: 50m
memory: 100Mi
presto:
spec:
coordinator:
resources:
limits:
cpu: 1
memory: 2Gi
requests:
cpu: 500m
memory: 1Gi
hive:
spec:
config:
db:
url: "jdbc:mysql://172.30.18.153:3306/metastore?enabledTLSProtocols=TLSv1.2"
driver: "com.mysql.jdbc.Driver"
username: "testuser"
password: "testpass"
metastore:
storage:
create: false
resources:
limits:
cpu: 2
memory: 2Gi
requests:
cpu: 500m
memory: 650Mi
server:
resources:
limits:
cpu: 1
memory: 1Gi
requests:
cpu: 500m
memory: 500Mi
7. after successful installation create a report to verify everything is working
8. Upgrade metering to 4.7 by doing `oc edit sub <metering_subscription_name>` and changing the spec.channel to '4.7' and spec.source to 'qe-app-registry'
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.7.3 extras update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0823 |