Bug 1915563 - Unable to connect to mysql server after upgrading to 4.6.0-202012161211.p0 with driver not found
Summary: Unable to connect to mysql server after upgrading to 4.6.0-202012161211.p0 wi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Metering Operator
Version: 4.7
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.7.z
Assignee: tflannag
QA Contact: Peter Ruan
URL:
Whiteboard:
Depends On: 1931534
Blocks: 1913800
TreeView+ depends on / blocked
 
Reported: 2021-01-12 22:24 UTC by tflannag
Modified: 2024-06-13 23:54 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1913800
: 1931534 (view as bug list)
Environment:
Last Closed: 2021-03-25 01:45:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kube-reporting hive pull 58 0 None closed Bug 1915563: Ensure the mysql-connector-java jar is present 2021-02-17 17:38:04 UTC
Red Hat Knowledge Base (Solution) 5962661 0 None None None 2021-04-14 08:11:30 UTC
Red Hat Product Errata RHBA-2021:0823 0 None None None 2021-03-25 01:45:15 UTC

Comment 3 Peter Ruan 2021-02-18 17:44:29 UTC
metastore pod failed during upgrade

Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
	at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
	at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:334)
	at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:164)
	at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1342)
	at com.mysql.cj.NativeSession.connect(NativeSession.java:157)
	at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956)
	at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826)
	... 63 more
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
	at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
	at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:98)
	at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:387)
	at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:336)
	at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:188)
	at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:99)
	at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:325)
	... 68 more

Comment 4 Peter Ruan 2021-02-18 17:44:29 UTC
metastore pod failed during upgrade

Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
	at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
	at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
	at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:334)
	at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:164)
	at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1342)
	at com.mysql.cj.NativeSession.connect(NativeSession.java:157)
	at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956)
	at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826)
	... 63 more
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
	at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
	at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:98)
	at sun.security.ssl.TransportContext.kickstart(TransportContext.java:220)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:387)
	at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:336)
	at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:188)
	at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:99)
	at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:325)
	... 68 more

Comment 8 Peter Ruan 2021-03-11 06:29:15 UTC
verified by doing the following:

1. spin up a OCP >= 4.7
2. install custom catalogsource with 4.5 support (this is needed because 4.5 channel is not visible in a OCP4.7 installation)
  $> oc create -f catalogsrc.yaml
### catalogsrc.yaml
apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: metering-custom-rhel
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/tflannag/index:4.5-rhel
  displayName: "Metering Custom 4.5 Image"
  publisher: "Metering Dev"

3. Install metering operator from Operatorhub using the 4.5 catalogsource
4. import custom mysql5.7 image into cluster
   `oc import-image --confirm mysql:5.7 --from=quay.io/tflannag/mysql:5.7`

5. run the following bash script to install mysql 
#! /bin/bash
set -eou pipefail
NAMESPACE=${1:?}
SECRET_NAME=${2:-"${NAMESPACE}-mysql-secret"}
MYSQL_IMAGE_STREAM=${MYSQL_IMAGE_STREAM:=mysql:5.7}
CREATE_OUTPUT_FILE=${CREATE_OUTPUT_FILE:=false}
if ! oc get ns ${NAMESPACE} > /dev/null 2>&1; then
  echo "Creating the namespace"
  oc create ns ${NAMESPACE}
fi
# TODO: this shouldn't flush output to stderr
echo "Creating the mysql instance"
oc -n ${NAMESPACE} new-app \
  --image-stream ${MYSQL_IMAGE_STREAM} \
	MYSQL_USER=testuser \
	MYSQL_PASSWORD=testpass \
	MYSQL_DATABASE=metastore \
	-l db=mysql > /dev/null 2>&1
echo "Creating the secret name containing the username and password"
oc -n ${NAMESPACE} create secret generic ${SECRET_NAME} \
    --from-literal=username=testuser \
    --from-literal=password=testpass 2>/dev/null
# TODO: not entirely sure why this was needed. Should be able to use the service DNS path instead
# of manually referencing the spec.ClusterIP of the service that gets created from `oc new-app`.
service=$(kubectl --namespace ${NAMESPACE} get svc -l db=mysql --no-headers | awk '{ print $1 }')
export CLUSTER_IP=$(kubectl --namespace ${NAMESPACE} get svc ${service} -o jsonpath='{.spec.clusterIP}')
while [[ $? != 0 ]]; do
    echo "Waiting for the 'mysql' Service to have a populated spec.ClusterIP"
    export CLUSTER_IP=$(kubectl --namespace ${NAMESPACE} get svc ${service} --jsonpath='{.spec.clusterIP}')
done
echo "Grabbed the MySQL Service ClusterIP: ${CLUSTER_IP}"

6.  install the following meteringconfig (replace the mysql exposed IP).  Please note the `url` line ==> url: "jdbc:mysql://172.30.18.153:3306/metastore?enabledTLSProtocols=TLSv1.2"


apiVersion: metering.openshift.io/v1
kind: MeteringConfig
metadata:
  name: operator-metering
  namespace: openshift-metering
spec:
  storage:
    hive:
      type: "s3"
      s3:
        bucket: "qe-metering-pruan-mac"
        region: "us-west-1"
        secretName: "my-aws-secret"
        # Set to false if you want to provide an existing bucket, instead of
        # having Metering create the bucket on your behalf.
        createBucket: true
    type: hive
  reporting-operator:
    spec:
      resources:
        limits:
          cpu: 500m
          memory: 500Mi
        requests:
          cpu: 50m
          memory: 100Mi
  presto:
    spec:
      coordinator:
        resources:
          limits:
            cpu: 1
            memory: 2Gi
          requests:
            cpu: 500m
            memory: 1Gi

  hive:
    spec:
      config:
        db:
          url: "jdbc:mysql://172.30.18.153:3306/metastore?enabledTLSProtocols=TLSv1.2"
          driver: "com.mysql.jdbc.Driver"
          username: "testuser"
          password: "testpass"
      metastore:
        storage:
          create: false
        resources:
          limits:
            cpu: 2
            memory: 2Gi
          requests:
            cpu: 500m
            memory: 650Mi
      server:
        resources:
          limits:
            cpu: 1
            memory: 1Gi
          requests:
            cpu: 500m
            memory: 500Mi

7.  after successful installation create a report to verify everything is working
8. Upgrade metering to 4.7 by doing `oc edit sub <metering_subscription_name>` and changing the spec.channel to '4.7' and spec.source to 'qe-app-registry'

Comment 11 errata-xmlrpc 2021-03-25 01:45:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.3 extras update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0823


Note You need to log in before you can comment on or make changes to this bug.