Bug 1326288

Summary: Got "ERROR 2006 (HY000): MySQL server has gone away" after new deployment finish for rolling deployment
Product: OKD Reporter: zhou ying <yinzhou>
Component: DeploymentsAssignee: Paul Weil <pweil>
Status: CLOSED CURRENTRELEASE QA Contact: zhou ying <yinzhou>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.xCC: aos-bugs, bparees, mifiedle
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-19 13:49:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description zhou ying 2016-04-12 10:53:12 UTC
Description of problem:
For rolling deployment, after new deploy, the connection to service was lost.

Version-Release number of selected component (if applicable):
openshift v1.1.6-72-g5aa33d1
kubernetes v1.2.0-36-g4a3f9c5
etcd 2.2.5

How reproducible:
Always

Steps to Reproduce:
1. Create a deployment with Rolling type;
    {
        "kind": "DeploymentConfig",
        "apiVersion": "v1",
        "metadata": {
            "name": "hooks",
            "creationTimestamp": null,
            "labels": {
                "name": "mysql"
            }
        },
        "spec": {
            "strategy": {
                "type": "Rolling",
                "rollingParams": {
                    "updatePeriodSeconds": 1,
                    "intervalSeconds": 1,
                    "timeoutSeconds": 600
                },
                "resources": {}
            },
            "triggers": [
                {
                    "type": "ConfigChange"
                }
            ],
            "replicas": 1,
            "selector": {
                "name": "mysql"
            },
            "template": {
                "metadata": {
                    "creationTimestamp": null,
                    "labels": {
                        "name": "mysql"
                    }
                },
                "spec": {
                    "containers": [
                        {
                            "name": "mysql-55-centos7",
                            "image": "openshift/mysql-55-centos7:latest",
                            "ports": [
                                {
                                    "containerPort": 3306,
                                    "protocol": "TCP"
                                }
                            ],
                            "env": [
                                {
                                    "name": "MYSQL_USER",
                                    "value": "user8Y2"
                                },
                                {
                                    "name": "MYSQL_PASSWORD",
                                    "value": "Plqe5Wev"
                                },
                                {
                                    "name": "MYSQL_DATABASE",
                                    "value": "root"
                                }
                            ],
                            "resources": {},
                            "terminationMessagePath": "/dev/termination-log",
                            "imagePullPolicy": "Always",
                            "securityContext": {
                                "capabilities": {},
                                "privileged": false
                            }
                        }
                    ],
                    "restartPolicy": "Always",
                    "dnsPolicy": "ClusterFirst"
                }
            }
        },
        "status": {}
    }

2. Expose the dc:
 `oc expose dc hooks`
3. On node use the svc, connect the mysql, and Don't close the terminal afer connect to mysql
 `oc get svc`
 NAME      CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
hooks     172.30.24.184   <none>        3306/TCP   12m
 `mysql -h 172.30.24.184 -u username -ppassword`
4. Triger a now deployment for the dc;
5. After the new deployment completed, check the connection for the mysql app by cmd:  show tables; in step3's terminal

Actual results:
step3: mysql could be connected normally without any error.
[root@openshift-123 ~]# mysql -h 172.30.24.184 -u user8Y2 -pPlqe5Wev
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.5.45 MySQL Community Server (GPL)

Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| root               |
+--------------------+
2 rows in set (0.00 sec)

MySQL [(none)]> use root
Database changed
MySQL [root]> show tables;
Empty set (0.00 sec)

5. After the deploy, show tables, there was reconnect actions:

MySQL [root]> show tables;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    1
Current database: root

Empty set (0.00 sec)


Expected results:
step5:
Should not lost mysql connection like 
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...

since old pod will terminate after new pod become Running status, and during deployment or after deployment, the mysql connection should not be lost.

Additional info:

Comment 3 zhou ying 2016-04-13 10:18:26 UTC
Thanks for replay, will close this.