Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1332995

Summary: systemctl returns error if rsync fail
Product: Red Hat Software Collections Reporter: Petr Sklenar <psklenar>
Component: mariadbAssignee: Michal Schorm <mschorm>
Status: CLOSED WORKSFORME QA Contact: qe-baseos-daemons
Severity: medium Docs Contact:
Priority: medium    
Version: rh-mariadb101CC: databases-maint, hhorak, kanderso, lzachar, mmuzila, tborcin
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-06 18:07:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Petr Sklenar 2016-05-04 13:38:10 UTC
Description of problem:
if I have cluster created systemctl returns error if rsync fail.

Version-Release number of selected component (if applicable):
rh-mariadb101-mariadb

How reproducible:
always
 ~ actually 'always' with my created setup with three nodes which I did once only

Steps to Reproduce:
1, setup cluster/galera using galera_new_cluster, galera.cnf
2, have a more nodes (lets say 3)
3, NODE1: systemctl stop rh-mariadb101-mariadb
4, NODE2: create databes XYZ;
5; NODE1: systemctl start rh-mariadb101-mariadb and it shows ERROR

in real there is NO error

Actual results:
[0 root@qeos-47 ~]# systemctl start rh-mariadb101-mariadb
Job for rh-mariadb101-mariadb.service failed because a fatal signal was delivered to the control process. See "systemctl status rh-mariadb101-mariadb.service" and "journalctl -xe" for details.
[0 root@qeos-47 ~]# systemctl status rh-mariadb101-mariadb
● rh-mariadb101-mariadb.service - MariaDB 10.1 database server
   Loaded: loaded (/usr/lib/systemd/system/rh-mariadb101-mariadb.service; disabled; vendor preset: disabled)
   Active: activating (start) since Wed 2016-05-04 09:23:38 EDT; 4s ago
  Process: 29021 ExecStopPost=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
  Process: 29100 ExecStartPre=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
  Process: 29072 ExecStartPre=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
  Process: 29066 ExecStartPre=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /usr/bin/scl_enabled rh-mariadb101 (code=exited, status=0/SUCCESS)
 Main PID: 29138 (mysqld)
   CGroup: /system.slice/rh-mariadb101-mariadb.service
           ├─29138 /opt/rh/rh-mariadb101/root/usr/libexec/mysqld --basedir=/opt/rh/rh-mariadb101/root/usr
           ├─29151 /bin/bash -ue /opt/rh/rh-mariadb101/root/usr//bin/wsrep_sst_rsync --role joiner --address 172.16.36.47 --datadir /var/opt/rh/rh-mariadb101/lib/mysql/ ...
           ├─29181 rsync --daemon --no-detach --port 4444 --config /var/opt/rh/rh-mariadb101/lib/mysql//rsync_sst.conf
           ├─29243 sleep 1
           └─29252 rsync --daemon --no-detach --port 4444 --config /var/opt/rh/rh-mariadb101/lib/mysql//rsync_sst.conf

May 04 09:23:42 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29246]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:42 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29246]: rsync to rsync_sst/./a50 from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:42 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29246]: receiving file list
May 04 09:23:42 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29248]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29248]: rsync to rsync_sst/./a60 from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29248]: receiving file list
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29250]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29250]: rsync to rsync_sst/./a66 from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29250]: receiving file list
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29252]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
[0 root@qeos-47 ~]# systemctl status rh-mariadb101-mariadb
● rh-mariadb101-mariadb.service - MariaDB 10.1 database server
   Loaded: loaded (/usr/lib/systemd/system/rh-mariadb101-mariadb.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-05-04 09:23:45 EDT; 8s ago
  Process: 29021 ExecStopPost=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
  Process: 29312 ExecStartPost=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-check-upgrade (code=exited, status=0/SUCCESS)
  Process: 29100 ExecStartPre=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-prepare-db-dir %n (code=exited, status=0/SUCCESS)
  Process: 29072 ExecStartPre=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /opt/rh/rh-mariadb101/root/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
  Process: 29066 ExecStartPre=/usr/bin/scl enable $RH_MARIADB101_SCLS_ENABLED -- /usr/bin/scl_enabled rh-mariadb101 (code=exited, status=0/SUCCESS)
 Main PID: 29138 (mysqld)
   Status: "Taking your SQL requests now..."
   CGroup: /system.slice/rh-mariadb101-mariadb.service
           └─29138 /opt/rh/rh-mariadb101/root/usr/libexec/mysqld --basedir=/opt/rh/rh-mariadb101/root/usr

May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29264]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29264]: rsync to rsync_sst/./ahoj5 from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29264]: receiving file list
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29267]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29267]: rsync to rsync_sst/./a44 from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29267]: receiving file list
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:44 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: rsync to rsync_sst/ from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:44 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: receiving file list
May 04 09:23:45 qeos-47.lab.eng.rdu2.redhat.com systemd[1]: Started MariaDB 10.1 database server.


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! error comes from rsync command:
journalctl -xe
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: connect from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:43 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29267]: sent 54 bytes  received 205 bytes  total size 65
May 04 09:23:44 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: rsync to rsync_sst/ from qeos-46.lab.eng.rdu2.redhat.com (172.16.36.46)
May 04 09:23:44 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: receiving file list
May 04 09:23:44 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29269]: sent 54 bytes  received 177 bytes  total size 42
May 04 09:23:45 qeos-47.lab.eng.rdu2.redhat.com rsyncd[29181]: rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(551) [Receiver=3.0.9]


Expected results:
no such an error

Additional info:

Comment 10 Joe Orton 2019-03-14 11:02:42 UTC
Red Hat does not currently plan to provide any further changes to this collection in a Red Hat Software Collections update release.

This software collection is nearing the retirement date (May 2019) after which customers are encouraged to upgrade to a later release.

Please contact Red Hat Support if you have further questions, or refer to the support lifecycle page for more information. https://access.redhat.com/support/policy/updates/rhscl/

Comment 11 Michal Schorm 2019-04-06 18:07:08 UTC
I wasn't able to reproduce the issue with following setup and steps:

Setup:
  machine:          3x RHEL-7.6-released
  selinux:          Enforcing
  firewalld:        inactive (dead)
  collection:       rh-mariadb-101
  package versions:
         mariadb*   1:10.1.29-3.el7
         galera     25.3.12-12.el7
         rsync      3.1.2-4.el7

Steps to reproduce:
  edit the /etc/opt/rh/rh-mariadb101/my.cnf.d/galera.cnf,
    so each will have their name as "wsrep_node_name=" (e.g. "NODE_2")
    NODE_1 will have "wsrep_cluster_address=gcomm://" set with IP of NODE_2
    NODE_2 & NODE_3 will have "wsrep_cluster_address=gcomm://" set with IP of NODE_1

  On NODE_1:
    # scl enable rh-mariadb101 /opt/rh/rh-mariadb101/root/usr/bin/galera_new_cluster
  On NODE_2 & NODE_3:
    # systemctl start rh-mariadb101-mariadb

  On all nodes:
    # scl enable rh-mariadb101 mysql
  On each node create an unique database and check the cluster is working as expected

  On NODE_1:
    # systemctl stop rh-mariadb101-mariadb
  On NODE_2 and 3 create unique databases

  On NODE_1:
    # systemctl start rh-mariadb101-mariadb



The NODE_1 will join and catch up with the rest of the cluster as expected.

--

I tested the same setup also with "rh-mariadb102" (1:10.2.8-5.el7) collection with the exact same results.

--

If you still have the problem, please test in against 10.2 or 10.3 collection and report / clone the bug there, since 10.1 goes EOL soon.