Bug 2037409 - [16.2] backup-and-restore : openstack overcloud backup fails at "Galera desync the MySQL node"
Summary: [16.2] backup-and-restore : openstack overcloud backup fails at "Galera desy...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: tripleo-ansible
Version: 16.2 (Train)
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: z3
: 16.2 (Train on RHEL 8.4)
Assignee: Juan Larriba
QA Contact: myadla
URL:
Whiteboard:
Depends On:
Blocks: 2039801
TreeView+ depends on / blocked
 
Reported: 2022-01-05 15:15 UTC by Jean-Francois Beaudoin
Modified: 2022-06-22 16:03 UTC (History)
7 users (show)

Fixed In Version: tripleo-ansible-0.8.1-2.20220215090656.c0ddf19.el8osttrunk
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 2039801 (view as bug list)
Environment:
Last Closed: 2022-06-22 16:03:23 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 828774 0 None MERGED [only train] BnR set default value for variable 2022-02-16 13:23:08 UTC
Red Hat Issue Tracker OSP-12031 0 None None None 2022-01-05 15:19:41 UTC
Red Hat Issue Tracker UPG-4893 0 None None None 2022-02-10 18:21:12 UTC
Red Hat Knowledge Base (Solution) 6627751 0 None None None 2022-01-07 00:45:30 UTC
Red Hat Product Errata RHBA-2022:4793 0 None None None 2022-06-22 16:03:57 UTC

Description Jean-Francois Beaudoin 2022-01-05 15:15:02 UTC
Description of problem:

Following this documentation:
https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/backing_up_and_restoring_the_undercloud_and_control_plane_nodes/assembly_backing-up-the-control-plane-nodes_br-undercloud-ctlplane

When running:
(undercloud) [stack@rhosp-inst ~]$ openstack overcloud backup --inventory /home/stack/tripleo-inventory.yaml


With 16.2:
~~~
TASK [backup-and-restore : Set the tripleo_backup_and_restore_mysql_container id] ***
ok: [rhosp16-1-ctl-0] => {"ansible_facts": {"tripleo_backup_and_restore_mysql_container": "xxxxxxxxxxxx"}, "changed": false}
ok: [rhosp16-1-ctl-1] => {"ansible_facts": {"tripleo_backup_and_restore_mysql_container": "xxxxxxxxxxxx"}, "changed": false}
ok: [rhosp16-1-ctl-2] => {"ansible_facts": {"tripleo_backup_and_restore_mysql_container": "xxxxxxxxxxxx"}, "changed": false}

TASK [backup-and-restore : Unpause mysql for backup if it is paused] ***********
skipping: [rhosp16-1-ctl-0] => {"changed": false, "skip_reason": "Conditional result was False"}
skipping: [rhosp16-1-ctl-1] => {"changed": false, "skip_reason": "Conditional result was False"}
skipping: [rhosp16-1-ctl-2] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [backup-and-restore : MySQL Grants backup] ********************************
changed: [rhosp16-1-ctl-0] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}
changed: [rhosp16-1-ctl-1] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}
changed: [rhosp16-1-ctl-2] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true}
~~~

With 16.2.1:
~~~
TASK [backup-and-restore : Set the tripleo_backup_and_restore_mysql_container id] ********************************************************************************************************************************
ok: [rhosp16-1-ctl-0] => {"ansible_facts": {"tripleo_backup_and_restore_mysql_container": "xxxxxxxxxxxx"}, "changed": false}
ok: [rhosp16-1-ctl-1] => {"ansible_facts": {"tripleo_backup_and_restore_mysql_container": "xxxxxxxxxxxx"}, "changed": false}
ok: [rhosp16-1-ctl-2] => {"ansible_facts": {"tripleo_backup_and_restore_mysql_container": "xxxxxxxxxxxx"}, "changed": false}

TASK [backup-and-restore : Unpause mysql for backup if it is paused] *********************************************************************************************************************************************
skipping: [rhosp16-1-ctl-0] => {"changed": false, "skip_reason": "Conditional result was False"}
skipping: [rhosp16-1-ctl-1] => {"changed": false, "skip_reason": "Conditional result was False"}
skipping: [rhosp16-1-ctl-2] => {"changed": false, "skip_reason": "Conditional result was False"}

TASK [backup-and-restore : Galera desync the MySQL node] *********************************************************************************************************************************************************
fatal: [rhosp16-1-ctl-0]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [rhosp16-1-ctl-1]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
fatal: [rhosp16-1-ctl-2]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"}
~~~

Version-Release number of selected component (if applicable):
OpenStack 16.2.1

How reproducible:
Every time.

Steps to Reproduce:
1.
2.
3.

Actual results:
Overcloud backup failing.

Expected results:
Being able to run `openstack overcloud backup --inventory /home/stack/tripleo-inventory.yaml` with 16.2.1

Additional info:
~~~
  * Container bundle set: galera-bundle [cluster.common.tag/openstack-mariadb:pcmklatest]:
    * galera-bundle-0   (ocf::heartbeat:galera):         Master rhosp16-1-ctl-1
    * galera-bundle-1   (ocf::heartbeat:galera):         Master rhosp16-1-ctl-0
    * galera-bundle-2   (ocf::heartbeat:galera):         Master rhosp16-1-ctl-2
~~~

Comment 3 Juan Larriba 2022-01-10 15:27:03 UTC
This issue is a race condition that was fixed after 16.2.1 release and, as such, will be included in 16.2.2.

Comment 8 Juan Larriba 2022-01-17 08:33:35 UTC
I am closing this issue as it is included in current release 16.2.1

Comment 18 errata-xmlrpc 2022-06-22 16:03:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 16.2.3 (Train)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:4793


Note You need to log in before you can comment on or make changes to this bug.