Bug 1631674

Description of problem:
-----------------------
After controllers got upgraded to RHOS-14 with upgrade_steps_playbook.yaml next step is to run deploy_steps_playbook.yaml.

This fails:
...
 u'        "Warning: This method is deprecated, please use match expressions with Stdlib::Compat::Array instead. They are described at https://docs.puppet.com/puppet/latest/reference/lang_data_type.html#match-ex
pressions. at [\\"/etc/puppet/modules/tripleo/manifests/profile/pacemaker/database/mysql_bundle.pp\\", 133]:[\\"unknown\\", 1]", ',
 u'        "   (at /etc/puppet/modules/stdlib/lib/puppet/functions/deprecation.rb:28:in `deprecation\')", ',
 u'        "Warning: This method is deprecated, please use the stdlib validate_legacy function,", ',
 u'        "                    with Stdlib::Compat::Hash. There is further documentation for validate_legacy function in the README. at [\\"/etc/puppet/modules/tripleo/manifests/profile/base/database/mysql.pp\\
", 103]:[\\"unknown\\", 1]", ',
 u'        "Warning: ModuleLoader: module \'mysql\' has unresolved dependencies - it will only see those that are resolved. Use \'puppet module list --tree\' to see information about modules", ',
 u'        "                    with Stdlib::Compat::String. There is further documentation for validate_legacy function in the README. at [\\"/etc/puppet/modules/aodh/manifests/db/mysql.pp\\", 57]:[\\"/etc/pupp
et/modules/tripleo/manifests/profile/base/database/mysql.pp\\", 175]", ',
 u'        "Warning: ModuleLoader: module \'cinder\' has unresolved dependencies - it will only see those that are resolved. Use \'puppet module list --tree\' to see information about modules", ',
 u'        "Warning: ModuleLoader: module \'nova\' has unresolved dependencies - it will only see those that are resolved. Use \'puppet module list --tree\' to see information about modules", ',
 u'        "Warning: ModuleLoader: module \'openstacklib\' has unresolved dependencies - it will only see those that are resolved. Use \'puppet module list --tree\' to see information about modules", ',
 u'        "                    with Pattern[]. There is further documentation for validate_legacy function in the README. at [\\"/etc/puppet/modules/openstacklib/manifests/db/mysql/host_access.pp\\", 43]:", ',
 u'        "Error: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0]", ',
 u'        "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Exec[galera-ready]/returns: change from notrun to 0 failed: /usr/bin/clustercheck >/dev/null returned 1 instead of one of [0]",
 ',
 u'        "Error: Failed to apply catalog: Execution of \'/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, \'@\',Host) AS User FROM mysql.user\' returned 1: ERROR 2002 (HY000): Can\'t connect to local MySQL server through socket \'/var/lib/mysql/mysql.sock\' (2 \\"No such file or directory\\")", ',
 u'        "+ rc=1", ',


Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-tripleo-heat-templates-9.0.0-0.20180831204457.17bb71e.0rc1.el7ost.noarch

python2-tripleo-common-9.3.1-0.20180831204016.bb0582a.el7ost.noarch
puppet-tripleo-9.3.1-0.20180831202649.8ec6c86.el7ost.noarch
ansible-role-tripleo-modify-image-1.0.1-0.20180903052248.40521ee.el7ost.noarch
python-tripleoclient-10.5.1-0.20180901082351.6d7aa74.el7ost.noarch
openstack-tripleo-image-elements-9.0.0-0.20180831210308.2dc678a.el7ost.noarch
openstack-tripleo-common-containers-9.3.1-0.20180831204016.bb0582a.el7ost.noarch
python-tripleoclient-heat-installer-10.5.1-0.20180901082351.6d7aa74.el7ost.noarch
openstack-tripleo-puppet-elements-9.0.0-0.20180831205939.0641fdc.el7ost.noarch
openstack-tripleo-validations-9.3.1-0.20180831205305.fbfd253.el7ost.noarch
openstack-tripleo-ui-9.3.1-0.20180903112814.7ed74d3.el7ost.noarch
ansible-tripleo-ipsec-9.0.1-0.20180827143021.d2b9234.el7ost.noarch
openstack-tripleo-common-9.3.1-0.20180831204016.bb0582a.el7ost.noarch

How reproducible:
-----------------
100%

Steps to Reproduce:
-------------------
1. Upgrade UC to RHOS-14
2. Prepare OC nodes (images/repos)
3. Start Controller upgrade 

   openstack overcloud upgrade run \
        --stack overcloud \
        --roles Controller --playbook upgrade_steps_playbook.yaml
4. Restart openvswitch on controllers
5. Run deployment playbook:

   openstack overcloud upgrade run \
        --stack overcloud \
        --roles Controller --playbook deploy_steps_playbook.yaml

Actual results:
---------------
Upgrade fails because 2 galera services are in 'slave' state and 3d one is in Failed(blocked) state.

Expected results:
-----------------
Galera is successfully upgraded

Additional info:
----------------
Virtual setup: 3controllers + 2computes + 3ceph