Bug 1235454

Summary: Mariadb service enabled by systemd in HA deployment
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: openstack-tripleo-heat-templatesAssignee: Giulio Fidente <gfidente>
Status: CLOSED ERRATA QA Contact: Marius Cornea <mcornea>
Severity: high Docs Contact:
Priority: high    
Version: 7.0 (Kilo)CC: calfonso, dmacpher, mburns, ohochman, rhel-osp-director-maint, rrosa
Target Milestone: beta   
Target Release: Director   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-0.8.6-17.el7ost Doc Type: Bug Fix
Doc Text:
The mariadb service started on boot, which caused Pacemaker's mariadb resource to fail after a reboot. This fix disables the mariadb service from automatically starting on boot. This means mariadb is fully controlled as a Pacemaker resource.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-08-05 13:55:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marius Cornea 2015-06-24 20:52:21 UTC
Description of problem:
Mariadb service is enabled by systemd in HA deployment with 3 controller nodes. This causes a node cluster resources not come up after a soft reboot.

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Deploy 3 controller node overcloud
2. Soft reboot one of the controllers.
3.

Actual results:
Failed actions:
    openstack-heat-engine_start_0 on overcloud-controller-1 'not running' (7): call=390, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:30:45 2015', queued=2001ms, exec=3ms
    openstack-cinder-volume_start_0 on overcloud-controller-1 'not running' (7): call=352, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:29:52 2015', queued=2001ms, exec=2ms
    openstack-heat-engine_start_0 on overcloud-controller-2 'not running' (7): call=387, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:30:45 2015', queued=2001ms, exec=3ms
    openstack-cinder-volume_start_0 on overcloud-controller-2 'not running' (7): call=343, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:29:47 2015', queued=2001ms, exec=7ms
    openstack-nova-scheduler_start_0 on overcloud-controller-0 'not running' (7): call=218, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:29:38 2015', queued=2002ms, exec=8ms
    openstack-heat-engine_start_0 on overcloud-controller-0 'not running' (7): call=254, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:30:45 2015', queued=2000ms, exec=4ms
    openstack-nova-consoleauth_start_0 on overcloud-controller-0 'not running' (7): call=220, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:29:38 2015', queued=2001ms, exec=2ms
    openstack-cinder-api_start_0 on overcloud-controller-0 'not running' (7): call=223, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:29:38 2015', queued=2001ms, exec=2ms
    neutron-server_start_0 on overcloud-controller-0 'not running' (7): call=224, status=complete, exit-reason='none', last-rc-change='Wed Jun 24 16:29:38 2015', queued=2000ms, exec=2ms


Expected results:
Resources would come up after reboot. 

Additional info:

Comment 3 Marios Andreou 2015-06-25 13:03:19 UTC
I just verified the fix in the review @ https://review.openstack.org/#/c/195353
applied and roles reloaded etc, deploying like --control-scale 3

Once create complete, I nova reboot controller-0-uuid and then checked it, seems all services fine to me (pcs status is green) and for example heat-engine since it is first in the description above:

[root@overcloud-controller-0 heat-admin]# service openstack-heat-engine status -l
Redirecting to /bin/systemctl status  -l openstack-heat-engine.service
openstack-heat-engine.service - Cluster Controlled openstack-heat-engine
   Loaded: loaded (/usr/lib/systemd/system/openstack-heat-engine.service; disabled)
  Drop-In: /run/systemd/system/openstack-heat-engine.service.d
           └─50-pacemaker.conf
   Active: active (running) since Thu 2015-06-25 08:59:40 EDT; 50s ago
 Main PID: 11455 (heat-engine)
   CGroup: /system.slice/openstack-heat-engine.service
           └─11455 /usr/bin/python /usr/bin/heat-engine

Jun 25 08:59:40 overcloud-controller-0.localdomain systemd[1]: Starting Cluster Controlled openstack-heat-engine...
Jun 25 08:59:40 overcloud-controller-0.localdomain systemd[1]: Started Cluster Controlled openstack-heat-engine.
Jun 25 08:59:41 overcloud-controller-0.localdomain heat-engine[11455]: /usr/lib/python2.7/site-packages/heat/engine/environment.py:263: UserWarning: OS::Heat::CWLiteAlarm is deprecated, use OS::Ceilometer::Alarm instead.
Jun 25 08:59:41 overcloud-controller-0.localdomain heat-engine[11455]: info.value.support_status.message))
Jun 25 08:59:41 overcloud-controller-0.localdomain heat-engine[11455]: /usr/lib/python2.7/site-packages/heat/engine/environment.py:263: UserWarning: The HARestarter resource type is unsupported and will be removed in a future release of Heat, once it has support for auto-healing any type of resource. Note that HARestarter does *not* actually restart servers - it deletes and then recreates them. It also does the same to all dependent resources, and may therefore exhibit unexpected and undesirable behaviour. Avoid.
Jun 25 08:59:41 overcloud-controller-0.localdomain heat-engine[11455]: info.value.support_status.message))
Jun 25 08:59:41 overcloud-controller-0.localdomain heat-engine[11455]: /usr/lib/python2.7/site-packages/heat/engine/environment.py:263: UserWarning: RouterGateway resource is deprecated and should not be used. Instead use the `external_gateway_info` property in the router resource to set up the gateway.
Jun 25 08:59:41 overcloud-controller-0.localdomain heat-engine[11455]: info.value.support_status.message))

Comment 5 Omri Hochman 2015-06-26 21:29:30 UTC
Verified openstack-tripleo-heat-templates-0.8.6-19.el7ost.noarch :

After HA deployment finish --> ssh controller --> [heat-admin@overcloud-controller-1 ~]$  systemctl status mariadb
mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled)
   Active: inactive (dead)

Comment 7 errata-xmlrpc 2015-08-05 13:55:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549