Bug 1298391

Summary:	In ipv6 deployment pacemaker vip resources fail to create
Product:	Red Hat OpenStack	Reporter:	Marius Cornea <mcornea>
Component:	openstack-puppet-modules	Assignee:	Emilien Macchi <emacchi>
Status:	CLOSED ERRATA	QA Contact:	yeylon <yeylon>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	7.0 (Kilo)	CC:	dnavale, emacchi, gchamoul, gdubreui, kbasil, mburns, mcornea, rhel-osp-director-maint, srevivo, yeylon
Target Milestone:	z4	Keywords:	ZStream
Target Release:	7.0 (Kilo)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-puppet-modules-2015.1.8-40.el7ost	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-02-18 16:44:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Marius Cornea 2016-01-13 23:53:04 UTC

Description of problem:
In ipv6 deployment pacemaker vip resources fail to create:

Error: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:4000:f816:3eff:fe63:4103', ':' is not a valid character for a resource name

Version-Release number of selected component (if applicable):
I'm doing the test following the instructions in:
https://etherpad.openstack.org/p/tripleo-ipv6-support
and enabling pacemaker by passing an additional $THT/environments/puppet-pacemaker.yaml environment file

How reproducible:
100%

Steps to Reproduce:

1. Apply workarounds for BZ#1295986 and BZ#1297850
2. Deploy ipv6 enabled overcloud

Actual results:
Deployment fails

[2016-01-13 23:40:01,199] (heat-config) [INFO] Error: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:4000:f816:3eff:fe63:4103', ':' is not a valid character for a resource name
Error: /Stage[main]/Main/Pacemaker::Resource::Ip[storage_mgmt_vip]/Pcmk_resource[ip-fd00:fd00:fd00:4000:f816:3eff:fe63:4103]/ensure: change from absent to present failed: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:4
Error: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:2000:f816:3eff:fe9a:4a28', ':' is not a valid character for a resource name
Error: /Stage[main]/Main/Pacemaker::Resource::Ip[redis_vip]/Pcmk_resource[ip-fd00:fd00:fd00:2000:f816:3eff:fe9a:4a28]/ensure: change from absent to present failed: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:2000:f81
Error: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:2000:f816:3eff:fef3:38fc', ':' is not a valid character for a resource name
Error: /Stage[main]/Main/Pacemaker::Resource::Ip[internal_api_vip]/Pcmk_resource[ip-fd00:fd00:fd00:2000:f816:3eff:fef3:38fc]/ensure: change from absent to present failed: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:2
Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/my
Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Error: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:3000:f816:3eff:fef9:adf1', ':' is not a valid character for a resource name
Error: /Stage[main]/Main/Pacemaker::Resource::Ip[storage_vip]/Pcmk_resource[ip-fd00:fd00:fd00:3000:f816:3eff:fef9:adf1]/ensure: change from absent to present failed: pcs create failed: Error: invalid resource name 'ip-fd00:fd00:fd00:3000:f
Error: pcs create failed: Error: invalid resource name 'ip-2001:db8:fd00:1000:f816:3eff:fed3:1da7', ':' is not a valid character for a resource name
Error: /Stage[main]/Main/Pacemaker::Resource::Ip[public_vip]/Pcmk_resource[ip-2001:db8:fd00:1000:f816:3eff:fed3:1da7]/ensure: change from absent to present failed: pcs create failed: Error: invalid resource name 'ip-2001:db8:fd00:1000:f816
Warning: /Stage[main]/Main/Pacemaker::Constraint::Base[internal_api_vip-then-haproxy]/Exec[Creating order constraint internal_api_vip-then-haproxy]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Colocation[storage_vip-with-haproxy]/Pcmk_constraint[colo-ip-fd00:fd00:fd00:3000:f816:3eff:fef9:adf1-haproxy-clone]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Colocation[storage_mgmt_vip-with-haproxy]/Pcmk_constraint[colo-ip-fd00:fd00:fd00:4000:f816:3eff:fe63:4103-haproxy-clone]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Colocation[public_vip-with-haproxy]/Pcmk_constraint[colo-ip-2001:db8:fd00:1000:f816:3eff:fed3:1da7-haproxy-clone]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Base[storage_mgmt_vip-then-haproxy]/Exec[Creating order constraint storage_mgmt_vip-then-haproxy]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Colocation[redis_vip-with-haproxy]/Pcmk_constraint[colo-ip-fd00:fd00:fd00:2000:f816:3eff:fe9a:4a28-haproxy-clone]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Base[public_vip-then-haproxy]/Exec[Creating order constraint public_vip-then-haproxy]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Base[redis_vip-then-haproxy]/Exec[Creating order constraint redis_vip-then-haproxy]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Base[storage_vip-then-haproxy]/Exec[Creating order constraint storage_vip-then-haproxy]: Skipping because of failed dependencies
Warning: /Stage[main]/Main/Pacemaker::Constraint::Colocation[internal_api_vip-with-haproxy]/Pcmk_constraint[colo-ip-fd00:fd00:fd00:2000:f816:3eff:fef3:38fc-haproxy-clone]: Skipping because of failed dependencies

Expected results:
Deployment succeeds.

Comment 1 Emilien Macchi 2016-01-14 04:17:53 UTC

That's weird, puppet-pacemaker create the resource without ":" thanks to this REGEX:
$resource_name = regsubst($ip_address, '(:)', '.', 'G')

Can you confirm your version of OPM please?

Can you confirm you have this patch in our OPM version (in puppet-pacemaker)

https://github.com/redhat-openstack/puppet-pacemaker/commit/e9a8f0d048c1bbc86a4bb1ee81417da3afe6673e


Thanks

Comment 2 Emilien Macchi 2016-01-14 04:23:00 UTC

I checked OPM and the patch is not backported downstream. We're taking care of that.

Comment 3 Marius Cornea 2016-01-14 10:02:46 UTC

After manually applying the patch mentioned in comment#1 the resources got created:

[root@overcloud-controller-0 heat-admin]# pcs status
Cluster name: tripleo_cluster
Last updated: Thu Jan 14 10:01:32 2016		Last change: Thu Jan 14 09:36:22 2016 by root via cibadmin on overcloud-controller-0
Stack: corosync
Current DC: overcloud-controller-0 (version 1.1.13-10.el7-44eb2dd) - partition with quorum
1 node and 7 resources configured

Online: [ overcloud-controller-0 ]

Full list of resources:

 ip-2001.db8.fd00.1000.f816.3eff.fe3e.50ae	(ocf::heartbeat:IPaddr2):	Stopped
 ip-192.0.2.6	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0
 Clone Set: haproxy-clone [haproxy]
     Started: [ overcloud-controller-0 ]
 ip-fd00.fd00.fd00.2000.f816.3eff.fe35.5071	(ocf::heartbeat:IPaddr2):	Stopped
 ip-fd00.fd00.fd00.2000.f816.3eff.fe3e.a57	(ocf::heartbeat:IPaddr2):	Stopped
 ip-fd00.fd00.fd00.4000.f816.3eff.fef8.ad5a	(ocf::heartbeat:IPaddr2):	Stopped

Comment 5 Marius Cornea 2016-01-19 10:44:24 UTC

[root@overcloud-controller-0 ~]# rpm -qa | grep puppet-modules
openstack-puppet-modules-2015.1.8-41.el7ost.noarch
[root@overcloud-controller-0 ~]# pcs status | grep ip-
 ip-192.0.2.6	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0
 ip-2001.db8.fd00.1000.f816.3eff.feb3.25fb	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0
 ip-fd00.fd00.fd00.4000.f816.3eff.fe88.dbfc	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0
 ip-fd00.fd00.fd00.2000.f816.3eff.feeb.38af	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0
 ip-fd00.fd00.fd00.3000.f816.3eff.fe2e.5873	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0
 ip-fd00.fd00.fd00.2000.f816.3eff.fe4a.871	(ocf::heartbeat:IPaddr2):	Started overcloud-controller-0

Comment 9 errata-xmlrpc 2016-02-18 16:44:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0265.html