Bug 1374855

Summary:	rhel-osp-director: 9.0 updated from 9.0GA overcloud with ipv6: cinder create returns "ERROR: Gateway Time-out (HTTP 504)"
Product:	Red Hat OpenStack	Reporter:	Alexander Chuzhoy <sasha>
Component:	rhosp-director	Assignee:	Angus Thomas <athomas>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Omri Hochman <ohochman>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	9.0 (Mitaka)	CC:	aschultz, dbecker, emacchi, jjoyce, mburns, mcornea, morazi, rhel-osp-director-maint, sasha, tshefi, tvignaud
Target Milestone:	async	Keywords:	Triaged, ZStream
Target Release:	9.0 (Mitaka)
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-11-23 13:58:50 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Alexander Chuzhoy 2016-09-09 20:25:44 UTC

rhel-osp-director:   9.0 updated from 9.0GA overcloud with ipv6:  cinder create returns "ERROR: Gateway Time-out (HTTP 504)"

[root@overcloud-cephstorage-0 ~]# systemctl|grep ceph
  ceph.service                                                      loaded active exited    LSB: Start Ceph distributed file system daemons at boot time
  run-16950.service                                                 loaded active exited    /bin/bash -c ulimit -n 32768; TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=128MB /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf --cluster ceph -f
  run-24865.service                                                 loaded active exited    /usr/bin/bash -c ulimit -n 32768; TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=128MB /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf --cluster ceph -f
  run-25305.service                                                 loaded active running   /usr/bin/bash -c ulimit -n 32768; TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=128MB /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf --cluster ceph -f



Environment:
openstack-tripleo-heat-templates-liberty-2.0.0-33.el7ost.noarch
instack-undercloud-4.0.0-13.el7ost.noarch
openstack-puppet-modules-8.1.8-2.el7ost.noarch
openstack-tripleo-heat-templates-2.0.0-33.el7ost.noarch


Steps to reproduce:
1. Deploy 9.0GA with:
openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --swift-storage-scale 0 --block-storage-scale 0 --neutron-disable-tunneling --neutron-network-type vlan --neutron-network-vlan-ranges datacentre:118:143 --neutron-bridge-mappings datacentre:br-ex --ntp-server clock.redhat.com --timeout 90 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation-v6.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans-v6.yaml -e network-environment.yaml -e /home/stack/ssl-heat-templates/environments/enable-tls.yaml -e /home/stack/ssl-heat-templates/environments/inject-trust-anchor.yaml --ceph-storage-scale 1

2. Populate the setup with objects.
3. Update to latest 9.0
4. Attempt to create a cinder volume
Result:
[stack@instack ~]$ cinder create 1
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 2001:db8:fd00:1000::10 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 2001:db8:fd00:1000::10 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning

ERROR: Gateway Time-out (HTTP 504)


[stack@instack ~]$ cinder list
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 2001:db8:fd00:1000::10 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 2001:db8:fd00:1000::10 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
+--------------------------------------+----------+------+------+-------------+----------+--------------------------------------+
|                  ID                  |  Status  | Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+----------+------+------+-------------+----------+--------------------------------------+
| b0c5c0e3-1ca2-4cc1-bd7f-60771295e832 | creating |  -   |  1   |      -      |  false   |                                      |
| e31e192b-4e3b-4a3b-878a-784684dea720 |  in-use  |  -   |  1   |      -      |  false   | 95c286ad-f9ee-4b48-94f5-88a389cf81b4 |
+--------------------------------------+----------+------+------+-------------+----------+--------------------------------------+



Checking the status of ceph on ceph node:
[root@overcloud-cephstorage-0 ~]# systemctl|grep ceph
  ceph.service                                                      loaded active exited    LSB: Start Ceph distributed file system daemons at boot time
  run-16950.service                                                 loaded active exited    /bin/bash -c ulimit -n 32768; TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=128MB /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf --cluster ceph -f
  run-24865.service                                                 loaded active exited    /usr/bin/bash -c ulimit -n 32768; TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=128MB /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf --cluster ceph -f
  run-25305.service                                                 loaded active running   /usr/bin/bash -c ulimit -n 32768; TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=128MB /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf --cluster ceph -f

Comment 3 Emilien Macchi 2017-01-31 20:34:05 UTC

is this one still valid?

Comment 4 Alexander Chuzhoy 2017-01-31 21:47:25 UTC

I had a successful run of the same setup on Sep/12.
Didn't test this scenario recently.

Comment 5 Tzach Shefi 2017-11-23 13:58:50 UTC

I've just tested this on osp11 IPv6 + ceph update from one release to latest. 
Created some instances and volumes before update as well as after. 
everything works OK create/attach/backup/snap of volumes all work. 

Closing bug as it might have been a fluke.