Bug 1367494

Summary: M/N upgrade fail on stopping openstack-core.
Product: Red Hat OpenStack Reporter: Sofer Athlan-Guyot <sathlang>
Component: rhosp-directorAssignee: Angus Thomas <athomas>
Status: CLOSED NOTABUG QA Contact: Omri Hochman <ohochman>
Severity: unspecified Docs Contact:
Priority: low    
Version: 10.0 (Newton)CC: apevec, dbecker, jcoufal, lhh, mburns, morazi, rhel-osp-director-maint, srevivo
Target Milestone: ---Keywords: Triaged
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-05 08:04:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Sofer Athlan-Guyot 2016-08-16 14:23:13 UTC
Description of problem:

When doing the major upgrade from mitaka to newton, the upgrade fails on:

  pcs resource disable openstack-core

The openstack-gnocchi-metricd service doesn't stop on controller-{1,2}

How reproducible:  reproduced only once yet


Steps to Reproduce:
1.  get https://review.openstack.org/#/c/354713/
2.  pin opensvswitch version (https://bugzilla.redhat.com/show_bug.cgi?id=1364540)
3. do the upgrade 

Actual results:

Failed Actions:
* openstack-gnocchi-metricd_stop_0 on overcloud-controller-1 'OCF_TIMEOUT' (198): call=360, status=Timed Out, exitreason='none',
    last-rc-change='Tue Aug 16 11:57:01 2016', queued=0ms, exec=199991ms
* openstack-gnocchi-metricd_stop_0 on overcloud-controller-2 'OCF_TIMEOUT' (198): call=350, status=Timed Out, exitreason='none',
    last-rc-change='Tue Aug 16 11:57:08 2016', queued=0ms, exec=199991ms

On controller 1:
2016-08-12 13:32:36.265 24853 ERROR gnocchi.cli [-] Unable to initialize storage: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-08-12 13:32:36.271 24843 ERROR gnocchi.cli [-] Unable to initialize storage: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-08-12 13:32:36.274 24854 ERROR gnocchi.cli [-] Unable to initialize storage: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)
2016-08-12 13:32:37.322 24853 ERROR gnocchi.cli [-] Unable to initialize storage: Unable to start coordinator: Error while reading from socket: ('Connection closed by server.',)



Expected results:


Additional info:

Comment 2 Sofer Athlan-Guyot 2016-08-25 13:59:04 UTC
Seems like a very slow systemctl restart command on a vm whith timeouts.

Comment 3 Sofer Athlan-Guyot 2016-08-29 21:15:20 UTC
Could not reproduce it yet.  Waiting a little longer before closing it.

Comment 4 Jaromir Coufal 2016-10-04 19:40:10 UTC
OK, Sofer, please close if not reproducable.

Comment 5 Sofer Athlan-Guyot 2016-10-05 08:04:34 UTC
Could not reproduce this problem.