Bug 1634704

Summary: [RHOS14] Stack update fails with "Property error: resources.Messaging.properties.key_name: Unable to establish connection to https://192.168.24.2:13774/v2.1/os-keypairs/default: ('Connection aborted.', BadStatusLine("''",)"
Product: Red Hat OpenStack Reporter: Artem Hrechanychenko <ahrechan>
Component: openstack-tripleo-heat-templatesAssignee: Emilien Macchi <emacchi>
Status: CLOSED WORKSFORME QA Contact: Gurenko Alex <agurenko>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 14.0 (Rocky)CC: agurenko, ahrechan, dbecker, jcoufal, m.alshafay, mburns, mcornea, morazi, ohochman
Target Milestone: betaKeywords: AutomationBlocker, Regression, Triaged
Target Release: 14.0 (Rocky)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-12 18:31:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/var/log/containers logs none

Description Artem Hrechanychenko 2018-10-01 11:39:40 UTC 
Description of problem:
While testing stack update with composable roles and blacklisting of networker nodes:

(undercloud) [stack@undercloud-0 ~]$ cat blacklist.yaml 
parameter_defaults:
  DeploymentServerBlacklist:
    - networker-0
    - networker-1


2018-10-01 11:21:12Z [overcloud.ControllerOpenstack]: UPDATE_COMPLETE  state changed

 Stack overcloud/1b4fee37-d23f-448e-a857-3b821697a934 UPDATE_FAILED 

overcloud.Messaging.1:
  resource_type: OS::TripleO::Messaging
  physical_resource_id: df5a6a8e-f449-483b-bd4a-78ec1259a39d
  status: UPDATE_FAILED
  status_reason: |
    resources[1]: Property error: resources.Messaging.properties.key_name: Unable to establish connection to https://192.168.24.2:13774/v2.1/os-keypairs/default: ('Connection aborted.', BadStatusLine("''",))
Heat Stack update failed.
Heat Stack update failed.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 Artem Hrechanychenko 2018-10-01 12:03:15 UTC 
Version-Release number of selected component (if applicable):
OSP14, puddle - 2018-09-27.3

How reproducible:


Steps to Reproduce:
1.Deploy OSP14 with composable roles - 3cont_1comp_3ceph_3db_2net_3msg
2.create blacklist.yaml with 2 networker nodes
3.trigger stack-update with -e blacklist.yaml 

Actual results:
Stack updated

Expected results:
Update_Failed

Additional info:
Comment 2 Artem Hrechanychenko 2018-10-01 12:04:20 UTC 
Created attachment 1489031 [details]
/var/log/containers logs
Comment 3 Artem Hrechanychenko 2018-10-01 12:18:44 UTC 
The reports should be available here: http://rhos-release.virt.bos.redhat.com/log/bz1634704
Comment 5 Emilien Macchi 2018-10-04 19:00:15 UTC 
Hey Artem, can you please give me access to this env, I don't have enough HW to reproduce myself.
I looked at sosreport and I didn't see ERROR in /var/log/containers on the Undercloud so I need to live debug it.
Comment 10 Marius Cornea 2018-10-12 18:31:18 UTC 
I wasn't able to reproduce the issue so I am closing this bug for now. Please re-open if it reproduces.