Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1685495

Summary: TLS everywhere failing with (RPC failed at server. change collided with another change)
Product: Red Hat OpenStack Reporter: Juan Antonio Osorio <josorior>
Component: openstack-tripleo-heat-templatesAssignee: Raildo Mascena de Sousa Filho <rmascena>
Status: CLOSED CURRENTRELEASE QA Contact: Pavan <pkesavar>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: akaiser, alee, ggrimaux, hrybacki, ltamagno, mburns, nchandek, rmascena
Target Milestone: zstreamKeywords: Reopened, TestOnly, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-8.3.1-44.el7ost, puppet-tripleo-8.4.1-10.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1715167 (view as bug list) Environment:
Last Closed: 2019-12-11 17:06:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1715167, 1715168, 1728930    

Description Juan Antonio Osorio 2019-03-05 10:40:51 UTC 
Description of problem:

this presents itself with the following error from certmonger:

"Error: /Stage[main]/Tripleo::Profile::Base::Certmonger_user/Tripleo::Certmonger::Haproxy[haproxy-internal_api]/Certmonger_certificate[haproxy-internal_api-cert]: Could not evaluate: Could not get certificate: Server at https://ipa.ooo.test/ipa/xml failed request, will retry: 4201 (RPC failed at server. change collided with another change).",

e.g. http://logs.rdoproject.org/98/604298/257/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/3e0dcec/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

Here are the logs for the FreeIPA host: http://logs.rdoproject.org/98/604298/257/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/3e0dcec/logs/supplemental/var/log/

The issue is caused because multiple hosts are trying to write to the same certificate entry in LDAP at the same time. So this ultimately is a concurrency problem.

How reproducible:

It's only reproduceable in some cases (depends on how fast the nodes are).

Additional info:

We should ideally be writing the certificates to unique entries in LDAP.
Comment 2 Harry Rybacki 2019-06-07 11:34:50 UTC 
Half the upstream reviews have merged. Moving bug to ASSIGNED.
Comment 4 Harry Rybacki 2019-06-14 10:28:08 UTC 
Up and downstream patches have merged. Builds created for openstack-tripleo-heat-templates and puppet-tripleo. Moving RHBZ to MODIFIED.
Comment 5 Lon Hohberger 2019-07-11 10:41:31 UTC 
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.3.1-54.el7ost.  This build is available now.
Comment 6 Lon Hohberger 2019-07-11 10:41:32 UTC 
According to our records, this should be resolved by puppet-tripleo-8.4.1-14.el7ost.  This build is available now.
Comment 7 Harry Rybacki 2019-07-29 13:05:02 UTC 
*** Bug 1728930 has been marked as a duplicate of this bug. ***