Bug 1685495 - TLS everywhere failing with (RPC failed at server. change collided with another change)
Summary: TLS everywhere failing with (RPC failed at server. change collided with anoth...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: zstream
: 13.0 (Queens)
Assignee: Raildo Mascena de Sousa Filho
QA Contact: Pavan
URL:
Whiteboard:
: 1728930 (view as bug list)
Depends On:
Blocks: 1715167 1715168 1728930
TreeView+ depends on / blocked
 
Reported: 2019-03-05 10:40 UTC by Juan Antonio Osorio
Modified: 2019-12-11 17:06 UTC (History)
8 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.3.1-44.el7ost, puppet-tripleo-8.4.1-10.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1715167 (view as bug list)
Environment:
Last Closed: 2019-12-11 17:06:04 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1818513 0 None None None 2019-03-05 10:40:50 UTC
OpenStack gerrit 640813 0 'None' MERGED Request certificate for using host service principals 2020-05-06 04:05:08 UTC
OpenStack gerrit 640948 0 'None' MERGED certmonger: Add dnsnames parameter to redis cert request 2020-05-06 04:05:08 UTC

Description Juan Antonio Osorio 2019-03-05 10:40:51 UTC
Description of problem:

this presents itself with the following error from certmonger:

"Error: /Stage[main]/Tripleo::Profile::Base::Certmonger_user/Tripleo::Certmonger::Haproxy[haproxy-internal_api]/Certmonger_certificate[haproxy-internal_api-cert]: Could not evaluate: Could not get certificate: Server at https://ipa.ooo.test/ipa/xml failed request, will retry: 4201 (RPC failed at server. change collided with another change).",

e.g. http://logs.rdoproject.org/98/604298/257/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/3e0dcec/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

Here are the logs for the FreeIPA host: http://logs.rdoproject.org/98/604298/257/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/3e0dcec/logs/supplemental/var/log/

The issue is caused because multiple hosts are trying to write to the same certificate entry in LDAP at the same time. So this ultimately is a concurrency problem.

How reproducible:

It's only reproduceable in some cases (depends on how fast the nodes are).

Additional info:

We should ideally be writing the certificates to unique entries in LDAP.

Comment 2 Harry Rybacki 2019-06-07 11:34:50 UTC
Half the upstream reviews have merged. Moving bug to ASSIGNED.

Comment 4 Harry Rybacki 2019-06-14 10:28:08 UTC
Up and downstream patches have merged. Builds created for openstack-tripleo-heat-templates and puppet-tripleo. Moving RHBZ to MODIFIED.

Comment 5 Lon Hohberger 2019-07-11 10:41:31 UTC
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.3.1-54.el7ost.  This build is available now.

Comment 6 Lon Hohberger 2019-07-11 10:41:32 UTC
According to our records, this should be resolved by puppet-tripleo-8.4.1-14.el7ost.  This build is available now.

Comment 7 Harry Rybacki 2019-07-29 13:05:02 UTC
*** Bug 1728930 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.