Bug 1916404

Summary: Puppet facts takes a long time to run in OSP 16.1
Product: Red Hat OpenStack Reporter: rosingh
Component: puppet-tripleoAssignee: OSP Team <rhos-maint>
Status: CLOSED CANTFIX QA Contact: Jeremy Agee <jagee>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.1 (Train)CC: aschultz, cjeanner, dwilde, ggrasza, hrybacki, jjoyce, jschluet, lbragsta, mircea.vutcovici, rcritten, slinaber, tvignaud
Target Milestone: zstreamKeywords: TestOnly, Triaged, ZStream
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-12-02 04:20:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1944227, 1950016, 1950307, 1950308, 1950331, 1950332    
Bug Blocks:    

Description rosingh 2021-01-14 17:31:20 UTC
Description of problem:

Running "puppet facts" from the cli on an overcloud node takes around 80 seconds to complete and "normal" would be taking 2-3 seconds.  

call is taking a really long time due to gathering ipa facts via /usr/share/openstack-puppet/modules/ipaclient/lib/facter/ipa_facts.rb caused by a DNS TXT lookup for "_kerberos" (which doesn't exist in DNS)

Looking in /usr/share/ruby/resolv.rb, the default timeout seems to be:
    # [ 5, second = 5 * 2 / nameserver_count, 2 * second, 4 * second ]

You can see the difference in timing when the ipa facts are pre-populated (the DNS lookup is skipped) or not:

[root@]# time FACTER_ipa_domain='' FACTER_ipa_server='' puppet facts >/dev/null

real    0m2.367s
user    0m1.746s
sys     0m0.392s

[root@]# time puppet facts >/dev/null

real    1m22.435s
user    0m1.735s
sys     0m0.428s

Comment 5 Alex Schultz 2021-01-15 19:01:46 UTC
In the recent versions of OSP, we're running the fact execution upfront and actually caching them. In general they aren't run as much as they used to be (it used to be a lot).  Since the issue seems to be in core ruby, it's likely we should kick this over to the ruby maintainers for RHEL.

Comment 12 Grzegorz Grasza 2021-05-25 07:47:48 UTC
This is will be fixed in rhel-8.4.0.z ruby-2.7.3-136.module+el8.4.0+10728+4c884998

https://bugzilla.redhat.com/show_bug.cgi?id=1952000

We probably need to target this to an OSP release for RHEL 8.4

Comment 13 Grzegorz Grasza 2021-05-25 07:52:10 UTC
There is another bug open for ruby 2.5 https://bugzilla.redhat.com/show_bug.cgi?id=1950332