Bug 1718012

Summary:	[RFE] Add a hard limit of 100 items to restrict any fact child-hash/array
Product:	Red Hat Satellite	Reporter:	sthirugn <sthirugn>
Component:	Fact	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED ERRATA	QA Contact:	Radovan Drazny <rdrazny>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.4.2	CC:	bkearney, egolov, ehelms, inecas, mhulan, sshtein
Target Milestone:	6.8.0	Keywords:	FieldEngineering, FutureFeature, Performance, PrioBumpField, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	foreman-1.24.0	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 12:58:39 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description sthirugn@redhat.com 2019-06-06 16:58:45 UTC

Description of problem:
[RFE] Add a hard limit of 100 items to restrict any fact child-hash/array

Version-Release number of selected component (if applicable):
Satellite 6.4.2

How reproducible:
At customer environment for now.

Steps to Reproduce:
1. We have seen situations at customer environments where facts like disks* has hundreds or thousands of child items which bloat the facts associated tables in the db.

Actual results:
As explained above

Expected results:
lzap recommended that adding a limit of 100 items for any child-hash/array will help alleviate this kind of issues in the future.  If the number of items > 100, just drop the hash/array completely and write a warning to logs.

Additional info:

Comment 3 Lukas Zapletal 2019-06-07 09:54:06 UTC

Created redmine issue https://projects.theforeman.org/issues/26984 from this bug

Comment 7 Bryan Kearney 2019-08-21 08:01:53 UTC

Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/26984 has been resolved.

Comment 9 Radovan Drazny 2020-08-17 12:59:57 UTC

Tested on Satellite 6.8 Snap 10. Created some dummy structured facts in "/etc/puppetlabs/facter/facts.d". If the there is a subhash with number of items exceeding 100, the hash is dropped during a import with the following error message: 

2020-08-17T08:16:32 [I|app|1e18fd8c]   Parameters: {"facts"=>"[FILTERED]", "name"=>"ibm-x3250m4-16.rhts.eng.bos.redhat.com", "certname"=>"ibm-x3250m4-16.rhts.eng.bos.redhat.com", "apiv"=>"v2", "host"=>{"certname"=>"ibm-x3250m4-16.rhts.eng.bos.redhat.com", "name"=>"ibm-x3250m4-16.rhts.eng.bos.redhat.com"}}
2020-08-17T08:16:33 [W|app|1e18fd8c] Some subtrees exceeded 100 limit of facts, dropped 120 keys
2020-08-17T08:16:35 [I|app|1e18fd8c] Import facts for 'ibm-x3250m4-16.rhts.eng.bos.redhat.com' completed. Added: 428, Updated: 12, Deleted 0 facts

Change to the "Administer->Settings->Provisioning->Maximum structured facts" setting is respected during the import. 

The filtering is enabled only for the first level of subfacts, meaning that 

parentfact:
    subfact1: fact1
    subfact2: fact2
    <...>
    subfact101: fact101

will fail to import, while 

parentfact:
    middlefact:
      subfact1: fact1
      subfact2: fact2
      <...>
      subfact101: fact101

will be imported successfully and not filtered out. According to the last line in the comment https://github.com/theforeman/foreman/pull/6850#issue-289611209  this is expected behavior.

Comment 12 errata-xmlrpc 2020-10-27 12:58:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366