Bug 1517962

Summary:	Refresh targets can grow unbounded causing Postgres InternalError
Product:	Red Hat CloudForms Management Engine	Reporter:	Satoe Imaishi <simaishi>
Component:	Providers	Assignee:	Adam Grare <agrare>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Matouš Mojžíš <mmojzis>
Severity:	urgent	Docs Contact:
Priority:	high
Version:	5.9.0	CC:	agrare, cpelland, gblomqui, jfrey, jhardy, mmojzis, obarenbo
Target Milestone:	GA
Target Release:	5.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	5.9.0.11	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1516401	Environment:
Last Closed:	2018-03-06 15:29:51 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	CFME Core	Target Upstream Version:
Embargoed:
Bug Depends On:	1516401
Bug Blocks:

Comment 2 CFME Bot 2017-11-29 20:24:52 UTC

New commit detected on ManageIQ/manageiq/gaprindashvili:
https://github.com/ManageIQ/manageiq/commit/d20081cd1cd829e03289734521bc37de6186b6a1

commit d20081cd1cd829e03289734521bc37de6186b6a1
Author:     Jason Frey <fryguy9>
AuthorDate: Mon Nov 27 12:03:04 2017 -0500
Commit:     Satoe Imaishi <simaishi>
CommitDate: Mon Nov 27 13:24:09 2017 -0500

    Merge pull request #16432 from agrare/add_back_ems_refresh_unique_target
    
    Unique EmsRefresh.refresh targets if there are over 1,000 targets
    (cherry picked from commit fdb09c20fdb778fb3ef0238044b2ce907f43e1ff)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1517962

 app/models/ems_refresh.rb | 1 +
 1 file changed, 1 insertion(+)

Comment 3 Matouš Mojžíš 2018-02-07 14:20:29 UTC

Hi,
what are the steps for reproducing this BZ?
Thanks

Comment 4 Adam Grare 2018-02-07 14:31:02 UTC

If you disable your inventory role but keep your events role enabled so that it keeps queueing refreshes that should cause the targets to pile up prior to this change.

Comment 5 Matouš Mojžíš 2018-02-08 14:12:32 UTC

I was not able to stop Provider inventory role from the UI.
Is there a way to do it from rails c?
Also one provider is enough to reproduce this BZ or is better to have more of them?

Comment 6 Adam Grare 2018-02-15 14:35:56 UTC

> I was not able to stop Provider inventory role from the UI.

You weren't able to find where it was or you couldn't turn it off?

It is under Configuration -> Server -> Server Control, the role is called 'Provider Inventory'

You can also verify from rails console:
ems = ExtManagementSystem.first; 2_000.times { ems.refresh_ems }; MiqQueue.find_by(:class_name => "EmsRefresh", :method_name => "refresh").data.count

This should confirm that the number of duplicate targets never grows above 1,000

Comment 7 Matouš Mojžíš 2018-02-20 13:20:49 UTC

I tried to do it in Diagnostics where it doesn't work as it should.
I stopped the server role last Thursday and still don't see any errors in evm.log. (tried to grep older logs too)

Comment 8 Matouš Mojžíš 2018-02-26 12:47:01 UTC

Verified in 5.9.0.21. Queue is erased when having more than 1k entries.