Bug 1754071

Summary: Refresh of OpenShift provider in CloudForms happen to panic apiserver
Product: Red Hat CloudForms Management Engine Reporter: Satoe Imaishi <simaishi>
Component: ProvidersAssignee: Oved Ourfali <oourfali>
Status: CLOSED ERRATA QA Contact: juwatts
Severity: high Docs Contact: Red Hat CloudForms Documentation <cloudforms-docs>
Priority: medium    
Version: 5.10.4CC: agrare, cben, ckoep, dmetzger, jfrey, jhardy, jprause, lsmola, mfeifer, mshriver, nansari, niroy, obarenbo, oourfali, simaishi, tuado
Target Milestone: GAKeywords: ZStream
Target Release: 5.10.11   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.10.11.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1722808 Environment:
Last Closed: 2019-10-09 13:51:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Container Management Target Upstream Version:
Embargoed:
Bug Depends On: 1722808    
Bug Blocks:    

Comment 2 CFME Bot 2019-09-20 18:36:46 UTC
New commits detected on ManageIQ/manageiq-providers-openshift/hammer:

https://github.com/ManageIQ/manageiq-providers-openshift/commit/8d509c0a6be052933c5d92c72cc3184ee84ae3c6
commit 8d509c0a6be052933c5d92c72cc3184ee84ae3c6
Author:     Beni Cherniavsky-Paskin <cben>
AuthorDate: Fri Sep 20 03:12:46 2019 -0400
Commit:     Beni Cherniavsky-Paskin <cben>
CommitDate: Fri Sep 20 03:12:46 2019 -0400

    Merge pull request #145 from agrare/fix_test_objects_generation

    Fix test_objects generation

    (cherry picked from commit 4da9f1d8990e81b9f08715372ee496a9ec9797ac)

    https://bugzilla.redhat.com/show_bug.cgi?id=1754071

 spec/vcr_cassettes/manageiq/providers/openshift/container_manager/test_objects_record.sh | 2 +-
 spec/vcr_cassettes/manageiq/providers/openshift/container_manager/test_objects_template.yml | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)


https://github.com/ManageIQ/manageiq-providers-openshift/commit/073749850c7e9cba80aa7c9e248cf1e2df36f5be
commit 073749850c7e9cba80aa7c9e248cf1e2df36f5be
Author:     Beni Cherniavsky-Paskin <cben>
AuthorDate: Fri Sep 20 09:59:07 2019 -0400
Commit:     Beni Cherniavsky-Paskin <cben>
CommitDate: Fri Sep 20 09:59:07 2019 -0400

    Merge pull request #146 from agrare/updates_for_api_chunking

    Updates for API Chunking support

    (cherry picked from commit 16955c1aee9aec465468c3bbae6ac20509cffe6d)

    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1754071

 config/settings.yml | 1 +
 spec/models/manageiq/providers/openshift/container_manager/refresher_inventory_object_spec.rb | 4 +-
 spec/models/manageiq/providers/openshift/container_manager/refresher_spec.rb | 10 +-
 spec/vcr_cassettes/manageiq/providers/openshift/container_manager/refresher_after_deletions.txt | 277 +-
 spec/vcr_cassettes/manageiq/providers/openshift/container_manager/refresher_after_deletions.yml | 3867 +-
 spec/vcr_cassettes/manageiq/providers/openshift/container_manager/refresher_before_deletions.txt | 339 +-
 spec/vcr_cassettes/manageiq/providers/openshift/container_manager/refresher_before_deletions.yml | 3924 +-
 7 files changed, 5443 insertions(+), 2979 deletions(-)

Comment 3 CFME Bot 2019-09-20 18:36:49 UTC
New commit detected on ManageIQ/manageiq-providers-kubernetes/hammer:

https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/b3577b80897095a285e26adee9aa8c18dd48fa9f
commit b3577b80897095a285e26adee9aa8c18dd48fa9f
Author:     Beni Cherniavsky-Paskin <cben>
AuthorDate: Fri Sep 20 09:32:29 2019 -0400
Commit:     Beni Cherniavsky-Paskin <cben>
CommitDate: Fri Sep 20 09:32:29 2019 -0400

    Merge pull request #312 from agrare/use_api_chunking

    API Chunking Support

    (cherry picked from commit bea27dd9fc49d1e488b8d17ec9c92bdc62046d22)

    https://bugzilla.redhat.com/show_bug.cgi?id=1754071

 app/models/manageiq/providers/kubernetes/container_manager/refresher_mixin.rb | 12 +-
 config/settings.yml | 1 +
 manageiq-providers-kubernetes.gemspec | 2 +-
 spec/models/manageiq/providers/kubernetes/container_manager/refresh_mixin_spec.rb | 3 +-
 spec/models/manageiq/providers/kubernetes/container_manager/refresher_spec.rb | 17 +-
 spec/vcr_cassettes/manageiq/providers/kubernetes/container_manager/refresher_after_deletions.txt | 277 +-
 spec/vcr_cassettes/manageiq/providers/kubernetes/container_manager/refresher_after_deletions.yml | 3867 +-
 spec/vcr_cassettes/manageiq/providers/kubernetes/container_manager/refresher_before_deletions.txt | 339 +-
 spec/vcr_cassettes/manageiq/providers/kubernetes/container_manager/refresher_before_deletions.yml | 3924 +-
 9 files changed, 5461 insertions(+), 2981 deletions(-)

Comment 4 juwatts 2019-10-01 17:19:43 UTC
Verified in 5.10.11.0.20190923184203_de5e4c8

This was only tested for OpenShift 3.9

Set the chunk_size For CFME to 10 in the advanced configuration. 

Ran automation tests while following evm.log, no api panic observed in the logs. 

Refreshes were behaving correctly.

Comment 6 errata-xmlrpc 2019-10-09 13:51:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2967

Comment 7 dmetzger 2019-10-28 18:14:15 UTC
*** Bug 1753788 has been marked as a duplicate of this bug. ***

Comment 8 Beni Paskin-Cherniavsky 2020-02-27 10:49:43 UTC
Commented on linked https://access.redhat.com/support/cases/#/case/02594750.
Hawkular was partially to fully down, later came up.  
Relation to this BZ not confirmed, there is no "panic" anywhere in logs.