Bug 1558209
| Summary: | Cloudforms v4.6 Integration with Openshift Container Platform v3.6 - Unable to refresh Provider Info | |||
|---|---|---|---|---|
| Product: | Red Hat CloudForms Management Engine | Reporter: | Saif Ali <saali> | |
| Component: | Providers | Assignee: | Beni Paskin-Cherniavsky <cben> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | juwatts | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 5.9.0 | CC: | cben, cpelland, dluong, gblomqui, jcutter, jfrey, jhardy, lsmola, myoder, obarenbo, oourfali, simaishi | |
| Target Milestone: | GA | Keywords: | TestOnly, ZStream | |
| Target Release: | 5.10.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | 5.10.0.0 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1559149 (view as bug list) | Environment: | ||
| Last Closed: | 2019-02-11 14:03:17 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | Container Management | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1559149 | |||
The issue is ContainerService.create!(:container_groups => nil) in https://github.com/Ladas/manageiq-providers-kubernetes/blob/f9bb30cde987500d482bd5507613b823bc3d3f5b/app/models/manageiq/providers/kubernetes/container_manager/refresh_parser.rb#L542 so it needs to be: h[:container_groups] = cgs_by_namespace_and_name.fetch_path(h[:namespace], h[:name]) || [] Not sure if it was clear by the previous comment, but this is a Rails thing, example: ContainerService.create!(:container_groups => nil) will fail (with the same error as we see in log) ContainerService.create!(:container_groups => []) will pass Thanks! Working on PR. https://github.com/ManageIQ/manageiq-providers-kubernetes/pull/242 (fix + test) https://github.com/ManageIQ/manageiq-providers-openshift/pull/92 (test) The exact condition that triggers this is Service without matching Endpoints. - This can happen for services without a selector. Such services are legitimate, but not really useful until one manually creates an Endpoints object: https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors Confirmed that creating an empty Endpoints would avoid the crash. - The PRs possibly also fix a race condition — refresh does separate requests for services and endpoints, so could in theory see one without the other. Bug present both in graph and classic refresh, so might be relevant to 5.8 or older too ? (untested) New commits detected on ManageIQ/manageiq-providers-kubernetes/master: https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/7350379389a70ad606380cbdb8ac6dc5e6a4d1d6 commit 7350379389a70ad606380cbdb8ac6dc5e6a4d1d6 Author: Beni Cherniavsky-Paskin <cben> AuthorDate: Wed Mar 21 13:27:35 2018 -0400 Commit: Beni Cherniavsky-Paskin <cben> CommitDate: Wed Mar 21 13:27:35 2018 -0400 [classic refresh] Fix refresh crash on Service without Endpoints Avoids "undefined method `map' for nil:NilClass" https://bugzilla.redhat.com/show_bug.cgi?id=1558209 app/models/manageiq/providers/kubernetes/container_manager/refresh_parser.rb | 1 + 1 file changed, 1 insertion(+) https://github.com/ManageIQ/manageiq-providers-kubernetes/commit/b5af5e078cd0fc3ca1c4acfe40a5594ac00e0024 commit b5af5e078cd0fc3ca1c4acfe40a5594ac00e0024 Author: Beni Cherniavsky-Paskin <cben> AuthorDate: Wed Mar 21 13:40:42 2018 -0400 Commit: Beni Cherniavsky-Paskin <cben> CommitDate: Wed Mar 21 13:40:42 2018 -0400 [graph refresh] Fix refresh crash on Service without Endpoints Avoids "undefined method `each' for nil:NilClass" https://bugzilla.redhat.com/show_bug.cgi?id=1558209 app/models/manageiq/providers/kubernetes/container_manager/refresh_parser.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) *** Bug 1568040 has been marked as a duplicate of this bug. *** Verified on: 5.10.0.20.20181016163900_fe677b4
Verification steps:
1) On the Openshift provider, created the following service json file:
# cat service_no_endpoint.json
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "hello-service"
},
"spec": {
"ports": [
{
"protocol": "TCP",
"port": 8888,
"targetPort": 8080
}
]
}
}
2) Added the service by running oc create -f service_no_endpoint.json
# oc get service
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-service <IP_Removed> <none> 8888/TCP 5s
#
3) On the CFME appliance, triggered a manual refresh
[----] I, [2018-10-24T10:49:52.423644 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) Refreshing all targets...
[----] I, [2018-10-24T10:49:52.423786 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) EMS: [ocp-36-hawk], id: [1] Refreshing targets for EMS...
[----] I, [2018-10-24T10:49:52.423902 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) EMS: [ocp-36-hawk], id: [1] ManageIQ::Providers::Openshift::ContainerManager [ocp-36-hawk] id [1]
[----] I, [2018-10-24T10:49:52.424085 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager#with_provider_connection) Connecting through ManageIQ::Providers::Openshift::ContainerManager: [ocp-36-hawk]
[----] I, [2018-10-24T10:49:52.640552 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager#with_provider_connection) Connecting through ManageIQ::Providers::Openshift::ContainerManager: [ocp-36-hawk]
[----] I, [2018-10-24T10:49:53.959485 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh_targets_for_ems) EMS: [ocp-36-hawk], id: [1] Refreshing target ManageIQ::Providers::Openshift::ContainerManager [ocp-36-hawk] id [1]...
[----] I, [2018-10-24T10:49:56.399006 #21231:b0af7c] INFO -- : EMS: [ocp-36-hawk], id: [1] Saving EMS Inventory...
[----] I, [2018-10-24T10:49:58.216387 #12239:b0af7c] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1.
[----] I, [2018-10-24T10:49:59.603220 #21231:b0af7c] INFO -- : EMS: [ocp-36-hawk], id: [1] Saving EMS Inventory...Complete
[----] I, [2018-10-24T10:49:59.603435 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh_targets_for_ems) EMS: [ocp-36-hawk], id: [1] Refreshing target ManageIQ::Providers::Openshift::ContainerManager [ocp-36-hawk] id [1]...Complete
[----] I, [2018-10-24T10:49:59.603571 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh_targets_for_ems) EMS: [ocp-36-hawk], id: [1] ManagerRefresh Post Processing ManageIQ::Providers::Openshift::ContainerManager [ocp-36-hawk] id [1]...
[----] I, [2018-10-24T10:49:59.603706 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh_targets_for_ems) EMS: [ocp-36-hawk], id: [1] ManagerRefresh Post Processing ManageIQ::Providers::Openshift::ContainerManager [ocp-36-hawk] id [1]...Complete
[----] I, [2018-10-24T10:49:59.603798 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) EMS: [ocp-36-hawk], id: [1] Refreshing targets for EMS...Complete - Timings {:collect_inventory_for_targets=>1.5353152751922607, :parse_targeted_inventory=>2.146233558654785, :save_inventory=>3.497368097305298, :manager_refresh_post_processing=>1.3589859008789062e-05, :ems_refresh=>7.179760217666626}
[----] I, [2018-10-24T10:49:59.617810 #21231:b0af7c] INFO -- : MIQ(ManageIQ::Providers::Openshift::ContainerManager::Refresher#refresh) Refreshing all targets...Complete
|
Description of problem: Adding Openshift Container Platform v3.6 to CloudForms 4.6 it discover and refresh most of the objects but it fail the refresh when its trying to collect the container services [----] D, [2018-03-16T16:27:33.893726 #30428:c1b13c] DEBUG -- : MIQ(ManagerRefresh::SaveCollection::Saver::Default#save!) Processing InventoryCollection:<ContainerService>, blacklist: [namespace] of size 44... [----] E, [2018-03-16T16:27:33.905039 #30428:c1b13c] ERROR -- : MIQ(ManagerRefresh::SaveCollection::Saver::Default#save!) Error when saving InventoryCollection:<ContainerService>, blacklist: [namespace] with strategy: , saver_strategy: default, targeted: false. Message: undefined method `each' for nil:NilClass container_services; // InventoryCollection:<ContainerService>, blacklist: [namespace] container_service_port_configs; // InventoryCollection:<ContainerServicePortConfig> [----] D, [2018-03-16T16:32:03.813700 #30428:c1b13c] DEBUG -- : MIQ(ManagerRefresh::SaveCollection::TopologicalSort.save_inventory_object_inventory) Saving collection InventoryCollection:<ContainerService>, blacklist: [namespace] of size 44 to the database, for the manager: 'Non-Prod Openshift Container Platform'... [----] D, [2018-03-16T16:32:03.817998 #30428:c1b13c] DEBUG -- : MIQ(ManagerRefresh::SaveCollection::Saver::Default#save!) Processing InventoryCollection:<ContainerService>, blacklist: [namespace] of size 44... [----] E, [2018-03-16T16:32:03.829032 #30428:c1b13c] ERROR -- : MIQ(ManagerRefresh::SaveCollection::Saver::Default#save!) Error when saving InventoryCollection:<ContainerService>, blacklist: [namespace] with strategy: , saver_strategy: default, targeted: false. Message: undefined method `each' for nil:NilClass container_services; // InventoryCollection:<ContainerService>, blacklist: [namespace] container_service_port_configs; // InventoryCollection:<ContainerServicePortConfig> Version-Release number of selected component (if applicable): 5.9.1.0.20180314203929_67fd99d How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: