Bug 1417273

Summary: Refresh of OSP10 OpenStack/Director undercloud failing
Product: Red Hat CloudForms Management Engine Reporter: Saif Ali <saali>
Component: ProvidersAssignee: Tzu-Mainn Chen <tzumainn>
Status: CLOSED CURRENTRELEASE QA Contact: Ola Pavlenko <opavlenk>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 5.8.0CC: cpelland, jfrey, jhajyahy, jhardy, obarenbo, rtweed, simaishi, tpapaioa
Target Milestone: GAKeywords: TestOnly, ZStream
Target Release: 5.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: openstack
Fixed In Version: 5.8.0.6 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1420917 1433094 (view as bug list) Environment:
Last Closed: 2017-06-12 17:25:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Openstack Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1390600, 1420917, 1433094    
Attachments:
Description Flags
emv.log
none
fog.log
none
latest 5.8.0.3 evm.log none

Description Saif Ali 2017-01-27 18:33:57 UTC 
Description of problem:

The "Last Refresh" status in the OpenStack Director Infrastructure Provider shows an Error - 14 Minutes Ago read timeout reached.


Version-Release number of selected component (if applicable):
CFME 5.7
OSP 10

How reproducible:


Steps to Reproduce:
1. Deploy new CFME 4.2  
2. Add OSP10 infra provider
3. Refresh OSP undercloud provider

Actual results:
The "Last Refresh" status in the OpenStack Director Infrastructure Provider shows an Error - 14 Minutes Ago read timeout reached.

Expected results:


Additional info:
Comment 2 Saif Ali 2017-01-27 18:34:19 UTC 
[----] I, [2017-01-25T12:05:06.664812 #4535:ded140]  INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh) Refreshing all targets...
[----] I, [2017-01-25T12:05:06.665000 #4535:ded140]  INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh) EMS: [OpenStack-Director], id: [1000000000008] Refreshing targets for EMS...
[----] I, [2017-01-25T12:05:06.665108 #4535:ded140]  INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh) EMS: [OpenStack-Director], id: [1000000000008]   ManageIQ::Providers::Openstack::InfraManager [OpenStack-Director] id [1000000000008]
[----] I, [2017-01-25T12:05:06.665243 #4535:ded140]  INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh_targets_for_ems) EMS: [OpenStack-Director], id: [1000000000008] Refreshing target ManageIQ::Providers::Openstack::InfraManager [OpenStack-Director] id [1000000000008]...
[----] E, [2017-01-25T12:11:18.234512 #4535:ded140] ERROR -- : <Fog> excon.error     #<Excon::Error::Timeout: read timeout reached>
[----] E, [2017-01-25T12:11:18.235030 #4535:ded140] ERROR -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh) EMS: [OpenStack-Director], id: [1000000000008] Refresh failed
[----] E, [2017-01-25T12:11:18.235363 #4535:ded140] ERROR -- : [Excon::Error::Timeout]: read timeout reached  Method:[rescue in block in refresh]
[----] E, [2017-01-25T12:11:18.235516 #4535:ded140] ERROR -- : /opt/rh/cfme-gemset/gems/excon-0.54.0/lib/excon/socket.rb:267:in `select_with_timeout'
[----] E, [2017-01-25T12:11:18.235688 #4535:ded140] ERROR -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh) EMS: [OpenStack-Director], id: [1000000000008] Unable to perform refresh for the following targets:
[----] E, [2017-01-25T12:11:18.235976 #4535:ded140] ERROR -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh)  --- ManageIQ::Providers::Openstack::InfraManager [OpenStack-Director] id [1000000000008]
[----] I, [2017-01-25T12:11:18.351117 #4535:ded140]  INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::Refresher#refresh) Refreshing all targets...Complete
[----] E, [2017-01-25T12:11:18.351402 #4535:ded140] ERROR -- : MIQ(MiqQueue#deliver) Message id: [1000000296611], Error: [read timeout reached]
[----] E, [2017-01-25T12:11:18.351557 #4535:ded140] ERROR -- : [EmsRefresh::Refreshers::EmsRefresherMixin::PartialRefreshError]: read timeout reached  Method:[rescue in deliver]
[----] E, [2017-01-25T12:11:18.351737 #4535:ded140] ERROR -- : /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:50:in `refresh'
Comment 4 Saif Ali 2017-02-02 15:26:20 UTC 
Created attachment 1247142 [details]
emv.log
Comment 5 Saif Ali 2017-02-02 15:27:16 UTC 
Created attachment 1247143 [details]
fog.log
Comment 7 Tzu-Mainn Chen 2017-02-02 18:44:27 UTC 
Saif, can you see if https://github.com/ManageIQ/manageiq/pull/13748 fixes the issue for you?  Thanks!
Comment 10 Ronnie Rasouli 2017-03-07 14:07:55 UTC 
Created attachment 1260827 [details]
latest 5.8.0.3 evm.log
Comment 11 Ronnie Rasouli 2017-03-07 14:08:46 UTC 
failed to refresh RHOS10 - attached evm.log
Comment 12 Tzu-Mainn Chen 2017-03-07 14:38:46 UTC 
Should be fixed by:

https://github.com/ManageIQ/manageiq/pull/14172
https://github.com/ManageIQ/manageiq-gems-pending/pull/84

These PRs add a few settings, one being read_timeout that gets passed down to excon, so you'd want to extend the read timeout to accommodate your system.
Comment 14 Ronnie Rasouli 2017-03-19 10:13:30 UTC 
verified on 5.8.0.6