Bug 1338739 - GCE Inventory Refresh Fails with Custom Machine Types
Summary: GCE Inventory Refresh Fails with Custom Machine Types
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.6.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.6.0
Assignee: Adam Grare
QA Contact: Dave Johnson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-23 11:14 UTC by Lutz Lange
Modified: 2019-08-06 20:06 UTC (History)
9 users (show)

Fixed In Version: 5.6.0.9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-29 16:05:14 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1348 0 normal SHIPPED_LIVE CFME 5.6.0 bug fixes and enhancement update 2016-06-29 18:50:04 UTC

Description Lutz Lange 2016-05-23 11:14:27 UTC
Description of problem:
  I configured and validated a GCE as a cloud provider. Validation works. Updating Relationships and Power info does not work.

Version-Release number of selected component (if applicable):
  cfme-gce-5.6.0.7-1.x86_64.tar.gz


How reproducible:
  Test connecting to GCE.

Actual results:
Error - 2 Minutes Ago
undefined method `[]' for nil:NilClass

Expected results:
instances should be visible

Additional info:
[----] I, [2016-05-23T07:10:21.922999 #14145:ff198c]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [1000000000232], State: [ok], Delivered in [0.003002618] seconds
[----] I, [2016-05-23T07:10:23.115626 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#do_work) EMS [] as [_] Event Monitor Thread gone. Restarting...
[----] I, [2016-05-23T07:10:23.115899 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#start_event_monitor) EMS [] as [_] Validating Connection/Credentials
[----] I, [2016-05-23T07:10:23.426164 #14135:ff198c]  INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [1000000000233], MiqWorker id: [1000000000002], Zone: [default], Role: [], Server: [7ad6e000-20d4-11e6-8374-42010af00012], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Session.check_session_timeout], Timeout: [600], Priority: [90], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [4.892643144] seconds
[----] I, [2016-05-23T07:10:23.426354 #14135:ff198c]  INFO -- : MIQ(MiqQueue#deliver) Message id: [1000000000233], Delivering...
[----] I, [2016-05-23T07:10:23.430329 #14135:ff198c]  INFO -- : MIQ(MiqQueue#delivered) Message id: [1000000000233], State: [ok], Delivered in [0.003973322] seconds
[----] I, [2016-05-23T07:10:23.991498 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#start_event_monitor) EMS [] as [_] Starting Event Monitor Thread
[----] I, [2016-05-23T07:10:23.991769 #16459:6935b74]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#monitor_events) EMS [] as [_] Monitoring for events
[----] I, [2016-05-23T07:10:23.991931 #16459:6935b74]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Google::CloudManager: [GCE]
[----] E, [2016-05-23T07:10:24.715271 #16459:6935b74] ERROR -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#start_event_monitor) EMS [] as [_] Event Monitor Thread aborted because [Resource not found (resource=manageiq-activity-log).]
[----] E, [2016-05-23T07:10:24.715448 #16459:6935b74] ERROR -- : [Fog::Errors::NotFound]: Resource not found (resource=manageiq-activity-log).  Method:[rescue in block in start_event_monitor]
[----] E, [2016-05-23T07:10:24.715519 #16459:6935b74] ERROR -- : /opt/rh/cfme-gemset/gems/fog-google-0.3.1/lib/fog/google/shared.rb:181:in `build_excon_response'
/opt/rh/cfme-gemset/gems/fog-google-0.3.1/lib/fog/google/shared.rb:154:in `request'
/opt/rh/cfme-gemset/gems/fog-google-0.3.1/lib/fog/google/requests/pubsub/create_subscription.rb:36:in `create_subscription'
/opt/rh/cfme-gemset/gems/fog-google-0.3.1/lib/fog/google/models/pubsub/subscription.rb:71:in `save'
/opt/rh/cfme-gemset/gems/fog-core-1.39.0/lib/fog/core/collection.rb:51:in `create'
/var/www/miq/vmdb/app/models/manageiq/providers/google/cloud_manager/event_catcher/stream.rb:42:in `get_or_create_subscription'
/var/www/miq/vmdb/app/models/manageiq/providers/google/cloud_manager/event_catcher/stream.rb:33:in `block in events'
/var/www/miq/vmdb/app/models/ext_management_system.rb:360:in `with_provider_connection'
/var/www/miq/vmdb/app/models/manageiq/providers/google/cloud_manager/event_catcher/stream.rb:32:in `events'
/var/www/miq/vmdb/app/models/manageiq/providers/google/cloud_manager/event_catcher/stream.rb:18:in `each_batch'
/var/www/miq/vmdb/app/models/manageiq/providers/google/cloud_manager/event_catcher/runner.rb:16:in `monitor_events'
/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:112:in `block in start_event_monitor'
/opt/rh/cfme-gemset/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
/opt/rh/cfme-gemset/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
[----] I, [2016-05-23T07:10:24.715653 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#start_event_monitor) EMS [] as [_] Started Event Monitor Thread
[----] I, [2016-05-23T07:10:33.081171 #12730:ff198c]  INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0038487911224365234, :worker_monitor=>1.355987548828125, :worker_dequeue=>0.008305788040161133, :total_time=>1.368506669998169}
[----] I, [2016-05-23T07:10:33.544895 #14169:ff198c]  INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1.
[----] I, [2016-05-23T07:10:33.553842 #14169:ff198c]  INFO -- : MIQ(MiqQueue.put) Message id: [1000000000234],  id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: []
[----] I, [2016-05-23T07:10:38.184331 #12730:ff198c]  INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200
[----] I, [2016-05-23T07:10:38.287693 #14151:ff198c]  INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [1000000000234], MiqWorker id: [1000000000005], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [4.737933781] seconds
[----] I, [2016-05-23T07:10:38.287854 #14151:ff198c]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [1000000000234], Delivering...
[----] I, [2016-05-23T07:10:38.290973 #14151:ff198c]  INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_jobs=>0.0010242462158203125, :jobs_to_dispatch_count=>0, :total_time=>0.002785921096801758}
[----] I, [2016-05-23T07:10:38.291287 #14151:ff198c]  INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [1000000000234], State: [ok], Delivered in [0.003427386] seconds
[----] I, [2016-05-23T07:10:39.719925 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#do_work) EMS [] as [_] Event Monitor Thread gone. Restarting...
[----] I, [2016-05-23T07:10:39.720298 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#start_event_monitor) EMS [] as [_] Validating Connection/Credentials
[----] I, [2016-05-23T07:10:40.451284 #16459:ff198c]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#start_event_monitor) EMS [] as [_] Starting Event Monitor Thread
[----] I, [2016-05-23T07:10:40.451678 #16459:7bf40f8]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager::EventCatcher::Runner#monitor_events) EMS [] as [_] Monitoring for events
[----] I, [2016-05-23T07:10:40.451814 #16459:7bf40f8]  INFO -- : MIQ(ManageIQ::Providers::Google::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Google::CloudManager: [GCE]

Comment 2 Greg Blomquist 2016-05-25 14:55:55 UTC
Lutz, can we get a full set of logs from this appliance?

It would be good to know if there are other errors cropping up here, as well.

Thanks!

Comment 3 Lutz Lange 2016-05-25 15:36:11 UTC
A strange thing happened . It started working by itself. I can give you access to my installation if you want to check the logs. Ping me via Mail.

Comment 4 Lutz Lange 2016-05-25 15:53:34 UTC
It might be the case that the provider refresh just took
Very long.

Comment 5 Lutz Lange 2016-05-25 17:14:30 UTC
A log time means several hours in this case.

Comment 9 Adam Grare 2016-05-25 20:17:29 UTC
Hi Lutz,

The original refresh was failing when setting the cpus and memory of the instance from the flavor.  The instance's flavor was not in our list of gce flavors, I was able to reproduce this by creating an instance with a custom machine type.

I talked to an engineer at google and they confirmed that custom images are not returned with the Fog Flavors list, and that we have to get them separately.

Adam

Comment 11 CFME Bot 2016-05-26 15:40:32 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/0706b4465c07d6c0e3dd31ac88b29325d4c1bc5d

commit 0706b4465c07d6c0e3dd31ac88b29325d4c1bc5d
Author:     Adam Grare <agrare>
AuthorDate: Wed May 25 16:15:04 2016 -0400
Commit:     Adam Grare <agrare>
CommitDate: Wed May 25 16:58:47 2016 -0400

    Fix handling of custom GCE flavors
    
    GCE custom machine_types/flavors are not returned with
    connection.flavors.all, and have to be retrieved with a direct
    call to connection.flavors.get(flavor_uid).
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1338739

 .../manageiq/providers/google/cloud_manager/refresh_parser.rb  | 10 ++++++++++
 1 file changed, 10 insertions(+)

Comment 14 errata-xmlrpc 2016-06-29 16:05:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348


Note You need to log in before you can comment on or make changes to this bug.