Bug 1425068

Summary: Provisioning against amazon fail because of unset flavor variable in best_fit_amazon stock placement method
Product: Red Hat CloudForms Management Engine Reporter: Felix Dewaleyne <fdewaley>
Component: ProvisioningAssignee: Patrik Kománek <pkomanek>
Status: CLOSED ERRATA QA Contact: Leo Khomenko <lkhomenk>
Severity: urgent Docs Contact:
Priority: medium    
Version: 5.6.0CC: cpelland, fdewaley, gmccullo, jhardy, lkhomenk, obarenbo, pkomanek, simaishi, tfitzger
Target Milestone: GA   
Target Release: 5.8.0   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: 5.8.0.6 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-31 14:35:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Felix Dewaleyne 2017-02-20 13:48:29 UTC
Description of problem:
Provisioning against amazon fail because of unset vlavor variable in best_fit_amazon stock placement method

Version-Release number of selected component (if applicable):
5.6.3.3-1

How reproducible:
customer can reproduce constantly

Steps to Reproduce:
1. provision against amazon (vm provision not orchestration)
2.
3.

Actual results:

18485:[----] E, [2017-01-11T16:34:20.336281 #6642:6ebf338] ERROR -- : Q-task_id([miq_provision_42000000000043]) <AEMethod best_fit_amazon> The following error occurred during method evaluation:
18486:[----] E, [2017-01-11T16:34:20.337382 #6642:6ebf338] ERROR -- : Q-task_id([miq_provision_42000000000043]) <AEMethod best_fit_amazon>   DRb::DRbRemoteError: Service Model not found (MiqAeException::ServiceNotFound)
18487:[----] E, [2017-01-11T16:34:20.340473 #6642:6ebf338] ERROR -- : Q-task_id([miq_provision_42000000000043]) <AEMethod best_fit_amazon>   (druby://127.0.0.1:42832) /var/www/miq/vmdb/lib/miq_automation_engine/engine/miq_ae_service_model_base.rb:24:in `rescue in method_missing'
18496:[----] E, [2017-01-11T16:34:20.344921 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR: (druby://127.0.0.1:42832) /var/www/miq/vmdb/lib/miq_automation_engine/engine/miq_ae_service_model_base.rb:24:in `rescue in method_missing': Service Model not found (MiqAeException::ServiceNotFound) (DRb::DRbRemoteError)
18497:[----] E, [2017-01-11T16:34:20.345083 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /var/www/miq/vmdb/lib/miq_automation_engine/engine/miq_ae_service_model_base.rb:21:in `method_missing'
18498:[----] E, [2017-01-11T16:34:20.345219 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/rh-ruby22/root/usr/share/ruby/drb/drb.rb:1624:in `perform_without_block'
18499:[----] E, [2017-01-11T16:34:20.345344 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/rh-ruby22/root/usr/share/ruby/drb/drb.rb:1584:in `perform'
18500:[----] E, [2017-01-11T16:34:20.345478 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/rh-ruby22/root/usr/share/ruby/drb/drb.rb:1657:in `block (2 levels) in main_loop'
18501:[----] E, [2017-01-11T16:34:20.345612 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/rh-ruby22/root/usr/share/ruby/drb/drb.rb:1653:in `loop'
18502:[----] E, [2017-01-11T16:34:20.345734 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/rh-ruby22/root/usr/share/ruby/drb/drb.rb:1653:in `block in main_loop'
18503:[----] E, [2017-01-11T16:34:20.345880 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/cfme-gemset/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `call'
18504:[----] E, [2017-01-11T16:34:20.346004 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from (druby://127.0.0.1:42832) /opt/rh/cfme-gemset/gems/logging-2.1.0/lib/logging/diagnostic_context.rb:450:in `block in create_with_logging_context'
18505:[----] E, [2017-01-11T16:34:20.346177 #6642:6e403a8] ERROR -- : Q-task_id([miq_provision_42000000000043]) Method STDERR:  from <code: flavor         = $evm.vmdb('flavor').find(instance_id)>:46:in `<main>'

Expected results:
deployment passes

Additional info:
the method is / ManageIQ / Cloud / VM / Provisioning / Placement / best_fit_amazon

Comment 3 Greg McCullough 2017-02-20 15:32:36 UTC
It is not a typo, but a minor difference in the way Rails finders work.  The "find" method will raise an error if the object cannot be found.  However, changing this to "find_by_id" will return a nil object if the object is not found.

The last debugging line is setup to handle a nil flavor by using "flavor.try(:name)".

I would suggest changing the line to the following and we will get the method updated as well.

flavor         = $evm.vmdb('flavor').find_by(:id => instance_id)

Comment 4 Felix Dewaleyne 2017-02-22 14:23:52 UTC
(In reply to Greg McCullough from comment #3)
> It is not a typo, but a minor difference in the way Rails finders work.  The
> "find" method will raise an error if the object cannot be found.  However,
> changing this to "find_by_id" will return a nil object if the object is not
> found.
> 
> The last debugging line is setup to handle a nil flavor by using
> "flavor.try(:name)".
> 
> I would suggest changing the line to the following and we will get the
> method updated as well.
> 
> flavor         = $evm.vmdb('flavor').find_by(:id => instance_id)

I recommended that action but it looks like the same error is produced at checkprovisioned (error in placement). I'll wait for more logs to comment further...

Comment 8 Tina Fitzgerald 2017-02-27 23:00:33 UTC
Hi Felix,

The case message from Feb17 states that the problem occurs with straight VM provisioning. 

".... 
It has also been stated already in this issue that this problem appears also when provisioning is attempted using automatic placement while not using service catalog but instances -> lifecycle -> provision ... which does not use preconfiguredialog at all. Both problems surface when automatic placement is attempted, as something there goes wrong, but it's not that instance_type (which in Amazon EC2 *is* the flavor) hasn't been provided....."

Lets take the Service Provision complexity out of play here and ask the customer to do a VM provision using automatic placement and send us the automation.log and evm.logs showing the error.

Is there something new in the customer environment? Did they just upgrade? Did this ever work properly?

Thanks,
Tina

Comment 12 Tina Fitzgerald 2017-03-03 14:17:09 UTC
Hi Felix,

The failing method is in the cloudforms_essentials domain.
The cloudforms essentials repo does not contain that method.

Can you ask the customer for all of the Automate domains?

Thanks,
Tina

Comment 13 Felix Dewaleyne 2017-03-03 15:45:23 UTC
(In reply to Tina Fitzgerald from comment #12)
> Hi Felix,
> 
> The failing method is in the cloudforms_essentials domain.
> The cloudforms essentials repo does not contain that method.
> 
> Can you ask the customer for all of the Automate domains?
> 
> Thanks,
> Tina

if you look up, we did the changes that were recommended in this very same bz to the method to progress. see https://bugzilla.redhat.com/show_bug.cgi?id=1425068#c3

I'll collect an automate export if you want but the customer confirms he hasn't done any other change to the method.

the method best_fit_amazon definitely is included in the manageIQ domain. the method is 

/ManageIQ/Cloud/VM/Provisioning/Placement/best_fit_amazon

Comment 15 Greg McCullough 2017-03-03 16:28:45 UTC
Felix - Two things:

1) There is a typo in the method that was updated.

Line 46 of /CloudForms_Essentials/Cloud/VM/Provisioning/Placement/best_fit_amazon is:
  flavor         = $evm.vmdb('flavor').find_by(:id=>instance_id)*
should be:
  flavor         = $evm.vmdb('flavor').find_by(:id=>instance_id)

(Note: Removed the "*" from the end of the line)


2) I would highly recommend NOT copying methods from the based ManageIQ domain to the CloudForms_Essentials domain.  They should have copied this to their own domain or possibly a new domain.  It is very confusing at first but easy to verify that this method did not come from CloudForms_Essentials.


Please have them edit their custom best_fit_amazon method and fix the typo noted above and retest.

Comment 16 Felix Dewaleyne 2017-03-03 16:57:08 UTC
(In reply to Greg McCullough from comment #15)
> Felix - Two things:
> 
> 1) There is a typo in the method that was updated.
> 
> Line 46 of
> /CloudForms_Essentials/Cloud/VM/Provisioning/Placement/best_fit_amazon is:
>   flavor         = $evm.vmdb('flavor').find_by(:id=>instance_id)*
> should be:
>   flavor         = $evm.vmdb('flavor').find_by(:id=>instance_id)
> 
> (Note: Removed the "*" from the end of the line)
> 
> 
> 2) I would highly recommend NOT copying methods from the based ManageIQ
> domain to the CloudForms_Essentials domain.  They should have copied this to
> their own domain or possibly a new domain.  It is very confusing at first
> but easy to verify that this method did not come from CloudForms_Essentials.
> 
> 
> Please have them edit their custom best_fit_amazon method and fix the typo
> noted above and retest.

ack, passed. Sorry for missing the *

Comment 17 Greg McCullough 2017-03-08 16:11:25 UTC
Felix - Can this issue be dropped to a lower severity?

Patrik - We should still make the change mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1425068#c3.

This means updating the best_fit_amazon method changing the flavor lookup to use this updated syntax:

flavor         = $evm.vmdb('flavor').find_by(:id => instance_id)

Comment 18 Felix Dewaleyne 2017-03-10 12:45:55 UTC
(In reply to Greg McCullough from comment #17)
> Felix - Can this issue be dropped to a lower severity?
> 
> Patrik - We should still make the change mentioned in
> https://bugzilla.redhat.com/show_bug.cgi?id=1425068#c3.
> 
> This means updating the best_fit_amazon method changing the flavor lookup to
> use this updated syntax:
> 
> flavor         = $evm.vmdb('flavor').find_by(:id => instance_id)

working on getting a lower severity. The case is getting there.

Comment 19 Greg McCullough 2017-03-16 17:06:56 UTC
Moving to POST now that https://github.com/ManageIQ/manageiq-content/pull/63 has been merged.

Comment 20 Leo Khomenko 2017-03-24 11:51:28 UTC
verified on 5.8.0.7

Comment 22 errata-xmlrpc 2017-05-31 14:35:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:1367