Bug 1303074

Summary: Error while executing a report
Product: Red Hat CloudForms Management Engine Reporter: Nikhil Gupta <ngupta>
Component: ReportingAssignee: Joe Rafaniello <jrafanie>
Status: CLOSED ERRATA QA Contact: Shveta <sshveta>
Severity: high Docs Contact:
Priority: high    
Version: 5.4.0CC: cpelland, dajohnso, jhardy, jprause, jrafanie, mfeifer, obarenbo, psavage
Target Milestone: GAKeywords: ZStream
Target Release: 5.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.6.0.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1304003 1304006 (view as bug list) Environment:
Last Closed: 2016-06-29 15:35:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: Bug
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1304003, 1304006    

Description Nikhil Gupta 2016-01-29 13:19:20 UTC
Description of problem:
Created a report to list all VMs with thin provisioned disk. However while queuing it throws below error:
~~~
[----] I, [2016-01-29T11:34:03.764370 #3646:7f7e98]  INFO -- : MIQ(MiqQueue.deliver)    Message id: [1000014279377], Delivering...
[----] I, [2016-01-29T11:34:03.765783 #3646:7f7e98]  INFO -- : MIQ(MiqTask.update_status) Task: [1000000097793] [Active] [Ok] [Generating report]
[----] E, [2016-01-29T11:34:04.290851 #3646:7f7e98] ERROR -- : [ActiveRecord::HasManyThroughSourceAssociationNotFoundError]: Could not find the source association(s) :vm in model MiqProvision. Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'. Is it one of :miq_request, :source, :destination, :miq_request_tasks, or :miq_request_task?  Method:[rescue in _async_generate_table]
[----] E, [2016-01-29T11:34:04.291085 #3646:7f7e98] ERROR -- : /opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/reflection.rb:509:in `check_validity!'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/associations/association.rb:26:in `initialize'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/associations/collection_association.rb:24:in `initialize'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/associations/has_many_through_association.rb:10:in `initialize'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/associations.rb:160:in `new'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/associations.rb:160:in `association'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/associations/builder/association.rb:44:in `block in define_readers'
/var/www/miq/vmdb/app/models/condition.rb:144:in `collect_children'
/var/www/miq/vmdb/app/models/condition.rb:170:in `_subst_find'
/var/www/miq/vmdb/app/models/condition.rb:102:in `block in subst'
/var/www/miq/vmdb/app/models/condition.rb:102:in `gsub!'
/var/www/miq/vmdb/app/models/condition.rb:102:in `subst'
/var/www/miq/vmdb/app/models/rbac.rb:523:in `matches_search_filters?'
/var/www/miq/vmdb/app/models/rbac.rb:424:in `block in search'
/var/www/miq/vmdb/app/models/rbac.rb:424:in `reject'
/var/www/miq/vmdb/app/models/rbac.rb:424:in `search'
/var/www/miq/vmdb/app/models/miq_report/generator.rb:269:in `_generate_table'
/var/www/miq/vmdb/app/models/miq_report/generator.rb:173:in `block in generate_table'
/var/www/miq/vmdb/app/models/user.rb:921:in `with_userid'
/var/www/miq/vmdb/app/models/miq_report/generator.rb:173:in `generate_table'
/var/www/miq/vmdb/app/models/miq_report/generator/async.rb:93:in `_async_generate_table'
/var/www/miq/vmdb/app/models/miq_report/generator/async.rb:42:in `_async_generate_table'
/var/www/miq/vmdb/app/models/miq_queue.rb:356:in `block in deliver'
/opt/rh/ruby200/root/usr/share/ruby/timeout.rb:66:in `timeout'
/var/www/miq/vmdb/app/models/miq_queue.rb:352:in `deliver'
/var/www/miq/vmdb/lib/workers/queue_worker_base.rb:107:in `deliver_queue_message'
/var/www/miq/vmdb/lib/workers/queue_worker_base.rb:135:in `deliver_message'
/var/www/miq/vmdb/lib/workers/queue_worker_base.rb:152:in `block in do_work'
/var/www/miq/vmdb/lib/workers/queue_worker_base.rb:146:in `loop'
/var/www/miq/vmdb/lib/workers/queue_worker_base.rb:146:in `do_work'
/var/www/miq/vmdb/lib/workers/worker_base.rb:323:in `block in do_work_loop'
/var/www/miq/vmdb/lib/workers/worker_base.rb:320:in `loop'
/var/www/miq/vmdb/lib/workers/worker_base.rb:320:in `do_work_loop'
/var/www/miq/vmdb/lib/workers/worker_base.rb:141:in `run'
/var/www/miq/vmdb/lib/workers/worker_base.rb:122:in `start'
/var/www/miq/vmdb/lib/workers/worker_base.rb:23:in `start_worker'
/var/www/miq/vmdb/lib/workers/bin/worker.rb:3:in `<top (required)>'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/railties/lib/rails/commands/runner.rb:52:in `eval'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/railties/lib/rails/commands/runner.rb:52:in `<top (required)>'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/railties/lib/rails/commands.rb:64:in `require'
/opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/railties/lib/rails/commands.rb:64:in `<top (required)>'
script/rails:6:in `require'
script/rails:6:in `<main>'
[----] W, [2016-01-29T11:34:04.367302 #3646:7f7e98]  WARN -- : <AuditFailure> MIQ(Async.rescue in _async_generate_table) userid: [admin] - Could not find the source association(s) :vm in model MiqProvision. Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'. Is it one of :miq_request, :source, :destination, :miq_request_tasks, or :miq_request_task?
[----] E, [2016-01-29T11:34:04.370600 #3646:7f7e98] ERROR -- : MIQ(MiqQueue.deliver)    Message id: [1000014279377], Error: [Could not find the source association(s) :vm in model MiqProvision. Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'. Is it one of :miq_request, :source, :destination, :miq_request_tasks, or :miq_request_task?]
[----] E, [2016-01-29T11:34:04.370710 #3646:7f7e98] ERROR -- : [ActiveRecord::HasManyThroughSourceAssociationNotFoundError]: Could not find the source association(s) :vm in model MiqProvision. Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'. Is it one of :miq_request, :source, :destination, :miq_request_tasks, or :miq_request_task?  Method:[rescue in deliver]
[----] E, [2016-01-29T11:34:04.370788 #3646:7f7e98] ERROR -- : /opt/rh/cfme-gemset/bundler/gems/rails-4842a8377644/activerecord/lib/active_record/reflection.rb:509:in `check_validity!'
~~~

Version-Release number of selected component (if applicable):
5.4.3.1-20151013111241_1c155a5

How reproducible:
Always

Steps to Reproduce:
1. Navigate to Cloud Intelligence ==> Reports ==> Reports.
2. Configuration ==> Add a new Report
3. In Columns tab, enter Name and Title ==? In Base the report on select "Virtual Machines"
4. In Editing report, select "Name"
5. In Consolidation, Column 1 ==> Name
6. In Filter, set following condition:
    FIND Virtual Machine.Provisioned VMs : Thin Provisioned = "true" CHECK ALL Thin Provisioned = "true" 
7. In Summary, Sort report by ==> Name
8. Save
9. Now Queue it.

Actual results:
Report is not getting generated. Shows error symbol.

Expected results:
Report should be generated and list all thin provisioned VMs.

Comment 3 Shveta 2016-02-03 00:13:46 UTC
Assigning to add test case

Comment 5 CFME Bot 2016-02-12 21:11:00 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/43ced0fbc46b0b801f7b108e8f3bea982d392ed7

commit 43ced0fbc46b0b801f7b108e8f3bea982d392ed7
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Fri Feb 12 15:02:45 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Fri Feb 12 15:26:00 2016 -0500

    Specify the has_many through source using the belongs_to, not an alias.
    
    Also, specify the source_type as rails requests.
    
    Note, we specify VmOrTemplate here because VmOrTemplate is where this
    has_many is located.  It seems like this has_many may belong in MiqTemplate.
    
    Fixes the following error:
    ```
    ActiveRecord::HasManyThroughSourceAssociationNotFoundError:
      Could not find the source association(s) :vm in model MiqProvision.
      Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'.
      Is it one of miq_request, source, destination, miq_request_tasks, miq_request_task, or tenant?
    ```
    
    This resolves the model bug reported in:
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074
    
    We still need to see if we can generate reports based on this now fixed
    has_many :through.

 app/models/vm_or_template.rb       |  2 +-
 spec/models/vm_or_template_spec.rb | 26 ++++++++++++++++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)

Comment 8 CFME Bot 2016-02-16 15:55:38 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/be75310804b9deb1544da61a382f0aaa342e2d78

commit be75310804b9deb1544da61a382f0aaa342e2d78
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Tue Feb 16 09:00:51 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Feb 16 09:07:48 2016 -0500

    Add a test for MiqReport#generate_table with a has_many through.
    
    Remove unused common test setup.
    
    Follow up to #6650
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074

 spec/models/miq_report_spec.rb | 52 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 49 insertions(+), 3 deletions(-)

Comment 10 CFME Bot 2016-02-23 21:50:53 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/fbac6c0c4026c31809e25706ec2537664449be1b

commit fbac6c0c4026c31809e25706ec2537664449be1b
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Tue Feb 23 15:27:20 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Feb 23 15:30:18 2016 -0500

    Refine the has_many through test based on a reported issue.
    
    The first cut of this test was a first attempt to recreate the problem
    by simplifying the report.
    
    This commit implements the exact reported issue and verifies the backend
    code does the correct thing.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074
    
    Followup to #6650 and #6690

 spec/models/miq_report_spec.rb | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

Comment 11 Joe Rafaniello 2016-03-01 19:28:04 UTC
We have fixed a bug in our code but there's also a change required in your report definition.

In the recreation steps there are a few changes to the report you wanted to do:

1. Navigate to Cloud Intelligence ==> Reports ==> Reports.
2. Configuration ==> Add a new Report
3. In Columns tab, enter Name and Title ==? In Base the report on select "Virtual Machines" 
4. In Editing report, select "Name"
5. In Consolidation, Column 1 ==> Name
6. In Filter, set following condition:
    FIND Virtual Machine.Provisioned VMs : Thin Provisioned = "true" CHECK ALL Thin Provisioned = "true" 
7. In Summary, Sort report by ==> Name
8. Save
9. Now Queue it.

[CHANGES] 
Step 3: Base the report on "Template", because you're looking for thin provisioned vms based on a template.

Step 4/5: you'll need select miq_provision_vms.name because you want to see the name of the thin provisioned vm, not the template's name (from step 3 change above).

Step 6: create a filter based on the template change from step 3:
    FIND Template.Provisioned VMs : Thin Provisioned = "true" CHECK ALL Thin Provisioned = "true"

Comment 12 CFME Bot 2016-03-01 21:31:49 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=9decefad3cbac3121438cf5cc30ef2acdf2131ef

commit 9decefad3cbac3121438cf5cc30ef2acdf2131ef
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Fri Feb 12 15:02:45 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Mar 1 15:43:07 2016 -0500

    Specify the has_many through source using the belongs_to, not an alias.
    
    Also, specify the source_type as rails requests.
    
    Note, we specify VmOrTemplate here because VmOrTemplate is where this
    has_many is located.  It seems like this has_many may belong in MiqTemplate.
    
    Fixes the following error:
    ```
    ActiveRecord::HasManyThroughSourceAssociationNotFoundError:
      Could not find the source association(s) :vm in model MiqProvision.
      Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'.
      Is it one of miq_request, source, destination, miq_request_tasks, miq_request_task, or tenant?
    ```
    
    This resolves the model bug reported in:
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074
    
    We still need to see if we can generate reports based on this now fixed
    has_many :through.

 vmdb/app/models/vm_or_template.rb       |  2 +-
 vmdb/spec/models/vm_or_template_spec.rb | 26 ++++++++++++++++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)

Comment 13 CFME Bot 2016-03-01 21:31:53 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=e3d5ac5b864b67204d06fe6ff58ce967de9a1df6

commit e3d5ac5b864b67204d06fe6ff58ce967de9a1df6
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Tue Feb 16 09:00:51 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Mar 1 15:43:07 2016 -0500

    Add a test for MiqReport#generate_table with a has_many through.
    
    Remove unused common test setup.
    
    Follow up to #6650
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074

 vmdb/spec/models/miq_report_spec.rb | 51 +++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

Comment 14 CFME Bot 2016-03-01 21:32:01 UTC
New commit detected on cfme/5.4.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=578ebf27f62f98a92ce7e1188d2191fc8805f419

commit 578ebf27f62f98a92ce7e1188d2191fc8805f419
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Tue Feb 23 15:27:20 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Mar 1 15:46:41 2016 -0500

    Refine the has_many through test based on a reported issue.
    
    The first cut of this test was a first attempt to recreate the problem
    by simplifying the report.
    
    This commit implements the exact reported issue and verifies the backend
    code does the correct thing.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074
    
    Followup to #6650 and #6690

 vmdb/spec/models/miq_report_spec.rb | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

Comment 15 CFME Bot 2016-03-01 22:10:49 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=d0d46372d98d6da02f6ce094ffcdb3838e47e6d7

commit d0d46372d98d6da02f6ce094ffcdb3838e47e6d7
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Tue Feb 16 09:00:51 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Mar 1 14:43:09 2016 -0500

    Add a test for MiqReport#generate_table with a has_many through.
    
    Remove unused common test setup.
    
    Follow up to #6650
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074

 spec/models/miq_report_spec.rb | 52 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 49 insertions(+), 3 deletions(-)

Comment 16 CFME Bot 2016-03-01 22:10:54 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=c719e038d5e518a078a92057d725a3f265cbb842

commit c719e038d5e518a078a92057d725a3f265cbb842
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Tue Feb 23 15:27:20 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Mar 1 14:43:28 2016 -0500

    Refine the has_many through test based on a reported issue.
    
    The first cut of this test was a first attempt to recreate the problem
    by simplifying the report.
    
    This commit implements the exact reported issue and verifies the backend
    code does the correct thing.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074
    
    Followup to #6650 and #6690

 spec/models/miq_report_spec.rb | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

Comment 17 CFME Bot 2016-03-01 22:10:59 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=24fd1825b0b4cb2dfa066e608fe3b70358d8a05e

commit 24fd1825b0b4cb2dfa066e608fe3b70358d8a05e
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Fri Feb 12 15:02:45 2016 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Mar 1 14:32:30 2016 -0500

    Specify the has_many through source using the belongs_to, not an alias.
    
    Also, specify the source_type as rails requests.
    
    Note, we specify VmOrTemplate here because VmOrTemplate is where this
    has_many is located.  It seems like this has_many may belong in MiqTemplate.
    
    Fixes the following error:
    ```
    ActiveRecord::HasManyThroughSourceAssociationNotFoundError:
      Could not find the source association(s) :vm in model MiqProvision.
      Try 'has_many :miq_provision_vms, :through => :miq_provisions_from_template, :source => <name>'.
      Is it one of miq_request, source, destination, miq_request_tasks, miq_request_task, or tenant?
    ```
    
    This resolves the model bug reported in:
    https://bugzilla.redhat.com/show_bug.cgi?id=1303074
    
    We still need to see if we can generate reports based on this now fixed
    has_many :through.

 app/models/vm_or_template.rb       |  2 +-
 spec/models/vm_or_template_spec.rb | 26 ++++++++++++++++++++++++++
 2 files changed, 27 insertions(+), 1 deletion(-)

Comment 18 Shveta 2016-04-18 20:32:02 UTC
Verified in 5.6.0.1-beta2.20160413141124_e25ac0e

Comment 20 errata-xmlrpc 2016-06-29 15:35:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348