Bug 1522846 - Service names starting with 'VM-' can cause report generation failures with "`load_missing_constant': Unable to autoload constant VM"
Summary: Service names starting with 'VM-' can cause report generation failures with "...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Reporting
Version: 5.8.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.9.0
Assignee: Tim Wade
QA Contact: Niyaz Akhtar Ansari
URL:
Whiteboard:
Depends On: 1521167
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-12-06 15:06 UTC by Sudhir Mallamprabhakara
Modified: 2021-06-10 13:51 UTC (History)
12 users (show)

Fixed In Version: 5.9.0.13
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1521167
Environment:
Last Closed: 2018-03-01 13:22:12 UTC
Category: ---
Cloudforms Team: CFME Core
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3262661 0 None None None 2017-12-06 15:06:15 UTC
Red Hat Product Errata RHSA-2018:0380 0 normal SHIPPED_LIVE Moderate: Red Hat CloudForms security, bug fix, and enhancement update 2018-03-01 18:37:12 UTC

Description Sudhir Mallamprabhakara 2017-12-06 15:06:15 UTC
+++ This bug was initially created as a clone of Bug #1521167 +++

Description of problem:
=======================
Running reports with conditionals, specific character arrangements with named services (in our case, VM-test or MY-test matches but not vm-test or My-Test), and so long as the service has a one or more instances within it, report generations will fail with something like the following (example traceback is VM-test service with a single VM):

[----] E, [2017-12-05T13:17:51.196617 #1955:83714c] ERROR -- : [LoadError]: Unable to autoload constant VM, expected /var/www/miq/vmdb/app/models/vm.rb to define it  Method:[rescue in _async_generate_table]
[----] W, [2017-12-05T13:17:51.224421 #1955:83714c]  WARN -- : <AuditFailure> MIQ(Async.rescue in _async_generate_table) userid: [admin] - Unable to autoload constant VM, expected /var/www/miq/vmdb/app/models/vm.
rb to define it
/opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/dependencies.rb:512:in `load_missing_constant': Unable to autoload constant VM, expected /var/www/miq/vmdb/app/models/vm.rb to define it (LoadError)
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/dependencies.rb:203:in `const_missing'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/inflector/methods.rb:268:in `const_get'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/inflector/methods.rb:268:in `block in constantize'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/inflector/methods.rb:266:in `each'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/inflector/methods.rb:266:in `inject'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/inflector/methods.rb:266:in `constantize'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/inflector/methods.rb:311:in `safe_constantize'
        from /opt/rh/cfme-gemset/gems/activesupport-5.0.3/lib/active_support/core_ext/string/inflections.rb:77:in `safe_constantize'
        from /var/www/miq/vmdb/lib/miq_expression/field.rb:25:in `is_field?'
        from /var/www/miq/vmdb/lib/miq_expression.rb:993:in `quote'
        from /var/www/miq/vmdb/app/models/condition.rb:121:in `_subst'
        from /var/www/miq/vmdb/app/models/condition.rb:101:in `block in subst'
        from /var/www/miq/vmdb/app/models/condition.rb:101:in `gsub!'
        from /var/www/miq/vmdb/app/models/condition.rb:101:in `subst'
        from /var/www/miq/vmdb/app/models/condition.rb:85:in `subst_matches?'
        from /var/www/miq/vmdb/lib/miq_expression.rb:821:in `lenient_evaluate'
        from /var/www/miq/vmdb/lib/rbac/filterer.rb:622:in `matches_search_filters?'
        from /var/www/miq/vmdb/lib/rbac/filterer.rb:246:in `block in search'
        from /opt/rh/cfme-gemset/gems/activerecord-5.0.3/lib/active_record/relation/delegation.rb:40:in `each'
        from /opt/rh/cfme-gemset/gems/activerecord-5.0.3/lib/active_record/relation/delegation.rb:40:in `each'
        from /var/www/miq/vmdb/lib/rbac/filterer.rb:246:in `reject'
        from /var/www/miq/vmdb/lib/rbac/filterer.rb:246:in `search'
        from /var/www/miq/vmdb/lib/rbac/filterer.rb:114:in `search'
        from /var/www/miq/vmdb/lib/rbac.rb:3:in `search'
        from /var/www/miq/vmdb/app/models/miq_report/generator.rb:304:in `_generate_table'
        from /var/www/miq/vmdb/app/models/miq_report/generator.rb:188:in `block in generate_table'
        from /var/www/miq/vmdb/app/models/user.rb:245:in `with_user'
        from /var/www/miq/vmdb/app/models/miq_report/generator.rb:188:in `generate_table'
        from /var/www/miq/vmdb/app/models/miq_report/generator/async.rb:96:in `_async_generate_table'
        from /var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver'
        from /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout'
        from /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch'
        from /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'
        from /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'
        from /opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:106:in `timeout'
        from /var/www/miq/vmdb/app/models/miq_queue.rb:343:in `deliver'
        from /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:107:in `deliver_queue_message'
        from /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:135:in `deliver_message'
        from /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:153:in `block in do_work'
        from /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `loop'
        from /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:147:in `do_work'
        from /var/www/miq/vmdb/app/models/miq_worker/runner.rb:340:in `block in do_work_loop'
        from /var/www/miq/vmdb/app/models/miq_worker/runner.rb:337:in `loop'
        from /var/www/miq/vmdb/app/models/miq_worker/runner.rb:337:in `do_work_loop'
        from /var/www/miq/vmdb/app/models/miq_worker/runner.rb:160:in `run'
        from /var/www/miq/vmdb/app/models/miq_worker/runner.rb:134:in `start'
        from /var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker'
        from /var/www/miq/vmdb/app/models/miq_worker.rb:339:in `block in start_runner'

Running a report without conditions will not trigger this issue.

After some debugging, I believe this is due to a regular expression in the MiqExpression::Field class here:

class MiqExpression::Field < MiqExpression::Target
  REGEX = /
(?<model_name>([[:upper:]][[:alnum:]]*(::)?)+)
(?!.*\b(managed|user_tag)\b)
\.?(?<associations>[a-z][0-9a-z_\.]+)?
-
(?:
  (?<virtual_custom_column>#{CustomAttributeMixin::CUSTOM_ATTRIBUTES_PREFIX}[a-z]+[:_\-.\/[:alnum:]]*)|
  (?<column>[a-z]+(_[[:alnum:]]+)*)
)
/x

For ease, the string provided by the above module is:

irb(main):001:0> CustomAttributeMixin::CUSTOM_ATTRIBUTES_PREFIX
=> "virtual_custom_attribute_"

The following regular expression, in the method is_field? will successfull match the string 'VM' from 'VM-test' causing the method to believe that, instead of a name in a column, it is a model that should be found, and therefore when attempting to run safe_constantize we except (as there is no VM constant):

  def self.is_field?(field)
    return false unless field.kind_of?(String)
    match = REGEX.match(field)				<===successfully matches 'VM' and therefore continues on past next statement
    return false unless match
    model = match[:model_name].safe_constantize		<===excepts here
    return false unless model
    !!(model < ApplicationRecord)
  end

Proving the odd matching and a few different scenarios:

irb(main):001:0> REGEX = /
irb(main):002:0/ (?<model_name>([[:upper:]][[:alnum:]]*(::)?)+)
irb(main):003:0/ (?!.*\b(managed|user_tag)\b)
irb(main):004:0/ \.?(?<associations>[a-z][0-9a-z_\.]+)?
irb(main):005:0/ -
irb(main):006:0/ (?:
irb(main):007:0/   (?<virtual_custom_column>virtual_custom_attribute_[a-z]+[:_\-.\/[:alnum:]]*)|
irb(main):008:0/   (?<column>[a-z]+(_[[:alnum:]]+)*)
irb(main):009:0/ )
irb(main):010:0/ /x
- - - - - - 8< - - - - - - 
irb(main):011:0> match = REGEX.match("VM-test")
=> #<MatchData "VM-test" model_name:"VM" associations:nil virtual_custom_column:nil column:"test">
irb(main):012:0> match = REGEX.match("vm-test")
=> nil
irb(main):013:0> match = REGEX.match("MY-test")
=> #<MatchData "MY-test" model_name:"MY" associations:nil virtual_custom_column:nil column:"test">
irb(main):014:0> match = REGEX.match("MYtest")
=> nil
irb(main):015:0> match = REGEX.match("My-Test")
=> nil
irb(main):016:0> match = REGEX.match("MY-Test")
=> nil
irb(main):017:0> match = REGEX.match("VM-Test")
=> nil
irb(main):018:0> match = REGEX.match("VM-test")
=> #<MatchData "VM-test" model_name:"VM" associations:nil virtual_custom_column:nil column:"test">
irb(main):019:0> match = REGEX.match("My-test")
=> #<MatchData "My-test" model_name:"My" associations:nil virtual_custom_column:nil column:"test">
irb(main):020:0> match = REGEX.match("My-potato")
=> #<MatchData "My-potato" model_name:"My" associations:nil virtual_custom_column:nil column:"potato">

This is due to the named capture group model_name:

	(?<model_name>([[:upper:]][[:alnum:]]*(::)?)+)

I see this is easy enough to change to ignore the above scenarios, however, since I don't know specifically what it would always be used for I am hesitant to submit a PR "fixing" this.

Version-Release number of selected component (if applicable):
=============================================================
Tested on 5.8.2.3 and on upstream ManageIQ/fine (devel branch from master)

How reproducible:
=================
Every time

Steps to Reproduce:
===================
1. Create a service with one of the above naming conventions
2. Have at least one VM in the service so the reporting will parse it
3. Create a report with a conditional filter in it, such as:
    conditions: !ruby/object:MiqExpression
      exp:
        and:
        - IS NOT NULL:
            field: Vm.service-name
        - IS NOT NULL:
            field: Vm-ems_cluster_name
3. Run the report, and it will traceback like the above.

Actual results:
===============
Report generation fails due to matching "model" regex conventions above

Expected results:
=================
Report generation should not fail

Additional info:
================
An easy workaround is simply changing the service name to not match the above regex, or best by avoiding dashes all together.

--- Additional comment from Robb Manes on 2017-12-05 17:30:05 EST ---

After some more debugging, the rest of the is_field? method catches most other matches, and services starting with strings containing 'VM-' are what cause the exception every time.  I can't seem to have any other named-service cause the above exception despite matching because we will fall into `return false unless field.kind_of?(String)`, whereas 'VM-' passes this conditional.

Comment 2 CFME Bot 2017-12-11 17:21:54 UTC
New commit detected on ManageIQ/manageiq/gaprindashvili:
https://github.com/ManageIQ/manageiq/commit/b175209421598ddb0f9743888f879d346f8d7e0e

commit b175209421598ddb0f9743888f879d346f8d7e0e
Author:     Jason Frey <fryguy9>
AuthorDate: Wed Dec 6 13:30:30 2017 -0500
Commit:     Satoe Imaishi <simaishi>
CommitDate: Mon Dec 11 12:12:11 2017 -0500

    Merge pull request #16608 from imtayadeway/bug/miq-expression-field-values
    
    Handle autoload error not caught by safe_constantize
    (cherry picked from commit 7a5fd377e48852964039bf9d674e9426cde34513)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1522846

 lib/miq_expression/field.rb     | 7 ++++++-
 spec/lib/miq_expression_spec.rb | 5 +++++
 2 files changed, 11 insertions(+), 1 deletion(-)

Comment 3 Niyaz Akhtar Ansari 2018-01-22 06:30:59 UTC
It works fine with no error.
Verified in Version 5.9.0.17.20180116225234_ac8b6f5

Comment 6 errata-xmlrpc 2018-03-01 13:22:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0380


Note You need to log in before you can comment on or make changes to this bug.