Description of problem: After doing the upgrade of CloudForms from version 4.5 to 4.6 the evmserverd fails to start. rake evm:status displays the below error Checking EVM status... Zone | Server | Status | ID | PID | SPID | URL | Started On | Last Heartbeat | MB Usage | Master? | Active Roles ---------+--------+---------+-----------------+------+------+---------------------------------------------------+----------------------+----------------------+----------+---------+---------------- default | DSG | quiesce | 101000000000001 | 4882 | 5011 | drbunix:///tmp/worker_monitor20181225-4882-1j2zke | 2018-12-24T20:01:46Z | 2018-12-24T20:34:14Z | 370 | false | database_owner rake aborted! ActiveRecord::SubclassNotFound: The single-table inheritance mechanism failed to locate the subclass: 'ManageIQ::Providers::StorageManager::CinderManager::EventCatcher'. This error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this column if you didn't intend it to be used for storing the inheritance class or overwrite MiqWorker.inheritance_column to use another column for that information. /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/inheritance.rb:182:in `rescue in find_sti_class' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/inheritance.rb:175:in `find_sti_class' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/inheritance.rb:163:in `discriminate_class_for_record' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/persistence.rb:67:in `instantiate' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:50:in `block (2 levels) in find_by_sql' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/result.rb:52:in `block in each' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/result.rb:52:in `each' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/result.rb:52:in `each' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:50:in `map' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:50:in `block in find_by_sql' /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/notifications/instrumenter.rb:21:in `instrument' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:49:in `find_by_sql' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation.rb:706:in `exec_queries' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/association_relation.rb:32:in `exec_queries' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation.rb:583:in `load' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation.rb:260:in `records' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation/delegation.rb:38:in `each' /var/www/miq/vmdb/lib/tasks/evm_application.rb:96:in `block in output_workers_status' /var/www/miq/vmdb/lib/tasks/evm_application.rb:95:in `output_workers_status' /var/www/miq/vmdb/lib/tasks/evm_application.rb:69:in `status' /var/www/miq/vmdb/lib/tasks/evm.rake:45:in `block (2 levels) in <top (required)>' /opt/rh/cfme-gemset/gems/rake-12.3.0/exe/rake:27:in `<top (required)>' Caused by: NameError: uninitialized constant ManageIQ::Providers::StorageManager::CinderManager::EventCatcher /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/inflector/methods.rb:270:in `const_get' /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/inflector/methods.rb:270:in `block in constantize' /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/inflector/methods.rb:266:in `each' /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/inflector/methods.rb:266:in `inject' /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/inflector/methods.rb:266:in `constantize' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/inheritance.rb:177:in `find_sti_class' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/inheritance.rb:163:in `discriminate_class_for_record' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/persistence.rb:67:in `instantiate' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:50:in `block (2 levels) in find_by_sql' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/result.rb:52:in `block in each' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/result.rb:52:in `each' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/result.rb:52:in `each' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:50:in `map' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:50:in `block in find_by_sql' /opt/rh/cfme-gemset/gems/activesupport-5.0.6/lib/active_support/notifications/instrumenter.rb:21:in `instrument' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/querying.rb:49:in `find_by_sql' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation.rb:706:in `exec_queries' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/association_relation.rb:32:in `exec_queries' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation.rb:583:in `load' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation.rb:260:in `records' /opt/rh/cfme-gemset/gems/activerecord-5.0.6/lib/active_record/relation/delegation.rb:38:in `each' /var/www/miq/vmdb/lib/tasks/evm_application.rb:96:in `block in output_workers_status' /var/www/miq/vmdb/lib/tasks/evm_application.rb:95:in `output_workers_status' /var/www/miq/vmdb/lib/tasks/evm_application.rb:69:in `status' /var/www/miq/vmdb/lib/tasks/evm.rake:45:in `block (2 levels) in <top (required)>' /opt/rh/cfme-gemset/gems/rake-12.3.0/exe/rake:27:in `<top (required)>' Tasks: TOP => evm:status (See full trace by running task with --trace) Version-Release number of selected component (if applicable): cfme-5.9.6.5-4.el7cf.x86_64 cfme-appliance-5.9.6.5-1.el7cf.x86_64 cfme-gemset-5.9.6.5-3.el7cf.x86_64 cfme-appliance-common-5.9.6.5-1.el7cf.x86_64 How reproducible: Steps to Reproduce: 1. Configure CF 4.5 with OpenStack 10 provider 2. Upgrade to CF 4.6 and apply hotfix 5.9.6.5-4 and gemset 5.9.6.5-3 3. reboot and attempt to start evmserverd Actual results: evmserverd is not starting and displaying the above error Expected results: evmserverd is able to start normally Additional info:
It looks like this class was removed here [1] but the miq_workers rows were not removed. We'll need to write a migration to remove these rows. Similar to bug 1644083 and bug 1655794. Until we have the fixed data migration released, the user can open a rails console and do the following: MiqWorker.where(:type => "ManageIQ::Providers::StorageManager::CinderManager::EventCatcher").delete_all [1] https://github.com/ManageIQ/manageiq/pull/14962
PR with data migration https://github.com/ManageIQ/manageiq-schema/pull/320
Reverting the 5.9.z -> 5.9.8 change. Migrations do not run on on Z-Stream updates, so the proposed PR will not help. If there is a way to resolve this in the 5.9 branch it can be considered for a future errata. Please verify whether this problem exists in 5.10.0.
IMO, we need to find a better general approach to this. We have a number of migrations that fix specific problems, but I think we need to do a more strategic fix that clears all errant worker records, regardless of STI.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days