Description of problem: Upgraded from 5.5 to 5.6 and am now getting the below error consistently in the evm.log. Following steps have been performed: - systemctl stop evmserverd - yum update - vmdb - rake db:migrate - rake evm:automate:reset - systemctl start evmserverd Snippet of evm.log ========================================================================= [----] E, [2016-08-09T10:42:37.008774 #11561:6a7994] ERROR -- : MIQ(MiqPriorityWorker::Runner) ID [1000000016152] PID [11561] GUID [449c667e-5e3f-11e6-b939-005056b83c74] An error has occurred during work processing: MIQ(MiqPriorityWorker::Runner) "undefined class/module ActiveRecord::ConnectionAdapters::PostgreSQL::OID::Integer" attempting to get next message /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:67:in `rescue in block in get_message_via_drb' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:59:in `block in get_message_via_drb' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:36:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:36:in `get_message_via_drb' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:87:in `get_message' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:150:in `block in do_work' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:146:in `loop' /var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:146:in `do_work' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:334:in `block in do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:331:in `loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:331:in `do_work_loop' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:128:in `start' /var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:346:in `block in start' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork' /var/www/miq/vmdb/app/models/miq_worker.rb:344:in `start' /var/www/miq/vmdb/app/models/miq_worker.rb:274:in `start_worker' /var/www/miq/vmdb/app/models/miq_worker.rb:154:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_worker.rb:154:in `times' /var/www/miq/vmdb/app/models/miq_worker.rb:154:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:52:in `block in sync_workers' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each' /var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers' /var/www/miq/vmdb/app/models/miq_server.rb:173:in `start' /var/www/miq/vmdb/app/models/miq_server.rb:265:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start' /var/www/miq/vmdb/lib/workers/evm_server.rb:92:in `start' /var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>' Worker exiting. ========================================================================= Version-Release number of selected component (if applicable): 5.6 Additional info:
Jared, can we get a dump of the customer's DB? If we can't get the entire DB then maybe just the miq_queue table. I'm thinking there may be a specific message in the queue that is causing this error.
I don't even know what this is, but created an account just to make this comment! This is the only place I have found that is getting - word for word - the exact same error as me. I'm working through a comprehensive upgrade of a decent sized 3 year old 7000+ commit Rails application and am hitting this error in one instance with the `audited` gem calling `associated_audits` on an object. I hope that helps you a bit, I will be keeping a close eye on this thread as this develops!
FYI, Nick, I believe Gregg's comment in comment 2 is really the only way to solve this. It hasn't been reported. In our case, I'm guessing there's something in our miq_queue row (which contains serialized yaml columns) that prevent us from updating that row with the reported error. Perhaps, the miq_queue table contained yaml serialized columns from a prior ruby and/or rails version and now it barfs on it. We have to get a recreation table export to understand it fully. Have you been able to track down your issue in https://github.com/collectiveidea/audited/issues/289? It seems yours is with yaml columns and contents of columns.
This commit in rails [1] removed ActiveRecord::ConnectionAdapters::PostgreSQL::OID::Integer. Once we upgraded to rails 5.0.0, this class no longer exists. It is possible that an existing miq_queue row created with rails 4.2 could have one of these private classes serialized in the args column. Note, it's unclear why we're able to find the row but can't call update_attributes [2] on this row with the error: ArgumentError: undefined class/module ActiveRecord::ConnectionAdapters::PostgreSQL::OID::Integer [1] https://github.com/rails/rails/commit/aafee233fb3b4211ee0bfb1fca776c159bd1067e [2] https://github.com/ManageIQ/manageiq/blob/darga-5/app/models/miq_queue_worker_base/runner.rb#L60
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days