Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 1484007 Details for
Bug 1593349
Can't delete selected router at the Networks -> Network Routers page
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
evm.log
evm.log (text/plain), 236.13 KB, created by
Danylo Kholodov
on 2018-09-17 12:37:53 UTC
(
hide
)
Description:
evm.log
Filename:
MIME Type:
Creator:
Danylo Kholodov
Created:
2018-09-17 12:37:53 UTC
Size:
236.13 KB
patch
obsolete
>[root@localhost log]# tail -1000 evm.log >[----] I, [2018-09-17T08:19:40.259147 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] VmallocTotal: 35184372087808 >[----] I, [2018-09-17T08:19:40.259203 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] VmallocUsed: 42184704 >[----] I, [2018-09-17T08:19:40.259259 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] Writeback: 8192 >[----] I, [2018-09-17T08:19:40.259336 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] WritebackTmp: 0 >[----] I, [2018-09-17T08:19:40.291910 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12099], MiqWorker id: [2], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [6.532933634] seconds >[----] I, [2018-09-17T08:19:40.292013 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12099], Delivering... >[----] I, [2018-09-17T08:19:40.342118 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] Disk Usage: >[----] I, [2018-09-17T08:19:40.342334 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] Filesystem Type Total Used Available %Used iTotal iUsed iFree %iUsed Mounted on >[----] I, [2018-09-17T08:19:40.388170 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_os xfs 4.49 GB 2.31 GB 2.18 GB 52% 2359296 101690 2257606 5% / >[----] I, [2018-09-17T08:19:40.390962 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] devtmpfs devtmpfs 4.3 GB 0 Bytes 4.3 GB 0% 1126918 432 1126486 1% /dev >[----] I, [2018-09-17T08:19:40.393471 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] tmpfs tmpfs 4.31 GB 20 KB 4.31 GB 1% 1130246 16 1130230 1% /dev/shm >[----] I, [2018-09-17T08:19:40.396272 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_repo xfs 509 MB 25.8 MB 483 MB 6% 262144 3 262141 1% /repo >[----] I, [2018-09-17T08:19:40.398590 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_home xfs 1010 MB 32.2 MB 982 MB 4% 524288 3 524285 1% /home >[----] I, [2018-09-17T08:19:40.400716 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/vda1 xfs 1010 MB 163 MB 851 MB 17% 524288 326 523962 1% /boot >[----] I, [2018-09-17T08:19:40.402829 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_tmp xfs 1010 MB 32.3 MB 982 MB 4% 524288 17 524271 1% /tmp >[----] I, [2018-09-17T08:19:40.405707 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_var xfs 12 GB 970 MB 11 GB 8% 6291456 71580 6219876 2% /var >[----] I, [2018-09-17T08:19:40.393854 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12099], State: [ok], Delivered in [0.101828165] seconds >[----] I, [2018-09-17T08:19:40.410128 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_log xfs 9.99 GB 109 MB 9.88 GB 2% 5242880 33 5242847 1% /var/www/miq/vmdb/log >[----] I, [2018-09-17T08:19:40.412373 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_var_log xfs 1010 MB 40.5 MB 974 MB 4% 524288 37 524251 1% /var/log >[----] I, [2018-09-17T08:19:40.424373 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/VG--CFME-lv_var_log_audit xfs 509 MB 25.8 MB 483 MB 6% 262144 4 262140 1% /var/log/audit >[----] I, [2018-09-17T08:19:40.427113 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] /dev/mapper/vg_pg-lv_pg xfs 4.99 GB 152 MB 4.84 GB 3% 2619392 3322 2616070 1% /var/opt/rh/rh-postgresql95/lib/pgsql >[----] I, [2018-09-17T08:19:40.430265 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] MiqQueue count for state=["dequeue"] by zone and role: {"default"=>{nil=>2}} >[----] I, [2018-09-17T08:19:40.430370 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_system_status) [EVM Server (24762)] MiqQueue count for state=["ready"] by zone and role: {"default"=>{nil=>3}} >[----] I, [2018-09-17T08:19:40.433919 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqServer.log_status) [EVM Server (24762)] Process info: Memory Usage [375971840], Memory Size [767401984], Proportional Set Size: [229364000], Memory % [4.06], CPU Time [28770.0], CPU % [3.85], Priority [20] >[----] I, [2018-09-17T08:19:40.434409 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12100], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [6.669638598] seconds >[----] I, [2018-09-17T08:19:40.434513 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12100], Delivering... >[----] I, [2018-09-17T08:19:40.436475 #27798:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqQueue#delivered) Message id: [12097], State: [ok], Delivered in [0.181343936] seconds >[----] I, [2018-09-17T08:19:40.438413 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12100], State: [ok], Delivered in [0.003897268] seconds >[----] I, [2018-09-17T08:19:40.450226 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12102], MiqWorker id: [1], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [65], PID [11041], GUID [f23ec03c-ba73-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}], Dequeued in: [1.932587184] seconds >[----] I, [2018-09-17T08:19:40.450345 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12102], Delivering... >[----] I, [2018-09-17T08:19:40.456775 #27798:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"evm_worker_start"> inputs=<{:event_details=>"Worker started: ID [65], PID [11041], GUID [f23ec03c-ba73-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5845, :miq_event_id=>5845, "EventStream::event_stream"=>5845, :event_stream_id=>5845}> >[----] I, [2018-09-17T08:19:40.474111 #27798:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12104], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [65], PID [11041], GUID [f23ec03c-ba73-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5845, :miq_event_id=>5845, "EventStream::event_stream"=>5845, :event_stream_id=>5845}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:19:40.474379 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12102], State: [ok], Delivered in [0.024026718] seconds >[----] W, [2018-09-17T08:19:40.673214 #28776:6226a90] WARN -- : MIQ(OpenstackEventMonitor.available_event_monitor) Error occured testing OpenstackCeilometerEventMonitor > for . Event collection will be disabled for > . MiqException::ServiceNotAvailable >[----] W, [2018-09-17T08:19:40.673263 #28776:6226a90] WARN -- : MIQ(OpenstackNullEventMonitor#initialize) There was an problem establishing a connection to the AMQP service on . Check the evm.log for more details. >[----] I, [2018-09-17T08:19:40.673405 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:121:in `monitor_events': must be implemented in subclass (NotImplementedError) > from /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:163:in `block in start_event_monitor' >[----] W, [2018-09-17T08:19:41.553871 #28786:61e7368] WARN -- : MIQ(OpenstackEventMonitor.available_event_monitor) Error occured testing OpenstackCeilometerEventMonitor > for . Event collection will be disabled for > . MiqException::ServiceNotAvailable >[----] W, [2018-09-17T08:19:41.553921 #28786:61e7368] WARN -- : MIQ(OpenstackNullEventMonitor#initialize) There was an problem establishing a connection to the AMQP service on . Check the evm.log for more details. >[----] I, [2018-09-17T08:19:41.554032 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:19:44.764847 #24762:a51108] INFO -- : MiqServer: local=Y, master=Y, status= started, id=00001, pid=24762, guid=b855343c-ba6e-11e8-85b9-525400153f41, name=EVM, zone=default, hostname=, ipaddress=10.0.0.30, version=5.8.5.1, build=20180911143221_8357814, active roles=automate:database_operations:database_owner:ems_inventory:ems_operations:event:reporting:scheduler:smartstate:user_interface:web_services:websocket >[----] I, [2018-09-17T08:19:44.767051 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 3 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:19:44.990009 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:19:45.506392 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12098], MiqWorker id: [3], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [log_status], Command: [MiqWorker.log_status_all], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [11.752265826] seconds >[----] I, [2018-09-17T08:19:45.506512 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqQueue#deliver) Message id: [12098], Delivering... >[----] I, [2018-09-17T08:19:45.513627 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12104], MiqWorker id: [4], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [65], PID [11041], GUID [f23ec03c-ba73-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5845, :miq_event_id=>5845, "EventStream::event_stream"=>5845, :event_stream_id=>5845}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [5.046029756] seconds >[----] I, [2018-09-17T08:19:45.513852 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12104], Delivering... >[----] I, [2018-09-17T08:19:45.517135 #27824:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [2] name [EvmGroup-super_administrator] >[----] I, [2018-09-17T08:19:45.517295 #27824:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [65], PID [11041], GUID [f23ec03c-ba73-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5845, :miq_event_id=>5845, "EventStream::event_stream"=>5845, :event_stream_id=>5845} for object [MiqServer.1] with state [] to Automate >[----] I, [2018-09-17T08:19:45.520151 #27824:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5845&MiqEvent%3A%3Amiq_event=5845&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B65%5D%2C%20PID%20%5B11041%5D%2C%20GUID%20%5Bf23ec03c-ba73-11e8-86bf-525400153f41%5D&event_stream_id=5845&event_type=evm_worker_start&miq_event_id=5845&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server] >[----] I, [2018-09-17T08:19:45.533454 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::NetworkManager::RefreshWorker#log_status) [Refresh Worker for Providers: uc_ci Network Manager] Worker ID [12], PID [28758], GUID [6b6e251e-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:35 UTC], Process Info: Memory Usage [330375168], Memory Size [673366016], Proportional Set Size: [214636000], Memory % [3.57], CPU Time [294.0], CPU % [0.04], Priority [27] >[----] I, [2018-09-17T08:19:45.534669 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::NetworkManager::RefreshWorker#log_status) [Refresh Worker for Providers: oc_ci Network Manager] Worker ID [17], PID [28958], GUID [93884994-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:29 UTC], Process Info: Memory Usage [416677888], Memory Size [761802752], Proportional Set Size: [306364000], Memory % [4.5], CPU Time [5865.0], CPU % [0.78], Priority [27] >[----] I, [2018-09-17T08:19:45.536039 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::InfraManager::RefreshWorker#log_status) [Refresh Worker for Providers: uc_ci] Worker ID [13], PID [28767], GUID [6be2c27a-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:31 UTC], Process Info: Memory Usage [400138240], Memory Size [735653888], Proportional Set Size: [291328000], Memory % [4.32], CPU Time [1276.0], CPU % [0.17], Priority [27] >[----] I, [2018-09-17T08:19:45.537288 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::StorageManager::CinderManager::RefreshWorker#log_status) [Refresh Worker for Providers: oc_ci Cinder Manager] Worker ID [18], PID [28968], GUID [93c425f4-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:35 UTC], Process Info: Memory Usage [362803200], Memory Size [709435392], Proportional Set Size: [248500000], Memory % [3.92], CPU Time [677.0], CPU % [0.09], Priority [27] >[----] I, [2018-09-17T08:19:45.538576 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::StorageManager::SwiftManager::RefreshWorker#log_status) [Refresh Worker for Providers: oc_ci Swift Manager] Worker ID [19], PID [28976], GUID [9400e7be-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:30 UTC], Process Info: Memory Usage [362741760], Memory Size [709906432], Proportional Set Size: [247144000], Memory % [3.92], CPU Time [594.0], CPU % [0.08], Priority [27] >[----] I, [2018-09-17T08:19:45.539834 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5845&MiqEvent%3A%3Amiq_event=5845&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B65%5D%2C%20PID%20%5B11041%5D%2C%20GUID%20%5Bf23ec03c-ba73-11e8-86bf-525400153f41%5D&event_stream_id=5845&event_type=evm_worker_start&miq_event_id=5845&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server ManageIQ/System] >[----] I, [2018-09-17T08:19:45.539898 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::CloudManager::EventCatcher#log_status) [Event Monitor for Providers: oc_ci] Worker ID [20], PID [28987], GUID [9427206e-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:25 UTC], Process Info: Memory Usage [390078464], Memory Size [742641664], Proportional Set Size: [273480000], Memory % [4.21], CPU Time [7359.0], CPU % [0.98], Priority [21] >[----] I, [2018-09-17T08:19:45.541118 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher#log_status) [Event Monitor for Providers: uc_ci Network Manager] Worker ID [14], PID [28776], GUID [6c2d2f68-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:39 UTC], Process Info: Memory Usage [330469376], Memory Size [675471360], Proportional Set Size: [199948000], Memory % [3.57], CPU Time [315.0], CPU % [0.04], Priority [21] >[----] I, [2018-09-17T08:19:45.542368 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher#log_status) [Event Monitor for Providers: oc_ci Network Manager] Worker ID [21], PID [28996], GUID [9464db98-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:35 UTC], Process Info: Memory Usage [382746624], Memory Size [737034240], Proportional Set Size: [265484000], Memory % [4.13], CPU Time [3697.0], CPU % [0.49], Priority [21] >[----] I, [2018-09-17T08:19:45.543560 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher#log_status) [Event Monitor for Providers: uc_ci] Worker ID [15], PID [28786], GUID [6c7af6ee-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:25 UTC], Process Info: Memory Usage [330424320], Memory Size [677576704], Proportional Set Size: [199660000], Memory % [3.57], CPU Time [277.0], CPU % [0.04], Priority [21] >[----] I, [2018-09-17T08:19:45.543950 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqEventHandler#log_status) [Event Handler] Worker ID [6], PID [27870], GUID [06645734-ba6f-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:36 UTC], Process Info: Memory Usage [394428416], Memory Size [721850368], Proportional Set Size: [294440000], Memory % [4.26], CPU Time [20580.0], CPU % [2.75], Priority [27] >[----] I, [2018-09-17T08:19:45.544239 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqGenericWorker#log_status) [Generic Worker] Worker ID [1], PID [27798], GUID [ff10a5f0-ba6e-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:37 UTC], Process Info: Memory Usage [372109312], Memory Size [689885184], Proportional Set Size: [279464000], Memory % [4.02], CPU Time [8115.0], CPU % [1.08], Priority [30] >[----] I, [2018-09-17T08:19:45.544474 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqGenericWorker#log_status) [Generic Worker] Worker ID [2], PID [27806], GUID [ff43334e-ba6e-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:37 UTC], Process Info: Memory Usage [384626688], Memory Size [750243840], Proportional Set Size: [290636000], Memory % [4.15], CPU Time [8071.0], CPU % [1.08], Priority [30] >[----] I, [2018-09-17T08:19:45.545695 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::Openstack::CloudManager::RefreshWorker#log_status) [Refresh Worker for Providers: oc_ci] Worker ID [16], PID [28947], GUID [93332086-ba70-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:34 UTC], Process Info: Memory Usage [419713024], Memory Size [763670528], Proportional Set Size: [310636000], Memory % [4.53], CPU Time [2059.0], CPU % [0.28], Priority [27] >[----] I, [2018-09-17T08:19:45.546031 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqPriorityWorker#log_status) [Priority Worker] Worker ID [3], PID [27816], GUID [ff55880a-ba6e-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:39 UTC], Process Info: Memory Usage [376918016], Memory Size [693039104], Proportional Set Size: [282428000], Memory % [4.07], CPU Time [38791.0], CPU % [5.19], Priority [21] >[----] I, [2018-09-17T08:19:45.546314 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqReportingWorker#log_status) [Reporting Worker] Worker ID [7], PID [27875], GUID [06905334-ba6f-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:38 UTC], Process Info: Memory Usage [323223552], Memory Size [657899520], Proportional Set Size: [177364000], Memory % [3.49], CPU Time [228.0], CPU % [0.03], Priority [27] >[----] I, [2018-09-17T08:19:45.546580 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqReportingWorker#log_status) [Reporting Worker] Worker ID [8], PID [27884], GUID [069a8ae8-ba6f-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:37 UTC], Process Info: Memory Usage [323252224], Memory Size [659054592], Proportional Set Size: [177420000], Memory % [3.49], CPU Time [219.0], CPU % [0.03], Priority [27] >[----] I, [2018-09-17T08:19:45.546876 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqScheduleWorker#log_status) [Schedule Worker] Worker ID [5], PID [27835], GUID [ff7a3af6-ba6e-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:33 UTC], Process Info: Memory Usage [316076032], Memory Size [651268096], Proportional Set Size: [217564000], Memory % [3.41], CPU Time [810.0], CPU % [0.11], Priority [23] >[----] I, [2018-09-17T08:19:45.547123 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqWebsocketWorker#log_status) [Websocket Worker] Worker ID [9], PID [27899], GUID [06b347cc-ba6f-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:30 UTC], Process Info: Memory Usage [326606848], Memory Size [670187520], Proportional Set Size: [197816000], Memory % [3.53], CPU Time [269.0], CPU % [0.04], Priority [21] >[----] I, [2018-09-17T08:19:45.547372 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqUiWorker#log_status) [User Interface Worker] Worker ID [10], PID [27908], GUID [071a0822-ba6f-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:30 UTC], Process Info: Memory Usage [421867520], Memory Size [755331072], Proportional Set Size: [335132000], Memory % [4.56], CPU Time [1207.0], CPU % [0.16], Priority [21] >[----] I, [2018-09-17T08:19:45.547628 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqWebServiceWorker#log_status) [Web Services Worker] Worker ID [11], PID [27917], GUID [07302300-ba6f-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:30 UTC], Process Info: Memory Usage [341110784], Memory Size [678043648], Proportional Set Size: [244404000], Memory % [3.68], CPU Time [465.0], CPU % [0.06], Priority [21] >[----] I, [2018-09-17T08:19:45.548859 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#log_status) [Event Monitor for Providers: oc_ci Cinder Manager] Worker ID [65], PID [11041], GUID [f23ec03c-ba73-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:40 UTC], Process Info: Memory Usage [], Memory Size [], Proportional Set Size: [], Memory % [], CPU Time [], CPU % [], Priority [] >[----] I, [2018-09-17T08:19:45.549123 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqPriorityWorker#log_status) [Priority Worker] Worker ID [4], PID [27824], GUID [ff5c6828-ba6e-11e8-86bf-525400153f41], Last Heartbeat [2018-09-17 12:19:39 UTC], Process Info: Memory Usage [386637824], Memory Size [702328832], Proportional Set Size: [291628000], Memory % [4.18], CPU Time [39089.0], CPU % [5.23], Priority [21] >[----] I, [2018-09-17T08:19:45.549339 #27816:a51108] INFO -- : Q-task_id([log_status]) MIQ(MiqQueue#delivered) Message id: [12098], State: [ok], Delivered in [0.042821438] seconds >[----] I, [2018-09-17T08:19:45.581796 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:19:45.602470 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:19:45.613007 #27824:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/evm_worker_start] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:19:45.624084 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:19:45.628075 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:19:45.636756 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:19:45.641910 #27824:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:19:45.644740 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<MiqServer id: 1, guid: "b855343c-ba6e-11e8-85b9-525400153f41", status: "started", started_on: "2018-09-17 11:44:11", stopped_on: nil, pid: 24762, build: "20180911143221_8357814", percent_memory: 4.06, percent_cpu: 3.85, cpu_time: 28770.0, name: "EVM", capabilities: {:vixDisk=>true, :concurrent_miqproxies=>0}, last_heartbeat: "2018-09-17 12:19:27", os_priority: 20, is_master: true, logo: nil, version: "5.8.5.1", zone_id: 1, upgrade_status: nil, upgrade_message: nil, memory_usage: 375971840, memory_size: 767401984, hostname: nil, ipaddress: "10.0.0.30", drb_uri: "drbunix:///tmp/worker_monitor20180917-24762-fp1fxm", mac_address: "52:54:00:15:3f:41", vm_id: nil, has_active_userinterface: true, has_active_webservices: true, sql_spid: 27687, rh_registered: nil, rh_subscribed: nil, last_update_check: nil, updates_available: nil, rhn_mirror: false, log_file_depot_id: nil, proportional_set_size: 229364000, has_active_websocket: true, system_memory_free: 1732235264, system_memory_used: 7526739968, system_swap_free: 9122607104, system_swap_used: 0>] >[----] I, [2018-09-17T08:19:45.644912 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [evm_worker_start] >[----] I, [2018-09-17T08:19:45.651483 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [evm_worker_start] >[----] I, [2018-09-17T08:19:45.651621 #27824:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [evm_worker_start] Target: MiqServer Name: [EVM], Id: [1] >[----] I, [2018-09-17T08:19:45.651968 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:19:45.652587 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:19:45.653195 #27824:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12104], State: [ok], Delivered in [0.139381308] seconds >[----] I, [2018-09-17T08:19:45.705950 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:19:46.461681 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12103], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}], Dequeued in: [6.659447784] seconds >[----] I, [2018-09-17T08:19:46.461783 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12103], Delivering... >[----] I, [2018-09-17T08:19:46.467794 #27806:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"ems_auth_valid"> inputs=<{"MiqEvent::miq_event"=>5846, :miq_event_id=>5846, "EventStream::event_stream"=>5846, :event_stream_id=>5846}> >[----] I, [2018-09-17T08:19:46.485641 #27806:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12105], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5846, :miq_event_id=>5846, "EventStream::event_stream"=>5846, :event_stream_id=>5846}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:19:46.485866 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12103], State: [ok], Delivered in [0.024052553] seconds >[----] I, [2018-09-17T08:19:48.778764 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:19:48.786826 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12106], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:19:49.783475 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:19:50.157582 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:19:50.171191 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12107], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:19:50.575544 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12105], MiqWorker id: [3], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5846, :miq_event_id=>5846, "EventStream::event_stream"=>5846, :event_stream_id=>5846}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [4.097794352] seconds >[----] I, [2018-09-17T08:19:50.575661 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12105], Delivering... >[----] I, [2018-09-17T08:19:50.579568 #27816:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [1] name [Tenant My Company access] >[----] I, [2018-09-17T08:19:50.579706 #27816:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5846, :miq_event_id=>5846, "EventStream::event_stream"=>5846, :event_stream_id=>5846} for object [ManageIQ::Providers::Openstack::CloudManager.5] with state [] to Automate >[----] I, [2018-09-17T08:19:50.582066 #27816:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5846&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5846&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5846&event_type=ems_auth_valid&miq_event_id=5846&object_name=Event&vmdb_object_type=ext_management_system] >[----] I, [2018-09-17T08:19:50.607087 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5846&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5846&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5846&event_type=ems_auth_valid&miq_event_id=5846&object_name=Event&vmdb_object_type=ext_management_system ManageIQ/System] >[----] I, [2018-09-17T08:19:50.666333 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:19:50.677578 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12106], MiqWorker id: [4], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [1.893331514] seconds >[----] I, [2018-09-17T08:19:50.677681 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12106], Delivering... >[----] I, [2018-09-17T08:19:50.681647 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.002244710922241211, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0022644996643066406, :pending_vm_jobs=>0.00042629241943359375, :vm_jobs_to_dispatch_count=>0, :total_time=>0.003751039505004883} >[----] I, [2018-09-17T08:19:50.681828 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12106], State: [ok], Delivered in [0.00414393] seconds >[----] I, [2018-09-17T08:19:50.691235 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:19:50.704437 #27816:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/ems_auth_valid] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:19:50.718597 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:19:50.722601 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:19:50.733430 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:19:50.739782 #27816:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:19:50.742841 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<ManageIQ::Providers::Openstack::CloudManager id: 5, name: "oc_ci", created_on: "2018-09-17 11:55:21", updated_on: "2018-09-17 12:11:32", guid: "8e1cb774-ba70-11e8-86bf-525400153f41", zone_id: 1, type: "ManageIQ::Providers::Openstack::CloudManager", api_version: "v3", uid_ems: "default", host_default_vnc_port_start: nil, host_default_vnc_port_end: nil, provider_region: "", last_refresh_error: nil, last_refresh_date: "2018-09-17 12:11:32", provider_id: 1, realm: nil, tenant_id: 1, project: nil, parent_ems_id: nil, subscription: nil, last_metrics_error: nil, last_metrics_update_date: nil, last_metrics_success_date: nil, tenant_mapping_enabled: false>] >[----] I, [2018-09-17T08:19:50.743021 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [ems_auth_valid] >[----] I, [2018-09-17T08:19:50.751254 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [ems_auth_valid] >[----] I, [2018-09-17T08:19:50.751405 #27816:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [ems_auth_valid] Target: ManageIQ::Providers::Openstack::CloudManager Name: [oc_ci], Id: [5] >[----] I, [2018-09-17T08:19:50.758361 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:19:50.759059 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:19:50.760301 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12105], State: [ok], Delivered in [0.184632828] seconds >[----] I, [2018-09-17T08:19:55.685566 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:19:55.685919 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:19:55.686254 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:19:55.686457 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:19:55.686781 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] W, [2018-09-17T08:19:55.755711 #24762:a51108] WARN -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#status_update) No such process [Event Monitor for Providers: oc_ci Cinder Manager] with PID=[11041], aborting worker. >[----] I, [2018-09-17T08:19:55.875426 #24762:a51108] INFO -- : MIQ(MiqServer#clean_worker_records) SQL Record for Worker [ManageIQ::Providers::StorageManager::CinderManager::EventCatcher] with ID: [65], PID: [11041], GUID: [f23ec03c-ba73-11e8-86bf-525400153f41], Status: [aborted] is being deleted >[----] I, [2018-09-17T08:19:55.979782 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:19:55.988899 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0028634071350097656, :worker_monitor=>1.179123878479004, :worker_dequeue=>0.01283407211303711, :total_time=>1.1951181888580322} >[----] I, [2018-09-17T08:19:56.556708 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:19:56.557105 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:19:56.557368 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:19:56.557580 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:19:56.557909 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:19:58.006851 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12107], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [7.842992089] seconds >[----] I, [2018-09-17T08:19:58.006950 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12107], Delivering... >[----] I, [2018-09-17T08:19:58.009260 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12107], State: [ok], Delivered in [0.002308291] seconds >[----] I, [2018-09-17T08:20:00.635171 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:00.989345 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:20:00 UTC]... >[----] I, [2018-09-17T08:20:00.996233 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:20:00 UTC]...Complete >[----] I, [2018-09-17T08:20:01.332939 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:03.795596 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:20:03.802266 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12108], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:20:05.174175 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:20:05.189976 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12109], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:20:06.016981 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:06.018392 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:20:06.738772 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12108], MiqWorker id: [4], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [2.938875129] seconds >[----] I, [2018-09-17T08:20:06.738888 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12108], Delivering... >[----] I, [2018-09-17T08:20:06.743763 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0024755001068115234, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0024976730346679688, :pending_vm_jobs=>0.0007193088531494141, :vm_jobs_to_dispatch_count=>0, :total_time=>0.004482746124267578} >[----] I, [2018-09-17T08:20:06.743964 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12108], State: [ok], Delivered in [0.005077106] seconds >[----] I, [2018-09-17T08:20:07.029895 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12109], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [1.848421359] seconds >[----] I, [2018-09-17T08:20:07.029997 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12109], Delivering... >[----] I, [2018-09-17T08:20:07.032098 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12109], State: [ok], Delivered in [0.002099824] seconds >[----] I, [2018-09-17T08:20:10.689470 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:10.689904 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:10.690181 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:10.690358 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:10.690647 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:11.245193 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.sync_workers) Workers are being synchronized: Current: [], Desired: ["ems_3"] >[----] I, [2018-09-17T08:20:11.319422 #24762:a51108] INFO -- : >--- >:kill_algorithm: > :name: :used_swap_percent_gt_value > :value: 80 >:miq_server_time_threshold: 120 >:nice_delta: 1 >:poll: 2 >:start_algorithm: > :name: :used_swap_percent_lt_value > :value: 60 >:sync_interval: 1800 >:wait_for_started_timeout: 600 >[----] I, [2018-09-17T08:20:11.496852 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.close_pg_sockets_inherited_from_parent) Closing socket: 25 >[----] I, [2018-09-17T08:20:11.613751 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:11.614090 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:11.614322 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:11.614450 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:11.614840 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:11.631900 #24762:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12110], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}] >[----] I, [2018-09-17T08:20:11.632017 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#start) Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41] >[----] I, [2018-09-17T08:20:11.854171 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#sync_config) ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41], Zone [default], Active Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Assigned Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Configuration: >[----] I, [2018-09-17T08:20:11.854914 #11110:a51108] INFO -- : >--- >:count: 1 >:gc_interval: 900 >:heartbeat_freq: 10 >:heartbeat_timeout: 120 >:memory_threshold: 2147483648 >:nice_delta: 1 >:parent_time_threshold: 180 >:poll: 10 >:poll_escalate_max: 30 >:poll_method: :normal >:restart_interval: 0 >:starting_timeout: 600 >:stopping_timeout: 600 >:flooding_events_per_minute: 30 >:flooding_monitor_enabled: false >:ems_event_page_size: 100 >:ems_event_thread_shutdown_timeout: 10 >[----] I, [2018-09-17T08:20:11.854965 #11110:a51108] INFO -- : --- >[----] I, [2018-09-17T08:20:11.855330 #11110:a51108] INFO -- : >--- >:guid: 05ed60e8-ba74-11e8-86bf-525400153f41 >:ems_id: 3 >[----] I, [2018-09-17T08:20:11.865886 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:20:12.651963 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:12.662442 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0031249523162841797, :worker_monitor=>1.6190671920776367, :worker_dequeue=>0.012407541275024414, :total_time=>1.6349527835845947} >[----] I, [2018-09-17T08:20:12.862891 #11110:a51108] INFO -- : MIQ(AuthUseridPassword#validation_successful) [ExtManagementSystem] [5], previously valid/invalid on: [2018-09-17 12:19:39 UTC]/[], previous status: [Valid] >[----] I, [2018-09-17T08:20:12.877889 #11110:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12111], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}] >[----] I, [2018-09-17T08:20:12.883368 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#after_initialize) EMS [10.0.0.101] as [admin] Event Catcher skipping the following events: > >NOTICE: identifier "MIQ|11110|1|66|1|StorageManager::CinderManager::EventCatcher|..." will be truncated to "MIQ|11110|1|66|1|StorageManager::CinderManager::EventCatcher|.." >[----] I, [2018-09-17T08:20:13.267407 #11110:a51108] INFO -- : ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner started. ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41], Zone [default], Role [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket] >[----] I, [2018-09-17T08:20:13.267499 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Checking that worker monitor has started before doing work >[----] I, [2018-09-17T08:20:13.269999 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Starting work since worker monitor has started >[----] I, [2018-09-17T08:20:13.270385 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:13.270674 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:20:13.270909 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:13.271587 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:13.273106 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#worker_monitor_drb) EMS [10.0.0.101] as [admin] Initializing DRb Connection to MiqServer with ID=[1], NAME=[EVM], PID=[24762], GUID=[b855343c-ba6e-11e8-85b9-525400153f41] DRb URI=[drbunix:///tmp/worker_monitor20180917-24762-fp1fxm] >[----] I, [2018-09-17T08:20:13.275842 #11110:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_work) EMS [10.0.0.101] as [admin] Waiting for the Monitor Thread to exit... >[----] I, [2018-09-17T08:20:13.522187 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12110], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}], Dequeued in: [1.89694218] seconds >[----] I, [2018-09-17T08:20:13.522330 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12110], Delivering... >[----] I, [2018-09-17T08:20:13.528065 #27806:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"evm_worker_start"> inputs=<{:event_details=>"Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5847, :miq_event_id=>5847, "EventStream::event_stream"=>5847, :event_stream_id=>5847}> >[----] I, [2018-09-17T08:20:13.539393 #27806:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12112], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5847, :miq_event_id=>5847, "EventStream::event_stream"=>5847, :event_stream_id=>5847}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:20:13.539593 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12110], State: [ok], Delivered in [0.017261479] seconds >/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:121:in `monitor_events': must be implemented in subclass (NotImplementedError) > from /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:163:in `block in start_event_monitor' >[----] I, [2018-09-17T08:20:15.931832 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:17.005966 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:17.667596 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:17.839353 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12112], MiqWorker id: [4], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5847, :miq_event_id=>5847, "EventStream::event_stream"=>5847, :event_stream_id=>5847}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [4.306161936] seconds >[----] I, [2018-09-17T08:20:17.839462 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12112], Delivering... >[----] I, [2018-09-17T08:20:17.842119 #27824:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [2] name [EvmGroup-super_administrator] >[----] I, [2018-09-17T08:20:17.842446 #27824:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [66], PID [11110], GUID [05ed60e8-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5847, :miq_event_id=>5847, "EventStream::event_stream"=>5847, :event_stream_id=>5847} for object [MiqServer.1] with state [] to Automate >[----] I, [2018-09-17T08:20:17.844593 #27824:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5847&MiqEvent%3A%3Amiq_event=5847&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B66%5D%2C%20PID%20%5B11110%5D%2C%20GUID%20%5B05ed60e8-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5847&event_type=evm_worker_start&miq_event_id=5847&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server] >[----] I, [2018-09-17T08:20:17.869360 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5847&MiqEvent%3A%3Amiq_event=5847&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B66%5D%2C%20PID%20%5B11110%5D%2C%20GUID%20%5B05ed60e8-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5847&event_type=evm_worker_start&miq_event_id=5847&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server ManageIQ/System] >[----] I, [2018-09-17T08:20:17.919218 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:20:17.942845 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:20:17.955447 #27824:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/evm_worker_start] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:20:17.970743 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:17.975377 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:20:17.985467 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:20:17.990998 #27824:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:20:17.993847 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<MiqServer id: 1, guid: "b855343c-ba6e-11e8-85b9-525400153f41", status: "started", started_on: "2018-09-17 11:44:11", stopped_on: nil, pid: 24762, build: "20180911143221_8357814", percent_memory: 4.06, percent_cpu: 3.85, cpu_time: 28770.0, name: "EVM", capabilities: {:vixDisk=>true, :concurrent_miqproxies=>0}, last_heartbeat: "2018-09-17 12:20:00", os_priority: 20, is_master: true, logo: nil, version: "5.8.5.1", zone_id: 1, upgrade_status: nil, upgrade_message: nil, memory_usage: 375971840, memory_size: 767401984, hostname: nil, ipaddress: "10.0.0.30", drb_uri: "drbunix:///tmp/worker_monitor20180917-24762-fp1fxm", mac_address: "52:54:00:15:3f:41", vm_id: nil, has_active_userinterface: true, has_active_webservices: true, sql_spid: 27687, rh_registered: nil, rh_subscribed: nil, last_update_check: nil, updates_available: nil, rhn_mirror: false, log_file_depot_id: nil, proportional_set_size: 229364000, has_active_websocket: true, system_memory_free: 1732235264, system_memory_used: 7526739968, system_swap_free: 9122607104, system_swap_used: 0>] >[----] I, [2018-09-17T08:20:17.994065 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [evm_worker_start] >[----] I, [2018-09-17T08:20:18.001185 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [evm_worker_start] >[----] I, [2018-09-17T08:20:18.001378 #27824:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [evm_worker_start] Target: MiqServer Name: [EVM], Id: [1] >[----] I, [2018-09-17T08:20:18.001688 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:18.002210 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:20:18.002913 #27824:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12112], State: [ok], Delivered in [0.163440522] seconds >[----] I, [2018-09-17T08:20:18.811152 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:20:18.818072 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12113], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:20:19.527271 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12111], MiqWorker id: [1], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}], Dequeued in: [6.653777958] seconds >[----] I, [2018-09-17T08:20:19.527418 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12111], Delivering... >[----] I, [2018-09-17T08:20:19.533561 #27798:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"ems_auth_valid"> inputs=<{"MiqEvent::miq_event"=>5848, :miq_event_id=>5848, "EventStream::event_stream"=>5848, :event_stream_id=>5848}> >[----] I, [2018-09-17T08:20:19.546577 #27798:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12114], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5848, :miq_event_id=>5848, "EventStream::event_stream"=>5848, :event_stream_id=>5848}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:20:19.547624 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12111], State: [ok], Delivered in [0.020202994] seconds >[----] I, [2018-09-17T08:20:20.192462 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:20:20.206628 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12115], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:20:22.682609 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:22.684119 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:20:22.863472 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12113], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [4.048088793] seconds >[----] I, [2018-09-17T08:20:22.863588 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12113], Delivering... >[----] I, [2018-09-17T08:20:22.867492 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0019044876098632812, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0019214153289794922, :pending_vm_jobs=>0.0005099773406982422, :vm_jobs_to_dispatch_count=>0, :total_time=>0.0036764144897460938} >[----] I, [2018-09-17T08:20:22.867655 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12113], State: [ok], Delivered in [0.004067547] seconds >[----] I, [2018-09-17T08:20:22.877436 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12114], MiqWorker id: [3], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5848, :miq_event_id=>5848, "EventStream::event_stream"=>5848, :event_stream_id=>5848}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [3.336846317] seconds >[----] I, [2018-09-17T08:20:22.877516 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12114], Delivering... >[----] I, [2018-09-17T08:20:22.881069 #27816:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [1] name [Tenant My Company access] >[----] I, [2018-09-17T08:20:22.881235 #27816:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5848, :miq_event_id=>5848, "EventStream::event_stream"=>5848, :event_stream_id=>5848} for object [ManageIQ::Providers::Openstack::CloudManager.5] with state [] to Automate >[----] I, [2018-09-17T08:20:22.883426 #27816:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5848&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5848&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5848&event_type=ems_auth_valid&miq_event_id=5848&object_name=Event&vmdb_object_type=ext_management_system] >[----] I, [2018-09-17T08:20:22.905944 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5848&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5848&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5848&event_type=ems_auth_valid&miq_event_id=5848&object_name=Event&vmdb_object_type=ext_management_system ManageIQ/System] >[----] I, [2018-09-17T08:20:22.958020 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:20:22.982573 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:20:22.994743 #27816:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/ems_auth_valid] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:20:23.007640 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:23.012226 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:20:23.022580 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:20:23.028001 #27816:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:20:23.030662 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<ManageIQ::Providers::Openstack::CloudManager id: 5, name: "oc_ci", created_on: "2018-09-17 11:55:21", updated_on: "2018-09-17 12:11:32", guid: "8e1cb774-ba70-11e8-86bf-525400153f41", zone_id: 1, type: "ManageIQ::Providers::Openstack::CloudManager", api_version: "v3", uid_ems: "default", host_default_vnc_port_start: nil, host_default_vnc_port_end: nil, provider_region: "", last_refresh_error: nil, last_refresh_date: "2018-09-17 12:11:32", provider_id: 1, realm: nil, tenant_id: 1, project: nil, parent_ems_id: nil, subscription: nil, last_metrics_error: nil, last_metrics_update_date: nil, last_metrics_success_date: nil, tenant_mapping_enabled: false>] >[----] I, [2018-09-17T08:20:23.030829 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [ems_auth_valid] >[----] I, [2018-09-17T08:20:23.037869 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [ems_auth_valid] >[----] I, [2018-09-17T08:20:23.037983 #27816:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [ems_auth_valid] Target: ManageIQ::Providers::Openstack::CloudManager Name: [oc_ci], Id: [5] >[----] I, [2018-09-17T08:20:23.038296 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:23.038942 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:20:23.039549 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12114], State: [ok], Delivered in [0.162015424] seconds >[----] I, [2018-09-17T08:20:25.060515 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12115], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [4.861570033] seconds >[----] I, [2018-09-17T08:20:25.060639 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12115], Delivering... >[----] I, [2018-09-17T08:20:25.063060 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12115], State: [ok], Delivered in [0.002415419] seconds >[----] I, [2018-09-17T08:20:25.693288 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:25.693763 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:25.694056 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:25.694190 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:25.694644 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:26.616899 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:26.617174 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:26.617403 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:26.617567 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:26.617822 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] W, [2018-09-17T08:20:28.631704 #24762:a51108] WARN -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#status_update) No such process [Event Monitor for Providers: oc_ci Cinder Manager] with PID=[11110], aborting worker. >[----] I, [2018-09-17T08:20:28.764985 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.002798795700073242, :worker_monitor=>1.0541036128997803, :worker_dequeue=>0.013962268829345703, :total_time=>1.071152687072754} >[----] I, [2018-09-17T08:20:31.590161 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:32.669686 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:33.765446 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:20:33 UTC]... >[----] I, [2018-09-17T08:20:33.772770 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:20:33 UTC]...Complete >[----] I, [2018-09-17T08:20:33.826912 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 4. >[----] I, [2018-09-17T08:20:33.835011 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12116], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:20:33.840821 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12117], id: [], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:20:33.846483 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12118], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:20:35.209413 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:20:35.222578 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12119], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:20:38.794604 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 3 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:38.796015 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:20:39.088834 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12116], MiqWorker id: [4], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [5.2574466] seconds >[----] I, [2018-09-17T08:20:39.088984 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12116], Delivering... >[----] I, [2018-09-17T08:20:39.093450 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.002293109893798828, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0023131370544433594, :pending_vm_jobs=>0.0005941390991210938, :vm_jobs_to_dispatch_count=>0, :total_time=>0.0041234493255615234} >[----] I, [2018-09-17T08:20:39.093707 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12116], State: [ok], Delivered in [0.004733656] seconds >[----] I, [2018-09-17T08:20:39.106304 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12117], MiqWorker id: [4], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [5.267781352] seconds >[----] I, [2018-09-17T08:20:39.106410 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12117], Delivering... >[----] I, [2018-09-17T08:20:39.157784 #27824:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12117], State: [ok], Delivered in [0.051358991] seconds >[----] I, [2018-09-17T08:20:40.092051 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12119], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [4.876348093] seconds >[----] I, [2018-09-17T08:20:40.092164 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12119], Delivering... >[----] I, [2018-09-17T08:20:40.094546 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12119], State: [ok], Delivered in [0.002357213] seconds >[----] I, [2018-09-17T08:20:40.595999 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12118], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [6.752016037] seconds >[----] I, [2018-09-17T08:20:40.596133 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12118], Delivering... >[----] I, [2018-09-17T08:20:40.598070 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12118], State: [ok], Delivered in [0.001951161] seconds >[----] I, [2018-09-17T08:20:40.697356 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:40.697759 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:40.698029 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:40.698190 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:40.698514 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:41.621392 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:41.621804 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:41.622098 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:41.622276 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:41.622727 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:43.998074 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.sync_workers) Workers are being synchronized: Current: [], Desired: ["ems_3"] >[----] I, [2018-09-17T08:20:44.072291 #24762:a51108] INFO -- : >--- >:kill_algorithm: > :name: :used_swap_percent_gt_value > :value: 80 >:miq_server_time_threshold: 120 >:nice_delta: 1 >:poll: 2 >:start_algorithm: > :name: :used_swap_percent_lt_value > :value: 60 >:sync_interval: 1800 >:wait_for_started_timeout: 600 >[----] I, [2018-09-17T08:20:44.267914 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.close_pg_sockets_inherited_from_parent) Closing socket: 25 >[----] I, [2018-09-17T08:20:44.333196 #24762:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12120], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}] >[----] I, [2018-09-17T08:20:44.333354 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#start) Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41] >[----] I, [2018-09-17T08:20:44.569504 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#sync_config) ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41], Zone [default], Active Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Assigned Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Configuration: >[----] I, [2018-09-17T08:20:44.570260 #11181:a51108] INFO -- : >--- >:count: 1 >:gc_interval: 900 >:heartbeat_freq: 10 >:heartbeat_timeout: 120 >:memory_threshold: 2147483648 >:nice_delta: 1 >:parent_time_threshold: 180 >:poll: 10 >:poll_escalate_max: 30 >:poll_method: :normal >:restart_interval: 0 >:starting_timeout: 600 >:stopping_timeout: 600 >:flooding_events_per_minute: 30 >:flooding_monitor_enabled: false >:ems_event_page_size: 100 >:ems_event_thread_shutdown_timeout: 10 >[----] I, [2018-09-17T08:20:44.570323 #11181:a51108] INFO -- : --- >[----] I, [2018-09-17T08:20:44.570685 #11181:a51108] INFO -- : >--- >:guid: 19730f28-ba74-11e8-86bf-525400153f41 >:ems_id: 3 >[----] I, [2018-09-17T08:20:44.581520 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:20:45.350121 #24762:a51108] INFO -- : MIQ(MiqServer#clean_worker_records) SQL Record for Worker [ManageIQ::Providers::StorageManager::CinderManager::EventCatcher] with ID: [66], PID: [11110], GUID: [05ed60e8-ba74-11e8-86bf-525400153f41], Status: [aborted] is being deleted >[----] I, [2018-09-17T08:20:45.460342 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:45.471101 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0032999515533447266, :worker_monitor=>1.649376392364502, :worker_dequeue=>0.012813091278076172, :total_time=>1.6657829284667969} >[----] I, [2018-09-17T08:20:45.611885 #11181:a51108] INFO -- : MIQ(AuthUseridPassword#validation_successful) [ExtManagementSystem] [5], previously valid/invalid on: [2018-09-17 12:20:12 UTC]/[], previous status: [Valid] >[----] I, [2018-09-17T08:20:45.630675 #11181:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12121], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}] >[----] I, [2018-09-17T08:20:45.637225 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#after_initialize) EMS [10.0.0.101] as [admin] Event Catcher skipping the following events: > >NOTICE: identifier "MIQ|11181|1|67|1|StorageManager::CinderManager::EventCatcher|..." will be truncated to "MIQ|11181|1|67|1|StorageManager::CinderManager::EventCatcher|.." >[----] I, [2018-09-17T08:20:46.183437 #11181:a51108] INFO -- : ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner started. ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41], Zone [default], Role [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket] >[----] I, [2018-09-17T08:20:46.183560 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Checking that worker monitor has started before doing work >[----] I, [2018-09-17T08:20:46.186074 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Starting work since worker monitor has started >[----] I, [2018-09-17T08:20:46.186473 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:46.186806 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:20:46.187118 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:46.188022 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:46.189570 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#worker_monitor_drb) EMS [10.0.0.101] as [admin] Initializing DRb Connection to MiqServer with ID=[1], NAME=[EVM], PID=[24762], GUID=[b855343c-ba6e-11e8-85b9-525400153f41] DRb URI=[drbunix:///tmp/worker_monitor20180917-24762-fp1fxm] >[----] I, [2018-09-17T08:20:46.192785 #11181:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_work) EMS [10.0.0.101] as [admin] Waiting for the Monitor Thread to exit... >[----] I, [2018-09-17T08:20:46.614561 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12120], MiqWorker id: [1], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}], Dequeued in: [2.289263947] seconds >[----] I, [2018-09-17T08:20:46.614729 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12120], Delivering... >[----] I, [2018-09-17T08:20:46.622887 #27798:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"evm_worker_start"> inputs=<{:event_details=>"Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5849, :miq_event_id=>5849, "EventStream::event_stream"=>5849, :event_stream_id=>5849}> >[----] I, [2018-09-17T08:20:46.635818 #27798:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12122], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5849, :miq_event_id=>5849, "EventStream::event_stream"=>5849, :event_stream_id=>5849}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:20:46.636050 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12120], State: [ok], Delivered in [0.021311065] seconds >[----] I, [2018-09-17T08:20:46.908589 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:121:in `monitor_events': must be implemented in subclass (NotImplementedError) > from /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:163:in `block in start_event_monitor' >[----] I, [2018-09-17T08:20:48.289477 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:20:48.858837 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:20:48.868665 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12123], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:20:50.225224 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:20:50.238583 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12124], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:20:50.531934 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 3 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:50.533607 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:20:51.287264 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12122], MiqWorker id: [4], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5849, :miq_event_id=>5849, "EventStream::event_stream"=>5849, :event_stream_id=>5849}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [4.658411863] seconds >[----] I, [2018-09-17T08:20:51.287262 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12123], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [2.422907646] seconds >[----] I, [2018-09-17T08:20:51.287410 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12123], Delivering... >[----] I, [2018-09-17T08:20:51.287445 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12122], Delivering... >[----] I, [2018-09-17T08:20:51.290868 #27824:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [2] name [EvmGroup-super_administrator] >[----] I, [2018-09-17T08:20:51.291031 #27824:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [67], PID [11181], GUID [19730f28-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5849, :miq_event_id=>5849, "EventStream::event_stream"=>5849, :event_stream_id=>5849} for object [MiqServer.1] with state [] to Automate >[----] I, [2018-09-17T08:20:51.292037 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0023183822631835938, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0023365020751953125, :pending_vm_jobs=>0.0005056858062744141, :vm_jobs_to_dispatch_count=>0, :total_time=>0.004346609115600586} >[----] I, [2018-09-17T08:20:51.292264 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12123], State: [ok], Delivered in [0.004853774] seconds >[----] I, [2018-09-17T08:20:51.293562 #27824:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5849&MiqEvent%3A%3Amiq_event=5849&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B67%5D%2C%20PID%20%5B11181%5D%2C%20GUID%20%5B19730f28-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5849&event_type=evm_worker_start&miq_event_id=5849&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server] >[----] I, [2018-09-17T08:20:51.319308 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5849&MiqEvent%3A%3Amiq_event=5849&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B67%5D%2C%20PID%20%5B11181%5D%2C%20GUID%20%5B19730f28-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5849&event_type=evm_worker_start&miq_event_id=5849&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server ManageIQ/System] >[----] I, [2018-09-17T08:20:51.372976 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:20:51.398592 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:20:51.411789 #27824:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/evm_worker_start] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:20:51.426849 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:51.431741 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:20:51.443392 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:20:51.449753 #27824:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:20:51.453416 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<MiqServer id: 1, guid: "b855343c-ba6e-11e8-85b9-525400153f41", status: "started", started_on: "2018-09-17 11:44:11", stopped_on: nil, pid: 24762, build: "20180911143221_8357814", percent_memory: 4.08, percent_cpu: 3.87, cpu_time: 29206.0, name: "EVM", capabilities: {:vixDisk=>true, :concurrent_miqproxies=>0}, last_heartbeat: "2018-09-17 12:20:33", os_priority: 20, is_master: true, logo: nil, version: "5.8.5.1", zone_id: 1, upgrade_status: nil, upgrade_message: nil, memory_usage: 378208256, memory_size: 769884160, hostname: nil, ipaddress: "10.0.0.30", drb_uri: "drbunix:///tmp/worker_monitor20180917-24762-fp1fxm", mac_address: "52:54:00:15:3f:41", vm_id: nil, has_active_userinterface: true, has_active_webservices: true, sql_spid: 27687, rh_registered: nil, rh_subscribed: nil, last_update_check: nil, updates_available: nil, rhn_mirror: false, log_file_depot_id: nil, proportional_set_size: 252748000, has_active_websocket: true, system_memory_free: 1965359104, system_memory_used: 7293616128, system_swap_free: 9122607104, system_swap_used: 0>] >[----] I, [2018-09-17T08:20:51.453733 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [evm_worker_start] >[----] I, [2018-09-17T08:20:51.461969 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [evm_worker_start] >[----] I, [2018-09-17T08:20:51.462084 #27824:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [evm_worker_start] Target: MiqServer Name: [EVM], Id: [1] >[----] I, [2018-09-17T08:20:51.462458 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:51.463463 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:20:51.464344 #27824:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12122], State: [ok], Delivered in [0.17690525] seconds >[----] I, [2018-09-17T08:20:52.119898 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12124], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [1.888067906] seconds >[----] I, [2018-09-17T08:20:52.119995 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12124], Delivering... >[----] I, [2018-09-17T08:20:52.122102 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12124], State: [ok], Delivered in [0.002105204] seconds >[----] I, [2018-09-17T08:20:52.626299 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12121], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}], Dequeued in: [7.001240642] seconds >[----] I, [2018-09-17T08:20:52.626422 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12121], Delivering... >[----] I, [2018-09-17T08:20:52.633600 #27806:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"ems_auth_valid"> inputs=<{"MiqEvent::miq_event"=>5850, :miq_event_id=>5850, "EventStream::event_stream"=>5850, :event_stream_id=>5850}> >[----] I, [2018-09-17T08:20:52.649586 #27806:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12125], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5850, :miq_event_id=>5850, "EventStream::event_stream"=>5850, :event_stream_id=>5850}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:20:52.649787 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12121], State: [ok], Delivered in [0.023366663] seconds >[----] I, [2018-09-17T08:20:55.550313 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:20:55.659045 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12125], MiqWorker id: [1], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5850, :miq_event_id=>5850, "EventStream::event_stream"=>5850, :event_stream_id=>5850}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [3.017111371] seconds >[----] I, [2018-09-17T08:20:55.659176 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12125], Delivering... >[----] I, [2018-09-17T08:20:55.662764 #27798:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [1] name [Tenant My Company access] >[----] I, [2018-09-17T08:20:55.662963 #27798:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5850, :miq_event_id=>5850, "EventStream::event_stream"=>5850, :event_stream_id=>5850} for object [ManageIQ::Providers::Openstack::CloudManager.5] with state [] to Automate >[----] I, [2018-09-17T08:20:55.665603 #27798:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5850&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5850&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5850&event_type=ems_auth_valid&miq_event_id=5850&object_name=Event&vmdb_object_type=ext_management_system] >[----] I, [2018-09-17T08:20:55.690805 #27798:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5850&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5850&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5850&event_type=ems_auth_valid&miq_event_id=5850&object_name=Event&vmdb_object_type=ext_management_system ManageIQ/System] >[----] I, [2018-09-17T08:20:55.700875 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:55.701221 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:55.701476 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:55.701621 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:55.703754 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:20:55.752755 #27798:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:20:55.780580 #27798:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:20:55.793618 #27798:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/ems_auth_valid] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:20:55.807833 #27798:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:55.812879 #27798:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:20:55.823770 #27798:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:20:55.829802 #27798:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:20:55.832852 #27798:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<ManageIQ::Providers::Openstack::CloudManager id: 5, name: "oc_ci", created_on: "2018-09-17 11:55:21", updated_on: "2018-09-17 12:11:32", guid: "8e1cb774-ba70-11e8-86bf-525400153f41", zone_id: 1, type: "ManageIQ::Providers::Openstack::CloudManager", api_version: "v3", uid_ems: "default", host_default_vnc_port_start: nil, host_default_vnc_port_end: nil, provider_region: "", last_refresh_error: nil, last_refresh_date: "2018-09-17 12:11:32", provider_id: 1, realm: nil, tenant_id: 1, project: nil, parent_ems_id: nil, subscription: nil, last_metrics_error: nil, last_metrics_update_date: nil, last_metrics_success_date: nil, tenant_mapping_enabled: false>] >[----] I, [2018-09-17T08:20:55.833039 #27798:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [ems_auth_valid] >[----] I, [2018-09-17T08:20:55.840357 #27798:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [ems_auth_valid] >[----] I, [2018-09-17T08:20:55.840549 #27798:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [ems_auth_valid] Target: ManageIQ::Providers::Openstack::CloudManager Name: [oc_ci], Id: [5] >[----] I, [2018-09-17T08:20:55.846480 #27798:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:20:55.847000 #27798:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:20:55.847609 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12125], State: [ok], Delivered in [0.188440274] seconds >[----] I, [2018-09-17T08:20:56.625000 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:20:56.625294 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:20:56.625598 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:20:56.625744 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:20:56.626079 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] W, [2018-09-17T08:21:01.481599 #24762:a51108] WARN -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#status_update) No such process [Event Monitor for Providers: oc_ci Cinder Manager] with PID=[11181], aborting worker. >[----] I, [2018-09-17T08:21:01.583929 #24762:a51108] INFO -- : MIQ(MiqServer#clean_worker_records) SQL Record for Worker [ManageIQ::Providers::StorageManager::CinderManager::EventCatcher] with ID: [67], PID: [11181], GUID: [19730f28-ba74-11e8-86bf-525400153f41], Status: [aborted] is being deleted >[----] I, [2018-09-17T08:21:01.692594 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0027954578399658203, :worker_monitor=>1.1164729595184326, :worker_dequeue=>0.011707305908203125, :total_time=>1.1312618255615234} >[----] I, [2018-09-17T08:21:02.573972 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:03.593469 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:03.881251 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:21:03.889487 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12126], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:21:05.241263 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:21:05.254806 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12127], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:21:06.693125 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:21:06 UTC]... >[----] I, [2018-09-17T08:21:06.701747 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:21:06 UTC]...Complete >[----] I, [2018-09-17T08:21:06.708937 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:06.710857 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:21:07.149833 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12127], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [1.90192322] seconds >[----] I, [2018-09-17T08:21:07.149969 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12127], Delivering... >[----] I, [2018-09-17T08:21:07.152195 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12127], State: [ok], Delivered in [0.002249041] seconds >[----] I, [2018-09-17T08:21:07.348394 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12126], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [3.461897404] seconds >[----] I, [2018-09-17T08:21:07.348502 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12126], Delivering... >[----] I, [2018-09-17T08:21:07.352725 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0020639896392822266, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0021181106567382812, :pending_vm_jobs=>0.0004932880401611328, :vm_jobs_to_dispatch_count=>0, :total_time=>0.00393366813659668} >[----] I, [2018-09-17T08:21:07.353012 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12126], State: [ok], Delivered in [0.004507077] seconds >[----] I, [2018-09-17T08:21:10.706006 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:10.706448 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:10.706718 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:10.706871 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:10.707218 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:11.629350 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:11.629795 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:11.630046 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:11.630255 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:11.630578 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:16.940087 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.sync_workers) Workers are being synchronized: Current: [], Desired: ["ems_3"] >[----] I, [2018-09-17T08:21:17.014811 #24762:a51108] INFO -- : >--- >:kill_algorithm: > :name: :used_swap_percent_gt_value > :value: 80 >:miq_server_time_threshold: 120 >:nice_delta: 1 >:poll: 2 >:start_algorithm: > :name: :used_swap_percent_lt_value > :value: 60 >:sync_interval: 1800 >:wait_for_started_timeout: 600 >[----] I, [2018-09-17T08:21:17.260150 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.close_pg_sockets_inherited_from_parent) Closing socket: 25 >[----] I, [2018-09-17T08:21:17.333026 #24762:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12128], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}] >[----] I, [2018-09-17T08:21:17.333182 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#start) Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41] >[----] I, [2018-09-17T08:21:17.533906 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#sync_config) ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41], Zone [default], Active Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Assigned Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Configuration: >[----] I, [2018-09-17T08:21:17.534619 #11259:a51108] INFO -- : >--- >:count: 1 >:gc_interval: 900 >:heartbeat_freq: 10 >:heartbeat_timeout: 120 >:memory_threshold: 2147483648 >:nice_delta: 1 >:parent_time_threshold: 180 >:poll: 10 >:poll_escalate_max: 30 >:poll_method: :normal >:restart_interval: 0 >:starting_timeout: 600 >:stopping_timeout: 600 >:flooding_events_per_minute: 30 >:flooding_monitor_enabled: false >:ems_event_page_size: 100 >:ems_event_thread_shutdown_timeout: 10 >[----] I, [2018-09-17T08:21:17.534672 #11259:a51108] INFO -- : --- >[----] I, [2018-09-17T08:21:17.534974 #11259:a51108] INFO -- : >--- >:guid: 2d15b7b0-ba74-11e8-86bf-525400153f41 >:ems_id: 3 >[----] I, [2018-09-17T08:21:17.545671 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:21:17.917814 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:18.342681 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:18.352619 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.002766132354736328, :worker_monitor=>1.5975258350372314, :worker_dequeue=>0.011775016784667969, :total_time=>1.6123793125152588} >[----] I, [2018-09-17T08:21:18.583979 #11259:a51108] INFO -- : MIQ(AuthUseridPassword#validation_successful) [ExtManagementSystem] [5], previously valid/invalid on: [2018-09-17 12:20:45 UTC]/[], previous status: [Valid] >[----] I, [2018-09-17T08:21:18.598579 #11259:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12129], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}] >[----] I, [2018-09-17T08:21:18.604000 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#after_initialize) EMS [10.0.0.101] as [admin] Event Catcher skipping the following events: > >[----] I, [2018-09-17T08:21:18.901161 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:21:18.909470 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12130], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:21:18.923591 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >NOTICE: identifier "MIQ|11259|1|68|1|StorageManager::CinderManager::EventCatcher|..." will be truncated to "MIQ|11259|1|68|1|StorageManager::CinderManager::EventCatcher|.." >[----] I, [2018-09-17T08:21:18.986190 #11259:a51108] INFO -- : ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner started. ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41], Zone [default], Role [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket] >[----] I, [2018-09-17T08:21:18.986305 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Checking that worker monitor has started before doing work >[----] I, [2018-09-17T08:21:18.989299 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Starting work since worker monitor has started >[----] I, [2018-09-17T08:21:18.989846 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:18.990218 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:21:18.990600 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:18.991261 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:18.993373 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#worker_monitor_drb) EMS [10.0.0.101] as [admin] Initializing DRb Connection to MiqServer with ID=[1], NAME=[EVM], PID=[24762], GUID=[b855343c-ba6e-11e8-85b9-525400153f41] DRb URI=[drbunix:///tmp/worker_monitor20180917-24762-fp1fxm] >[----] I, [2018-09-17T08:21:18.996153 #11259:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_work) EMS [10.0.0.101] as [admin] Waiting for the Monitor Thread to exit... >[----] I, [2018-09-17T08:21:19.688112 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12128], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}], Dequeued in: [2.377066619] seconds >[----] I, [2018-09-17T08:21:19.688218 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12128], Delivering... >[----] I, [2018-09-17T08:21:19.694215 #27806:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"evm_worker_start"> inputs=<{:event_details=>"Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5851, :miq_event_id=>5851, "EventStream::event_stream"=>5851, :event_stream_id=>5851}> >[----] I, [2018-09-17T08:21:19.705726 #27806:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12131], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5851, :miq_event_id=>5851, "EventStream::event_stream"=>5851, :event_stream_id=>5851}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:21:19.705961 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12128], State: [ok], Delivered in [0.017740014] seconds >/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:121:in `monitor_events': must be implemented in subclass (NotImplementedError) > from /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:163:in `block in start_event_monitor' >[----] I, [2018-09-17T08:21:20.258426 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:21:20.278801 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12132], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:21:23.357442 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 3 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:23.358787 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:21:23.415961 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12130], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [4.509667914] seconds >[----] I, [2018-09-17T08:21:23.416067 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12130], Delivering... >[----] I, [2018-09-17T08:21:23.420116 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.002112865447998047, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0021288394927978516, :pending_vm_jobs=>0.0005295276641845703, :vm_jobs_to_dispatch_count=>0, :total_time=>0.003785848617553711} >[----] I, [2018-09-17T08:21:23.420272 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12130], State: [ok], Delivered in [0.004203687] seconds >[----] I, [2018-09-17T08:21:23.430552 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12131], MiqWorker id: [3], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5851, :miq_event_id=>5851, "EventStream::event_stream"=>5851, :event_stream_id=>5851}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [3.730884979] seconds >[----] I, [2018-09-17T08:21:23.430650 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12131], Delivering... >[----] I, [2018-09-17T08:21:23.433469 #27816:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [2] name [EvmGroup-super_administrator] >[----] I, [2018-09-17T08:21:23.433644 #27816:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [68], PID [11259], GUID [2d15b7b0-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5851, :miq_event_id=>5851, "EventStream::event_stream"=>5851, :event_stream_id=>5851} for object [MiqServer.1] with state [] to Automate >[----] I, [2018-09-17T08:21:23.436280 #27816:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5851&MiqEvent%3A%3Amiq_event=5851&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B68%5D%2C%20PID%20%5B11259%5D%2C%20GUID%20%5B2d15b7b0-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5851&event_type=evm_worker_start&miq_event_id=5851&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server] >[----] I, [2018-09-17T08:21:23.460622 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5851&MiqEvent%3A%3Amiq_event=5851&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B68%5D%2C%20PID%20%5B11259%5D%2C%20GUID%20%5B2d15b7b0-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5851&event_type=evm_worker_start&miq_event_id=5851&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server ManageIQ/System] >[----] I, [2018-09-17T08:21:23.519450 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:21:23.544827 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:21:23.556966 #27816:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/evm_worker_start] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:21:23.570735 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:21:23.575441 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:21:23.586440 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:21:23.592691 #27816:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:21:23.596309 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<MiqServer id: 1, guid: "b855343c-ba6e-11e8-85b9-525400153f41", status: "started", started_on: "2018-09-17 11:44:11", stopped_on: nil, pid: 24762, build: "20180911143221_8357814", percent_memory: 4.08, percent_cpu: 3.87, cpu_time: 29206.0, name: "EVM", capabilities: {:vixDisk=>true, :concurrent_miqproxies=>0}, last_heartbeat: "2018-09-17 12:21:06", os_priority: 20, is_master: true, logo: nil, version: "5.8.5.1", zone_id: 1, upgrade_status: nil, upgrade_message: nil, memory_usage: 378208256, memory_size: 769884160, hostname: nil, ipaddress: "10.0.0.30", drb_uri: "drbunix:///tmp/worker_monitor20180917-24762-fp1fxm", mac_address: "52:54:00:15:3f:41", vm_id: nil, has_active_userinterface: true, has_active_webservices: true, sql_spid: 27687, rh_registered: nil, rh_subscribed: nil, last_update_check: nil, updates_available: nil, rhn_mirror: false, log_file_depot_id: nil, proportional_set_size: 252748000, has_active_websocket: true, system_memory_free: 1965359104, system_memory_used: 7293616128, system_swap_free: 9122607104, system_swap_used: 0>] >[----] I, [2018-09-17T08:21:23.596469 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [evm_worker_start] >[----] I, [2018-09-17T08:21:23.603845 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [evm_worker_start] >[----] I, [2018-09-17T08:21:23.604002 #27816:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [evm_worker_start] Target: MiqServer Name: [EVM], Id: [1] >[----] I, [2018-09-17T08:21:23.604452 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:21:23.604998 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:21:23.606298 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12131], State: [ok], Delivered in [0.175647334] seconds >[----] I, [2018-09-17T08:21:25.181626 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12132], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [4.916136334] seconds >[----] I, [2018-09-17T08:21:25.181738 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12132], Delivering... >[----] I, [2018-09-17T08:21:25.183889 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12132], State: [ok], Delivered in [0.002152347] seconds >[----] I, [2018-09-17T08:21:25.709607 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:25.710017 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:25.710282 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:25.710425 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:25.710731 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:25.722437 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12129], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}], Dequeued in: [7.128379375] seconds >[----] I, [2018-09-17T08:21:25.722546 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12129], Delivering... >[----] I, [2018-09-17T08:21:25.728444 #27806:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"ems_auth_valid"> inputs=<{"MiqEvent::miq_event"=>5852, :miq_event_id=>5852, "EventStream::event_stream"=>5852, :event_stream_id=>5852}> >[----] I, [2018-09-17T08:21:25.741091 #27806:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12133], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5852, :miq_event_id=>5852, "EventStream::event_stream"=>5852, :event_stream_id=>5852}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:21:25.741327 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12129], State: [ok], Delivered in [0.018796692] seconds >[----] I, [2018-09-17T08:21:26.632597 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:26.632917 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:26.633140 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:26.633278 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:26.633622 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:28.372747 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:28.601077 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12133], MiqWorker id: [4], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5852, :miq_event_id=>5852, "EventStream::event_stream"=>5852, :event_stream_id=>5852}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [2.865533633] seconds >[----] I, [2018-09-17T08:21:28.601193 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12133], Delivering... >[----] I, [2018-09-17T08:21:28.604784 #27824:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [1] name [Tenant My Company access] >[----] I, [2018-09-17T08:21:28.604929 #27824:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5852, :miq_event_id=>5852, "EventStream::event_stream"=>5852, :event_stream_id=>5852} for object [ManageIQ::Providers::Openstack::CloudManager.5] with state [] to Automate >[----] I, [2018-09-17T08:21:28.607008 #27824:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5852&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5852&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5852&event_type=ems_auth_valid&miq_event_id=5852&object_name=Event&vmdb_object_type=ext_management_system] >[----] I, [2018-09-17T08:21:28.629397 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5852&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5852&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5852&event_type=ems_auth_valid&miq_event_id=5852&object_name=Event&vmdb_object_type=ext_management_system ManageIQ/System] >[----] I, [2018-09-17T08:21:28.679244 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:21:28.704616 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:21:28.716516 #27824:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/ems_auth_valid] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:21:28.729814 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:21:28.734113 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:21:28.743808 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:21:28.749897 #27824:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:21:28.754654 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<ManageIQ::Providers::Openstack::CloudManager id: 5, name: "oc_ci", created_on: "2018-09-17 11:55:21", updated_on: "2018-09-17 12:11:32", guid: "8e1cb774-ba70-11e8-86bf-525400153f41", zone_id: 1, type: "ManageIQ::Providers::Openstack::CloudManager", api_version: "v3", uid_ems: "default", host_default_vnc_port_start: nil, host_default_vnc_port_end: nil, provider_region: "", last_refresh_error: nil, last_refresh_date: "2018-09-17 12:11:32", provider_id: 1, realm: nil, tenant_id: 1, project: nil, parent_ems_id: nil, subscription: nil, last_metrics_error: nil, last_metrics_update_date: nil, last_metrics_success_date: nil, tenant_mapping_enabled: false>] >[----] I, [2018-09-17T08:21:28.754819 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [ems_auth_valid] >[----] I, [2018-09-17T08:21:28.762229 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [ems_auth_valid] >[----] I, [2018-09-17T08:21:28.762333 #27824:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [ems_auth_valid] Target: ManageIQ::Providers::Openstack::CloudManager Name: [oc_ci], Id: [5] >[----] I, [2018-09-17T08:21:28.762713 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:21:28.763239 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:21:28.763905 #27824:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12133], State: [ok], Delivered in [0.162703378] seconds >[----] I, [2018-09-17T08:21:33.298368 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:33.920949 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 4. >[----] I, [2018-09-17T08:21:33.927865 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12134], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:21:33.932333 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12135], id: [], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:21:33.938837 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12136], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [ready], Deliver On: [], Data: [], Args: [] >[----] W, [2018-09-17T08:21:34.079437 #24762:a51108] WARN -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#status_update) No such process [Event Monitor for Providers: oc_ci Cinder Manager] with PID=[11259], aborting worker. >[----] I, [2018-09-17T08:21:34.442309 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 3 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:34.452791 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0029599666595458984, :worker_monitor=>1.0528578758239746, :worker_dequeue=>0.01259613037109375, :total_time=>1.0687947273254395} >[----] I, [2018-09-17T08:21:34.605556 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:34.665749 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12134], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [0.7405022] seconds >[----] I, [2018-09-17T08:21:34.665883 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12134], Delivering... >[----] I, [2018-09-17T08:21:34.671279 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0027687549591064453, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.002793550491333008, :pending_vm_jobs=>0.0007202625274658203, :vm_jobs_to_dispatch_count=>0, :total_time=>0.005051851272583008} >[----] I, [2018-09-17T08:21:34.671531 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12134], State: [ok], Delivered in [0.005602608] seconds >[----] I, [2018-09-17T08:21:34.684004 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12135], MiqWorker id: [3], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [0.75387184] seconds >[----] I, [2018-09-17T08:21:34.684134 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12135], Delivering... >[----] I, [2018-09-17T08:21:34.746479 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12135], State: [ok], Delivered in [0.062333884] seconds >[----] I, [2018-09-17T08:21:34.780676 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12136], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [0.844860357] seconds >[----] I, [2018-09-17T08:21:34.780810 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12136], Delivering... >[----] I, [2018-09-17T08:21:34.782776 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12136], State: [ok], Delivered in [0.001982223] seconds >[----] I, [2018-09-17T08:21:35.281597 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:21:35.294268 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12137], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:21:39.453313 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:21:39 UTC]... >[----] I, [2018-09-17T08:21:39.461931 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:21:39 UTC]...Complete >[----] I, [2018-09-17T08:21:39.469101 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:21:40.211635 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12137], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [4.923929091] seconds >[----] I, [2018-09-17T08:21:40.211734 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12137], Delivering... >[----] I, [2018-09-17T08:21:40.213958 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12137], State: [ok], Delivered in [0.00222309] seconds >[----] I, [2018-09-17T08:21:40.714076 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:40.714544 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:40.714880 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:40.715060 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:40.715566 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:41.636813 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:41.637205 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:41.637493 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:41.637700 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:41.638155 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:48.622003 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:48.952519 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:21:48.959349 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12138], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:21:49.697741 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.sync_workers) Workers are being synchronized: Current: [], Desired: ["ems_3"] >[----] I, [2018-09-17T08:21:49.768021 #24762:a51108] INFO -- : >--- >:kill_algorithm: > :name: :used_swap_percent_gt_value > :value: 80 >:miq_server_time_threshold: 120 >:nice_delta: 1 >:poll: 2 >:start_algorithm: > :name: :used_swap_percent_lt_value > :value: 60 >:sync_interval: 1800 >:wait_for_started_timeout: 600 >[----] I, [2018-09-17T08:21:49.956620 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:21:49.981199 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.close_pg_sockets_inherited_from_parent) Closing socket: 25 >[----] I, [2018-09-17T08:21:50.129880 #24762:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12139], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}] >[----] I, [2018-09-17T08:21:50.130012 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#start) Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41] >[----] I, [2018-09-17T08:21:50.306789 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#sync_config) ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41], Zone [default], Active Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Assigned Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Configuration: >[----] I, [2018-09-17T08:21:50.307798 #11348:a51108] INFO -- : >--- >:count: 1 >:gc_interval: 900 >:heartbeat_freq: 10 >:heartbeat_timeout: 120 >:memory_threshold: 2147483648 >:nice_delta: 1 >:parent_time_threshold: 180 >:poll: 10 >:poll_escalate_max: 30 >:poll_method: :normal >:restart_interval: 0 >:starting_timeout: 600 >:stopping_timeout: 600 >:flooding_events_per_minute: 30 >:flooding_monitor_enabled: false >:ems_event_page_size: 100 >:ems_event_thread_shutdown_timeout: 10 >[----] I, [2018-09-17T08:21:50.307852 #11348:a51108] INFO -- : --- >[----] I, [2018-09-17T08:21:50.308164 #11348:a51108] INFO -- : >--- >:guid: 409b6e4c-ba74-11e8-86bf-525400153f41 >:ems_id: 3 >[----] I, [2018-09-17T08:21:50.321814 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:21:51.026690 #24762:a51108] INFO -- : MIQ(MiqServer#clean_worker_records) SQL Record for Worker [ManageIQ::Providers::StorageManager::CinderManager::EventCatcher] with ID: [68], PID: [11259], GUID: [2d15b7b0-ba74-11e8-86bf-525400153f41], Status: [aborted] is being deleted >[----] I, [2018-09-17T08:21:51.148836 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:51.161511 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0029442310333251953, :worker_monitor=>1.647451639175415, :worker_dequeue=>0.015611410140991211, :total_time=>1.6663243770599365} >[----] I, [2018-09-17T08:21:51.299454 #11348:a51108] INFO -- : MIQ(AuthUseridPassword#validation_successful) [ExtManagementSystem] [5], previously valid/invalid on: [2018-09-17 12:21:18 UTC]/[], previous status: [Valid] >[----] I, [2018-09-17T08:21:51.316058 #11348:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12140], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}] >[----] I, [2018-09-17T08:21:51.321889 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#after_initialize) EMS [10.0.0.101] as [admin] Event Catcher skipping the following events: > >NOTICE: identifier "MIQ|11348|1|69|1|StorageManager::CinderManager::EventCatcher|..." will be truncated to "MIQ|11348|1|69|1|StorageManager::CinderManager::EventCatcher|.." >[----] I, [2018-09-17T08:21:51.817021 #11348:a51108] INFO -- : ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner started. ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41], Zone [default], Role [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket] >[----] I, [2018-09-17T08:21:51.817125 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Checking that worker monitor has started before doing work >[----] I, [2018-09-17T08:21:51.819843 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Starting work since worker monitor has started >[----] I, [2018-09-17T08:21:51.820249 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:51.820550 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:21:51.820806 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:51.821590 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:51.823203 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#worker_monitor_drb) EMS [10.0.0.101] as [admin] Initializing DRb Connection to MiqServer with ID=[1], NAME=[EVM], PID=[24762], GUID=[b855343c-ba6e-11e8-85b9-525400153f41] DRb URI=[drbunix:///tmp/worker_monitor20180917-24762-fp1fxm] >[----] I, [2018-09-17T08:21:51.826810 #11348:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_work) EMS [10.0.0.101] as [admin] Waiting for the Monitor Thread to exit... >[----] I, [2018-09-17T08:21:52.121983 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12138], MiqWorker id: [4], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [3.165336089] seconds >[----] I, [2018-09-17T08:21:52.122132 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12138], Delivering... >[----] I, [2018-09-17T08:21:52.127780 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.002641439437866211, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.002662181854248047, :pending_vm_jobs=>0.0008881092071533203, :vm_jobs_to_dispatch_count=>0, :total_time=>0.004887104034423828} >[----] I, [2018-09-17T08:21:52.128084 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12138], State: [ok], Delivered in [0.005944101] seconds >/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:121:in `monitor_events': must be implemented in subclass (NotImplementedError) > from /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:163:in `block in start_event_monitor' >[----] I, [2018-09-17T08:21:53.108845 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12139], MiqWorker id: [2], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}], Dequeued in: [2.984989997] seconds >[----] I, [2018-09-17T08:21:53.108954 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12139], Delivering... >[----] I, [2018-09-17T08:21:53.116306 #27806:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"evm_worker_start"> inputs=<{:event_details=>"Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5853, :miq_event_id=>5853, "EventStream::event_stream"=>5853, :event_stream_id=>5853}> >[----] I, [2018-09-17T08:21:53.132420 #27806:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12141], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5853, :miq_event_id=>5853, "EventStream::event_stream"=>5853, :event_stream_id=>5853}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:21:53.132712 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12139], State: [ok], Delivered in [0.023750834] seconds >[----] I, [2018-09-17T08:21:55.717898 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:55.718285 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:55.718542 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:55.718679 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:55.719063 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:56.221963 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:21:56.640636 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:21:56.641030 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:21:56.641311 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:21:56.641489 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:21:56.641808 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:21:57.134753 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12141], MiqWorker id: [3], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5853, :miq_event_id=>5853, "EventStream::event_stream"=>5853, :event_stream_id=>5853}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [4.009782874] seconds >[----] I, [2018-09-17T08:21:57.134915 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12141], Delivering... >[----] I, [2018-09-17T08:21:57.138083 #27816:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [2] name [EvmGroup-super_administrator] >[----] I, [2018-09-17T08:21:57.138235 #27816:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [69], PID [11348], GUID [409b6e4c-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5853, :miq_event_id=>5853, "EventStream::event_stream"=>5853, :event_stream_id=>5853} for object [MiqServer.1] with state [] to Automate >[----] I, [2018-09-17T08:21:57.140793 #27816:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5853&MiqEvent%3A%3Amiq_event=5853&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B69%5D%2C%20PID%20%5B11348%5D%2C%20GUID%20%5B409b6e4c-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5853&event_type=evm_worker_start&miq_event_id=5853&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server] >[----] I, [2018-09-17T08:21:57.167729 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5853&MiqEvent%3A%3Amiq_event=5853&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B69%5D%2C%20PID%20%5B11348%5D%2C%20GUID%20%5B409b6e4c-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5853&event_type=evm_worker_start&miq_event_id=5853&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server ManageIQ/System] >[----] I, [2018-09-17T08:21:57.224457 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:21:57.250163 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:21:57.263795 #27816:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/evm_worker_start] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:21:57.279064 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:21:57.283945 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:21:57.296880 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:21:57.305741 #27816:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:21:57.309586 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<MiqServer id: 1, guid: "b855343c-ba6e-11e8-85b9-525400153f41", status: "started", started_on: "2018-09-17 11:44:11", stopped_on: nil, pid: 24762, build: "20180911143221_8357814", percent_memory: 4.09, percent_cpu: 3.92, cpu_time: 29781.0, name: "EVM", capabilities: {:vixDisk=>true, :concurrent_miqproxies=>0}, last_heartbeat: "2018-09-17 12:21:39", os_priority: 20, is_master: true, logo: nil, version: "5.8.5.1", zone_id: 1, upgrade_status: nil, upgrade_message: nil, memory_usage: 378322944, memory_size: 770105344, hostname: nil, ipaddress: "10.0.0.30", drb_uri: "drbunix:///tmp/worker_monitor20180917-24762-fp1fxm", mac_address: "52:54:00:15:3f:41", vm_id: nil, has_active_userinterface: true, has_active_webservices: true, sql_spid: 27687, rh_registered: nil, rh_subscribed: nil, last_update_check: nil, updates_available: nil, rhn_mirror: false, log_file_depot_id: nil, proportional_set_size: 252956000, has_active_websocket: true, system_memory_free: 1961947136, system_memory_used: 7297028096, system_swap_free: 9122607104, system_swap_used: 0>] >[----] I, [2018-09-17T08:21:57.309747 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [evm_worker_start] >[----] I, [2018-09-17T08:21:57.317409 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [evm_worker_start] >[----] I, [2018-09-17T08:21:57.317542 #27816:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [evm_worker_start] Target: MiqServer Name: [EVM], Id: [1] >[----] I, [2018-09-17T08:21:57.317986 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:21:57.318702 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:21:57.319347 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12141], State: [ok], Delivered in [0.184433519] seconds >[----] I, [2018-09-17T08:21:59.130377 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12140], MiqWorker id: [1], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}], Dequeued in: [7.819210096] seconds >[----] I, [2018-09-17T08:21:59.130556 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12140], Delivering... >[----] I, [2018-09-17T08:21:59.137666 #27798:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"ems_auth_valid"> inputs=<{"MiqEvent::miq_event"=>5854, :miq_event_id=>5854, "EventStream::event_stream"=>5854, :event_stream_id=>5854}> >[----] I, [2018-09-17T08:21:59.152047 #27798:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12142], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5854, :miq_event_id=>5854, "EventStream::event_stream"=>5854, :event_stream_id=>5854}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:21:59.152248 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12140], State: [ok], Delivered in [0.02170427] seconds >[----] I, [2018-09-17T08:22:01.240032 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:22:01.341154 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12142], MiqWorker id: [3], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5854, :miq_event_id=>5854, "EventStream::event_stream"=>5854, :event_stream_id=>5854}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [2.195616989] seconds >[----] I, [2018-09-17T08:22:01.341288 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12142], Delivering... >[----] I, [2018-09-17T08:22:01.344980 #27816:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [1] name [Tenant My Company access] >[----] I, [2018-09-17T08:22:01.345107 #27816:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5854, :miq_event_id=>5854, "EventStream::event_stream"=>5854, :event_stream_id=>5854} for object [ManageIQ::Providers::Openstack::CloudManager.5] with state [] to Automate >[----] I, [2018-09-17T08:22:01.347427 #27816:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5854&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5854&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5854&event_type=ems_auth_valid&miq_event_id=5854&object_name=Event&vmdb_object_type=ext_management_system] >[----] I, [2018-09-17T08:22:01.371871 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5854&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5854&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5854&event_type=ems_auth_valid&miq_event_id=5854&object_name=Event&vmdb_object_type=ext_management_system ManageIQ/System] >[----] I, [2018-09-17T08:22:01.429997 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:22:01.456410 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:22:01.470870 #27816:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/ems_auth_valid] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:22:01.485843 #27816:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:22:01.490798 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:22:01.502372 #27816:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:22:01.509489 #27816:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:22:01.512821 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<ManageIQ::Providers::Openstack::CloudManager id: 5, name: "oc_ci", created_on: "2018-09-17 11:55:21", updated_on: "2018-09-17 12:11:32", guid: "8e1cb774-ba70-11e8-86bf-525400153f41", zone_id: 1, type: "ManageIQ::Providers::Openstack::CloudManager", api_version: "v3", uid_ems: "default", host_default_vnc_port_start: nil, host_default_vnc_port_end: nil, provider_region: "", last_refresh_error: nil, last_refresh_date: "2018-09-17 12:11:32", provider_id: 1, realm: nil, tenant_id: 1, project: nil, parent_ems_id: nil, subscription: nil, last_metrics_error: nil, last_metrics_update_date: nil, last_metrics_success_date: nil, tenant_mapping_enabled: false>] >[----] I, [2018-09-17T08:22:01.513016 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [ems_auth_valid] >[----] I, [2018-09-17T08:22:01.521329 #27816:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [ems_auth_valid] >[----] I, [2018-09-17T08:22:01.521489 #27816:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [ems_auth_valid] Target: ManageIQ::Providers::Openstack::CloudManager Name: [oc_ci], Id: [5] >[----] I, [2018-09-17T08:22:01.521843 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:22:01.522502 #27816:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:22:01.523240 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12142], State: [ok], Delivered in [0.181922014] seconds >[----] I, [2018-09-17T08:22:03.968540 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:22:03.975682 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12143], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:22:04.298432 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:22:05.320736 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:22:05.336660 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12144], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:22:05.645560 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] W, [2018-09-17T08:22:07.221861 #24762:a51108] WARN -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#status_update) No such process [Event Monitor for Providers: oc_ci Cinder Manager] with PID=[11348], aborting worker. >[----] I, [2018-09-17T08:22:07.323444 #24762:a51108] INFO -- : MIQ(MiqServer#clean_worker_records) SQL Record for Worker [ManageIQ::Providers::StorageManager::CinderManager::EventCatcher] with ID: [69], PID: [11348], GUID: [409b6e4c-ba74-11e8-86bf-525400153f41], Status: [aborted] is being deleted >[----] I, [2018-09-17T08:22:07.422141 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:22:07.423734 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:22:07.433407 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0027468204498291016, :worker_monitor=>1.1653831005096436, :worker_dequeue=>0.013202190399169922, :total_time=>1.1815588474273682} >[----] I, [2018-09-17T08:22:07.553829 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12143], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [3.580731599] seconds >[----] I, [2018-09-17T08:22:07.553927 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12143], Delivering... >[----] I, [2018-09-17T08:22:07.598752 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0019545555114746094, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0019850730895996094, :pending_vm_jobs=>0.04057931900024414, :vm_jobs_to_dispatch_count=>0, :total_time=>0.044562339782714844} >[----] I, [2018-09-17T08:22:07.599064 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12143], State: [ok], Delivered in [0.045127534] seconds >[----] I, [2018-09-17T08:22:10.259645 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12144], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [4.9319609] seconds >[----] I, [2018-09-17T08:22:10.259758 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12144], Delivering... >[----] I, [2018-09-17T08:22:10.262180 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12144], State: [ok], Delivered in [0.002429511] seconds >[----] I, [2018-09-17T08:22:10.722376 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:22:10.722774 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:22:10.723069 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:22:10.723203 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:22:10.723478 #28776:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:22:11.644540 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:22:11.644839 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:22:11.645048 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:22:11.645192 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:22:11.645482 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:22:12.433887 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:22:12 UTC]... >[----] I, [2018-09-17T08:22:12.441594 #24762:a51108] INFO -- : MIQ(MiqServer#heartbeat) Heartbeat [2018-09-17 12:22:12 UTC]...Complete >[----] I, [2018-09-17T08:22:18.986587 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 1. >[----] I, [2018-09-17T08:22:18.993021 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12145], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:22:19.625619 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:22:20.340099 #28996:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::NetworkManager::EventCatcher::Runner#queue_event) EMS [10.0.0.101] as [admin] Caught event [router.delete.end] >[----] I, [2018-09-17T08:22:20.355369 #28996:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12146], id: [], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}] >[----] I, [2018-09-17T08:22:20.955341 #28996:105a1f8] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >[----] I, [2018-09-17T08:22:22.681352 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.sync_workers) Workers are being synchronized: Current: [], Desired: ["ems_3"] >[----] I, [2018-09-17T08:22:22.762445 #24762:a51108] INFO -- : >--- >:kill_algorithm: > :name: :used_swap_percent_gt_value > :value: 80 >:miq_server_time_threshold: 120 >:nice_delta: 1 >:poll: 2 >:start_algorithm: > :name: :used_swap_percent_lt_value > :value: 60 >:sync_interval: 1800 >:wait_for_started_timeout: 600 >[----] I, [2018-09-17T08:22:22.995968 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher.close_pg_sockets_inherited_from_parent) Closing socket: 25 >[----] I, [2018-09-17T08:22:23.081022 #24762:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12147], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}] >[----] I, [2018-09-17T08:22:23.081177 #24762:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher#start) Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41] >[----] I, [2018-09-17T08:22:23.307847 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#sync_config) ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41], Zone [default], Active Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Assigned Roles [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket], Configuration: >[----] I, [2018-09-17T08:22:23.308569 #11400:a51108] INFO -- : >--- >:count: 1 >:gc_interval: 900 >:heartbeat_freq: 10 >:heartbeat_timeout: 120 >:memory_threshold: 2147483648 >:nice_delta: 1 >:parent_time_threshold: 180 >:poll: 10 >:poll_escalate_max: 30 >:poll_method: :normal >:restart_interval: 0 >:starting_timeout: 600 >:stopping_timeout: 600 >:flooding_events_per_minute: 30 >:flooding_monitor_enabled: false >:ems_event_page_size: 100 >:ems_event_thread_shutdown_timeout: 10 >[----] I, [2018-09-17T08:22:23.308622 #11400:a51108] INFO -- : --- >[----] I, [2018-09-17T08:22:23.308898 #11400:a51108] INFO -- : >--- >:guid: 54460268-ba74-11e8-86bf-525400153f41 >:ems_id: 3 >[----] I, [2018-09-17T08:22:23.319280 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:22:24.099062 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:22:24.100482 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 1 miq_queue rows for queue_name=ems, wcount=1, priority=200 >[----] I, [2018-09-17T08:22:24.109500 #24762:a51108] INFO -- : MIQ(MiqServer#monitor_loop) Server Monitoring Complete - Timings: {:server_dequeue=>0.0028421878814697266, :worker_monitor=>1.6195969581604004, :worker_dequeue=>0.01256251335144043, :total_time=>1.635352373123169} >[----] I, [2018-09-17T08:22:24.260332 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12145], MiqWorker id: [4], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [5.269660439] seconds >[----] I, [2018-09-17T08:22:24.260443 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12145], Delivering... >[----] I, [2018-09-17T08:22:24.264969 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.0022139549255371094, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.0022521018981933594, :pending_vm_jobs=>0.0005166530609130859, :vm_jobs_to_dispatch_count=>0, :total_time=>0.004198312759399414} >[----] I, [2018-09-17T08:22:24.265276 #27824:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12145], State: [ok], Delivered in [0.004752701] seconds >[----] I, [2018-09-17T08:22:24.317428 #11400:a51108] INFO -- : MIQ(AuthUseridPassword#validation_successful) [ExtManagementSystem] [5], previously valid/invalid on: [2018-09-17 12:21:51 UTC]/[], previous status: [Valid] >[----] I, [2018-09-17T08:22:24.338211 #11400:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12148], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [ready], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}] >[----] I, [2018-09-17T08:22:24.345234 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#after_initialize) EMS [10.0.0.101] as [admin] Event Catcher skipping the following events: > >NOTICE: identifier "MIQ|11400|1|70|1|StorageManager::CinderManager::EventCatcher|..." will be truncated to "MIQ|11400|1|70|1|StorageManager::CinderManager::EventCatcher|.." >[----] I, [2018-09-17T08:22:24.854114 #11400:a51108] INFO -- : ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner started. ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41], Zone [default], Role [automate,database_operations,database_owner,ems_inventory,ems_operations,event,reporting,scheduler,smartstate,user_interface,web_services,websocket] >[----] I, [2018-09-17T08:22:24.854230 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Checking that worker monitor has started before doing work >[----] I, [2018-09-17T08:22:24.857077 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_wait_for_worker_monitor) EMS [10.0.0.101] as [admin] Starting work since worker monitor has started >[----] I, [2018-09-17T08:22:24.857454 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:22:24.857778 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:22:24.858085 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:22:24.858845 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#start_event_monitor) EMS [10.0.0.101] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:22:24.860369 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#worker_monitor_drb) EMS [10.0.0.101] as [admin] Initializing DRb Connection to MiqServer with ID=[1], NAME=[EVM], PID=[24762], GUID=[b855343c-ba6e-11e8-85b9-525400153f41] DRb URI=[drbunix:///tmp/worker_monitor20180917-24762-fp1fxm] >[----] I, [2018-09-17T08:22:24.863110 #11400:a51108] INFO -- : MIQ(ManageIQ::Providers::StorageManager::CinderManager::EventCatcher::Runner#do_work) EMS [10.0.0.101] as [admin] Waiting for the Monitor Thread to exit... >[----] I, [2018-09-17T08:22:25.289298 #27870:a51108] INFO -- : MIQ(MiqEventHandler::Runner#get_message_via_drb) Message id: [12146], MiqWorker id: [6], Zone: [default], Role: [event], Server: [], Ident: [ems], Target id: [6], Instance id: [], Task id: [], Command: [EmsEvent.add], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [{:event_type=>"router.delete.end", :source=>"OPENSTACK", :message=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}, :timestamp=>"2018-09-17T12:11:36.214079", :username=>nil, :full_data=>{:content=>{"message_id"=>"641b97fb-94ee-4582-a375-b1195f6911cb", "event_type"=>"router.delete.end", "timestamp"=>"2018-09-17T12:11:36.214079", "payload"=>{"name"=>"test_router_2", "project_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "request_id"=>"req-4cd340ef-28ff-4e48-ab53-d9d8885bb60e", "resource_id"=>"399865fb-7147-4dae-91ec-3b7f697de504", "service"=>"network.controller-1.localdomain", "tenant_id"=>"8de153dec58b4df89e3d966dfc5e21f7", "user_id"=>"269c82cd333742a7a349f3f6f81c8c3b"}}, :context=>{}, :user_id=>nil, :priority=>nil, :content_type=>nil}, :ems_id=>6}], Dequeued in: [4.941311039] seconds >[----] I, [2018-09-17T08:22:25.289416 #27870:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12146], Delivering... >[----] I, [2018-09-17T08:22:25.291745 #27870:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12146], State: [ok], Delivered in [0.002335851] seconds >/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:121:in `monitor_events': must be implemented in subclass (NotImplementedError) > from /var/www/miq/vmdb/app/models/manageiq/providers/base_manager/event_catcher/runner.rb:163:in `block in start_event_monitor' >[----] I, [2018-09-17T08:22:26.212080 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12147], MiqWorker id: [1], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["MiqServer", 1], "evm_worker_start", {:event_details=>"Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher"}], Dequeued in: [3.164119857] seconds >[----] I, [2018-09-17T08:22:26.212238 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12147], Delivering... >[----] I, [2018-09-17T08:22:26.219444 #27798:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"evm_worker_start"> inputs=<{:event_details=>"Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5855, :miq_event_id=>5855, "EventStream::event_stream"=>5855, :event_stream_id=>5855}> >[----] I, [2018-09-17T08:22:26.232230 #27798:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12149], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5855, :miq_event_id=>5855, "EventStream::event_stream"=>5855, :event_stream_id=>5855}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:22:26.232469 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12147], State: [ok], Delivered in [0.020243281] seconds >[----] I, [2018-09-17T08:22:26.647967 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#do_work) EMS [192.168.24.2] as [admin] Event Monitor Thread gone. Restarting... >[----] I, [2018-09-17T08:22:26.648375 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Validating Connection/Credentials >[----] I, [2018-09-17T08:22:26.648614 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::InfraManager: [uc_ci] >[----] I, [2018-09-17T08:22:26.648776 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Starting Event Monitor Thread >[----] I, [2018-09-17T08:22:26.649338 #28786:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::InfraManager::EventCatcher::Runner#start_event_monitor) EMS [192.168.24.2] as [admin] Started Event Monitor Thread >[----] I, [2018-09-17T08:22:29.115088 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 2 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:22:29.222987 #27806:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12149], MiqWorker id: [2], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"MiqServer", :object_id=>1, :attrs=>{:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5855, :miq_event_id=>5855, "EventStream::event_stream"=>5855, :event_stream_id=>5855}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [2.997349595] seconds >[----] I, [2018-09-17T08:22:29.223110 #27806:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12149], Delivering... >[----] I, [2018-09-17T08:22:29.226134 #27806:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [2] name [EvmGroup-super_administrator] >[----] I, [2018-09-17T08:22:29.226314 #27806:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"evm_worker_start", :event_details=>"Worker started: ID [70], PID [11400], GUID [54460268-ba74-11e8-86bf-525400153f41]", :type=>"ManageIQ::Providers::StorageManager::CinderManager::EventCatcher", "MiqEvent::miq_event"=>5855, :miq_event_id=>5855, "EventStream::event_stream"=>5855, :event_stream_id=>5855} for object [MiqServer.1] with state [] to Automate >[----] I, [2018-09-17T08:22:29.229000 #27806:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5855&MiqEvent%3A%3Amiq_event=5855&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B70%5D%2C%20PID%20%5B11400%5D%2C%20GUID%20%5B54460268-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5855&event_type=evm_worker_start&miq_event_id=5855&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server] >[----] I, [2018-09-17T08:22:29.245039 #27798:a51108] INFO -- : MIQ(MiqGenericWorker::Runner#get_message_via_drb) Message id: [12148], MiqWorker id: [1], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqEvent.raise_evm_event], Timeout: [600], Priority: [100], State: [dequeue], Deliver On: [], Data: [], Args: [["ManageIQ::Providers::Openstack::CloudManager", 5], "ems_auth_valid", {}], Dequeued in: [4.912900675] seconds >[----] I, [2018-09-17T08:22:29.245143 #27798:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12148], Delivering... >[----] I, [2018-09-17T08:22:29.250845 #27798:a51108] INFO -- : <AutomationEngine> MiqAeEvent.build_evm_event >> event=<"ems_auth_valid"> inputs=<{"MiqEvent::miq_event"=>5856, :miq_event_id=>5856, "EventStream::event_stream"=>5856, :event_stream_id=>5856}> >[----] I, [2018-09-17T08:22:29.252824 #27806:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5855&MiqEvent%3A%3Amiq_event=5855&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_details=Worker%20started%3A%20ID%20%5B70%5D%2C%20PID%20%5B11400%5D%2C%20GUID%20%5B54460268-ba74-11e8-86bf-525400153f41%5D&event_stream_id=5855&event_type=evm_worker_start&miq_event_id=5855&object_name=Event&type=ManageIQ%3A%3AProviders%3A%3AStorageManager%3A%3ACinderManager%3A%3AEventCatcher&vmdb_object_type=miq_server ManageIQ/System] >[----] I, [2018-09-17T08:22:29.264361 #27798:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12150], id: [], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5856, :miq_event_id=>5856, "EventStream::event_stream"=>5856, :event_stream_id=>5856}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}] >[----] I, [2018-09-17T08:22:29.264575 #27798:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12148], State: [ok], Delivered in [0.019427439] seconds >[----] I, [2018-09-17T08:22:29.316411 #27806:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:22:29.339456 #27806:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:22:29.350762 #27806:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/evm_worker_start] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:22:29.362603 #27806:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:22:29.366667 #27806:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:22:29.376024 #27806:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:22:29.381235 #27806:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:22:29.384339 #27806:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<MiqServer id: 1, guid: "b855343c-ba6e-11e8-85b9-525400153f41", status: "started", started_on: "2018-09-17 11:44:11", stopped_on: nil, pid: 24762, build: "20180911143221_8357814", percent_memory: 4.09, percent_cpu: 3.92, cpu_time: 29781.0, name: "EVM", capabilities: {:vixDisk=>true, :concurrent_miqproxies=>0}, last_heartbeat: "2018-09-17 12:22:12", os_priority: 20, is_master: true, logo: nil, version: "5.8.5.1", zone_id: 1, upgrade_status: nil, upgrade_message: nil, memory_usage: 378322944, memory_size: 770105344, hostname: nil, ipaddress: "10.0.0.30", drb_uri: "drbunix:///tmp/worker_monitor20180917-24762-fp1fxm", mac_address: "52:54:00:15:3f:41", vm_id: nil, has_active_userinterface: true, has_active_webservices: true, sql_spid: 27687, rh_registered: nil, rh_subscribed: nil, last_update_check: nil, updates_available: nil, rhn_mirror: false, log_file_depot_id: nil, proportional_set_size: 252956000, has_active_websocket: true, system_memory_free: 1961947136, system_memory_used: 7297028096, system_swap_free: 9122607104, system_swap_used: 0>] >[----] I, [2018-09-17T08:22:29.384504 #27806:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [evm_worker_start] >[----] I, [2018-09-17T08:22:29.393417 #27806:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [evm_worker_start] >[----] I, [2018-09-17T08:22:29.393581 #27806:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [evm_worker_start] Target: MiqServer Name: [EVM], Id: [1] >[----] I, [2018-09-17T08:22:29.399229 #27806:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:22:29.399748 #27806:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/evm_worker_start#create] >[----] I, [2018-09-17T08:22:29.400408 #27806:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12149], State: [ok], Delivered in [0.177305455] seconds >[----] I, [2018-09-17T08:22:29.922209 #27908:1316a54] INFO -- : <AuditSuccess> MIQ(NetworkRouterController.block in process_network_routers) userid: [admin] - [foEMGSmzsk] Record delete initiated >[----] I, [2018-09-17T08:22:29.939560 #27908:1316a54] INFO -- : MIQ(MiqQueue.put) Message id: [12151], id: [], Zone: [default], Role: [ems_operations], Server: [], Ident: [generic], Target id: [], Instance id: [6], Task id: [], Command: [ManageIQ::Providers::Openstack::NetworkManager::NetworkRouter.raw_delete_network_router], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:22:29.939722 #27908:1316a54] INFO -- : MIQ(MiqTask.generic_action_with_callback) Task: [5] Queued the action: [deleting Network Router for user admin] being run for user: [admin] >[----] I, [2018-09-17T08:22:34.002094 #27835:a51108] INFO -- : MIQ(MiqScheduleWorker::Runner#do_work) Number of scheduled items to be processed: 4. >[----] I, [2018-09-17T08:22:34.008749 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12152], id: [], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:22:34.013173 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12153], id: [], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:22:34.018203 #27835:a51108] INFO -- : MIQ(MiqQueue.put) Message id: [12154], id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [Job.check_jobs_for_timeout], Timeout: [600], Priority: [90], State: [ready], Deliver On: [], Data: [], Args: [] >[----] I, [2018-09-17T08:22:34.131191 #24762:a51108] INFO -- : MIQ(MiqServer#populate_queue_messages) Fetched 5 miq_queue rows for queue_name=generic, wcount=4, priority=200 >[----] I, [2018-09-17T08:22:34.321772 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12150], MiqWorker id: [4], Zone: [default], Role: [automate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqAeEngine.deliver], Timeout: [3600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [{:object_type=>"ManageIQ::Providers::Openstack::CloudManager", :object_id=>5, :attrs=>{:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5856, :miq_event_id=>5856, "EventStream::event_stream"=>5856, :event_stream_id=>5856}, :instance_name=>"Event", :user_id=>1, :miq_group_id=>1, :tenant_id=>1, :automate_message=>nil}], Dequeued in: [5.063851342] seconds >[----] I, [2018-09-17T08:22:34.321909 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12150], Delivering... >[----] I, [2018-09-17T08:22:34.326293 #27824:a51108] INFO -- : <AutomationEngine> User [admin] with current group ID [1] name [Tenant My Company access] >[----] I, [2018-09-17T08:22:34.326453 #27824:a51108] INFO -- : MIQ(MiqAeEngine.deliver) Delivering {:event_type=>"ems_auth_valid", "MiqEvent::miq_event"=>5856, :miq_event_id=>5856, "EventStream::event_stream"=>5856, :event_stream_id=>5856} for object [ManageIQ::Providers::Openstack::CloudManager.5] with state [] to Automate >[----] I, [2018-09-17T08:22:34.328772 #27824:a51108] INFO -- : <AutomationEngine> Instantiating [/System/Process/Event?EventStream%3A%3Aevent_stream=5856&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5856&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5856&event_type=ems_auth_valid&miq_event_id=5856&object_name=Event&vmdb_object_type=ext_management_system] >[----] I, [2018-09-17T08:22:34.355789 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [/System/Process/Event?EventStream%3A%3Aevent_stream=5856&ExtManagementSystem%3A%3Aext_management_system=5&MiqEvent%3A%3Amiq_event=5856&MiqServer%3A%3Amiq_server=1&User%3A%3Auser=1&event_stream_id=5856&event_type=ems_auth_valid&miq_event_id=5856&object_name=Event&vmdb_object_type=ext_management_system ManageIQ/System] >[----] I, [2018-09-17T08:22:34.420360 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:22:34.447239 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create ManageIQ/System/Event/MiqEvent] >[----] I, [2018-09-17T08:22:34.460602 #27824:a51108] INFO -- : <AutomationEngine> Instance [/ManageIQ/System/Event/MiqEvent/POLICY/ems_auth_valid] not found in MiqAeDatastore - trying [.missing] >[----] I, [2018-09-17T08:22:34.474295 #27824:a51108] INFO -- : <AutomationEngine> Following Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:22:34.478752 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [miqaedb:/System/event_handlers/event_enforce_policy#create ManageIQ/System] >[----] I, [2018-09-17T08:22:34.489363 #27824:a51108] INFO -- : <AutomationEngine> Updated namespace [System/event_handlers/event_enforce_policy ManageIQ/System] >[----] I, [2018-09-17T08:22:34.495559 #27824:a51108] INFO -- : <AutomationEngine> Invoking [builtin] method [/ManageIQ/System/event_handlers/event_enforce_policy] with inputs [{}] >[----] I, [2018-09-17T08:22:34.498393 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) target = [#<ManageIQ::Providers::Openstack::CloudManager id: 5, name: "oc_ci", created_on: "2018-09-17 11:55:21", updated_on: "2018-09-17 12:11:32", guid: "8e1cb774-ba70-11e8-86bf-525400153f41", zone_id: 1, type: "ManageIQ::Providers::Openstack::CloudManager", api_version: "v3", uid_ems: "default", host_default_vnc_port_start: nil, host_default_vnc_port_end: nil, provider_region: "", last_refresh_error: nil, last_refresh_date: "2018-09-17 12:11:32", provider_id: 1, realm: nil, tenant_id: 1, project: nil, parent_ems_id: nil, subscription: nil, last_metrics_error: nil, last_metrics_update_date: nil, last_metrics_success_date: nil, tenant_mapping_enabled: false>] >[----] I, [2018-09-17T08:22:34.498625 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Event Raised [ems_auth_valid] >[----] I, [2018-09-17T08:22:34.508414 #27824:a51108] INFO -- : MIQ(MiqEvent#process_evm_event) Alert for Event [ems_auth_valid] >[----] I, [2018-09-17T08:22:34.508545 #27824:a51108] INFO -- : MIQ(MiqAlert.evaluate_alerts) [ems_auth_valid] Target: ManageIQ::Providers::Openstack::CloudManager Name: [oc_ci], Id: [5] >[----] I, [2018-09-17T08:22:34.514365 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/event_handlers/event_enforce_policy#create] >[----] I, [2018-09-17T08:22:34.514990 #27824:a51108] INFO -- : <AutomationEngine> Followed Relationship [miqaedb:/System/Event/MiqEvent/POLICY/ems_auth_valid#create] >[----] I, [2018-09-17T08:22:34.515628 #27824:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12150], State: [ok], Delivered in [0.193717871] seconds >[----] I, [2018-09-17T08:22:34.525673 #27824:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12151], MiqWorker id: [4], Zone: [default], Role: [ems_operations], Server: [], Ident: [generic], Target id: [], Instance id: [6], Task id: [], Command: [ManageIQ::Providers::Openstack::NetworkManager::NetworkRouter.raw_delete_network_router], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [4.590618132] seconds >[----] I, [2018-09-17T08:22:34.525814 #27824:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12151], Delivering... >[----] I, [2018-09-17T08:22:34.534123 #27824:a51108] INFO -- : MIQ(ManageIQ::Providers::Openstack::CloudManager#with_provider_connection) Connecting through ManageIQ::Providers::Openstack::CloudManager: [oc_ci] >[----] I, [2018-09-17T08:22:34.727924 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12152], MiqWorker id: [3], Zone: [default], Role: [smartstate], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [job_dispatcher], Command: [JobProxyDispatcher.dispatch], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [0.72162806] seconds >[----] I, [2018-09-17T08:22:34.728061 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#deliver) Message id: [12152], Delivering... >[----] I, [2018-09-17T08:22:34.732295 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(JobProxyDispatcher#dispatch) Complete - Timings: {:pending_container_jobs=>0.002234220504760742, :container_jobs_to_dispatch_count=>0, :container_dispatching=>0.002251863479614258, :pending_vm_jobs=>0.0005221366882324219, :vm_jobs_to_dispatch_count=>0, :total_time=>0.003970146179199219} >[----] I, [2018-09-17T08:22:34.732484 #27816:a51108] INFO -- : Q-task_id([job_dispatcher]) MIQ(MiqQueue#delivered) Message id: [12152], State: [ok], Delivered in [0.004428001] seconds >[----] I, [2018-09-17T08:22:34.741484 #27816:a51108] INFO -- : MIQ(MiqPriorityWorker::Runner#get_message_via_drb) Message id: [12153], MiqWorker id: [3], Zone: [default], Role: [], Server: [b855343c-ba6e-11e8-85b9-525400153f41], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [MiqServer.status_update], Timeout: [600], Priority: [20], State: [dequeue], Deliver On: [], Data: [], Args: [], Dequeued in: [0.730647284] seconds >[----] I, [2018-09-17T08:22:34.741580 #27816:a51108] INFO -- : MIQ(MiqQueue#deliver) Message id: [12153], Delivering... >[----] I, [2018-09-17T08:22:34.791101 #27816:a51108] INFO -- : MIQ(MiqQueue#delivered) Message id: [12153], State: [ok], Delivered in [0.049499811] seconds >[----] I, [2018-09-17T08:22:34.957856 #28987:1099718] INFO -- : Querying OpenStack for events newer than 2018-09-17T12:11:36.214079... >
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=1484007">View the attachment on a separate page</a>.</b>
View Attachment As Raw
Actions:
View
Attachments on
bug 1593349
:
1453239
|
1453240
|
1479207
|
1484005
|
1484006
| 1484007