Bug 1429714

Summary:	CHRONYD getting stopped/failed in CFME-4.2 Appliances after 5-10 minutes
Product:	Red Hat CloudForms Management Engine	Reporter:	Jared Deubel <jdeubel>
Component:	Appliance	Assignee:	Nick Carboni <ncarboni>
Status:	CLOSED CURRENTRELEASE	QA Contact:	luke couzens <lcouzens>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	5.7.0	CC:	abellott, cpelland, gtanzill, jhardy, lcouzens, ncarboni, obarenbo, sacpatil, simaishi, tachoi
Target Milestone:	GA	Keywords:	TestOnly, ZStream
Target Release:	5.8.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	5.8.0.4	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1429999 (view as bug list)		Environment:
Last Closed:	2017-06-12 16:31:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1429999

Description Jared Deubel 2017-03-06 23:39:11 UTC

Description of problem:
CHRONYD getting stopped/failed  in CFME-4.2 Appliances after 5-10 minutes

snippet from evm.log
=============================================
[----] I, [2017-03-01T13:18:23.607926 #13280:cc513c]  INFO -- : MIQ(MiqServer#ntp_reload) Synchronizing ntp settings: {:server=>["ntp-test-01.service.test", "ntp-inf-02.service.acn"]}
[----] I, [2017-03-01T13:18:23.610255 #13272:cc513c]  INFO -- : MIQ(MiqServer#ntp_reload) Synchronizing ntp settings: {:server=>["ntp-test-01.service.test", "ntp-test-02.service.test"]}
[----] E, [2017-03-01T13:18:23.763361 #13272:cc513c] ERROR -- : AwesomeSpawn: /bin/systemctl exit code: 1
[----] E, [2017-03-01T13:18:23.763556 #13272:cc513c] ERROR -- : AwesomeSpawn: Job for chronyd.service failed because start of the service was attempted too often. See "systemctl status chronyd.service" and "journalctl -xe" for details.
To force a start use "systemctl reset-failed chronyd.service" followed by "systemctl start chronyd.service" again.

[----] E, [2017-03-01T13:18:23.763863 #13272:cc513c] ERROR -- : MIQ(MiqQueue#deliver) Message id: [99000012205780], Error: [/bin/systemctl exit code: 1]
[----] E, [2017-03-01T13:18:23.763926 #13280:cc513c] ERROR -- : AwesomeSpawn: /bin/systemctl exit code: 1
[----] E, [2017-03-01T13:18:23.764103 #13280:cc513c] ERROR -- : AwesomeSpawn: Job for chronyd.service failed because start of the service was attempted too often. See "systemctl status chronyd.service" and "journalctl -xe" for details.
To force a start use "systemctl reset-failed chronyd.service" followed by "systemctl start chronyd.service" again.

[----] E, [2017-03-01T13:18:23.764243 #13272:cc513c] ERROR -- : [AwesomeSpawn::CommandResultError]: /bin/systemctl exit code: 1  Method:[rescue in deliver]
[----] E, [2017-03-01T13:18:23.764624 #13280:cc513c] ERROR -- : MIQ(MiqQueue#deliver) Message id: [99000012205779], Error: [/bin/systemctl exit code: 1]
[----] E, [2017-03-01T13:18:23.764731 #13272:cc513c] ERROR -- : /opt/rh/cfme-gemset/gems/awesome_spawn-1.4.1/lib/awesome_spawn.rb:105:in `run!'
/opt/rh/cfme-gemset/gems/linux_admin-0.19.0/lib/linux_admin/common.rb:24:in `run!'
/opt/rh/cfme-gemset/gems/linux_admin-0.19.0/lib/linux_admin/service/systemd_service.rb:21:in `start'
/opt/rh/cfme-gemset/gems/linux_admin-0.19.0/lib/linux_admin/service/systemd_service.rb:40:in `restart'
/opt/rh/cfme-gemset/gems/linux_admin-0.19.0/lib/linux_admin/chrony.rb:28:in `restart_service_if_running'
/opt/rh/cfme-gemset/gems/linux_admin-0.19.0/lib/linux_admin/chrony.rb:21:in `add_servers'
/var/www/miq/vmdb/app/models/miq_server/ntp_management.rb:63:in `apply_ntp_server_settings'
/var/www/miq/vmdb/app/models/miq_server/ntp_management.rb:51:in `ntp_reload'
/var/www/miq/vmdb/app/models/miq_queue.rb:347:in `block in deliver'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:91:in `block in timeout'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `block in catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:33:in `catch'
/opt/rh/rh-ruby23/root/usr/share/ruby/timeout.rb:106:in `timeout'
/var/www/miq/vmdb/app/models/miq_queue.rb:343:in `deliver'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:106:in `deliver_queue_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:134:in `deliver_message'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:152:in `block in do_work'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:146:in `loop'
/var/www/miq/vmdb/app/models/miq_queue_worker_base/runner.rb:146:in `do_work'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:334:in `block in do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:331:in `loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:331:in `do_work_loop'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:153:in `run'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:128:in `start'
/var/www/miq/vmdb/app/models/miq_worker/runner.rb:21:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:343:in `block in start'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork'
/opt/rh/cfme-gemset/gems/nakayoshi_fork-0.0.3/lib/nakayoshi_fork.rb:24:in `fork'
/var/www/miq/vmdb/app/models/miq_worker.rb:341:in `start'
/var/www/miq/vmdb/app/models/miq_worker.rb:270:in `start_worker'
/var/www/miq/vmdb/app/models/miq_worker.rb:150:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_worker.rb:150:in `times'
/var/www/miq/vmdb/app/models/miq_worker.rb:150:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:52:in `block in sync_workers'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `each'
/var/www/miq/vmdb/app/models/miq_server/worker_management/monitor.rb:50:in `sync_workers'
/var/www/miq/vmdb/app/models/miq_server.rb:158:in `start'
/var/www/miq/vmdb/app/models/miq_server.rb:249:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:65:in `start'
/var/www/miq/vmdb/lib/workers/evm_server.rb:92:in `start'
/var/www/miq/vmdb/lib/workers/bin/evm_server.rb:4:in `<main>'
===========================


Version-Release number of selected component (if applicable):
5.7.0.17

Comment 2 Sachin 2017-03-07 12:08:53 UTC

Can we have ETA for hotfix?

Comment 3 Gregg Tanzillo 2017-03-07 15:28:53 UTC

https://github.com/ManageIQ/manageiq/pull/14208

Comment 7 luke couzens 2017-03-27 18:03:25 UTC

Verified in 5.8.0.7