Bug 1851952

Summary:	"candlepin_events FAIL Not running" and wont restart
Product:	Red Hat Satellite	Reporter:	Stephen Wadeley <swadeley>
Component:	SELinux	Assignee:	Lukas Zapletal <lzap>
Status:	CLOSED ERRATA	QA Contact:	Lukáš Hellebrandt <lhellebr>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.8.0	CC:	jpathan, jturel, lhellebr, lzap, mmccune, pcreech
Target Milestone:	6.8.0	Keywords:	AutomationBlocker, Triaged, UpgradeBlocker, Upgrades
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	katello-selinux-3.2.0-1	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 13:03:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Stephen Wadeley 2020-06-29 14:07:49 UTC

Description of problem:

While verifying some bugs using snap 6, I noticed package list was empty.
Then I noticed:
"candlepin_events FAIL Not running"

Version-Release number of selected component (if applicable):

[root@dell-r330-12 ~]# rpm -qa satellite
satellite-6.8.0-0.6.beta.el7sat.noarch

[root@dell-r330-12 ~]# rpm -qa candlepin
candlepin-3.1.14-1.el7sat.noarch

How reproducible:
Unknown

Steps to Reproduce:
1. Unsure, but I ran a test to install and delete some Content Hosts
2. Varifed https://bugzilla.redhat.com/show_bug.cgi?id=1840525#c8
3. Noticed "candlepin_events FAIL Not running" and ran foreman-maintain health check

Actual results:


Starting the following service(s):
rh-mongodb34-mongod, postgresql, qdrouterd, qpidd, rh-redis5-redis, squid, pulp_celerybeat, pulp_resource_manager, pulp_streamer, pulp_workers, smart_proxy_dynflow_core, tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, puppetserver, dynflow-sidekiq@worker, dynflow-sidekiq@worker-hosts-queue, foreman-proxy
/ All services started                                                          
| Try 1/5: checking status of hammer ping                                       
Couldn't connect to the server: undefined method `to_sym' for nil:NilClass
- Try 2/5: checking status of hammer ping                                       
Couldn't connect to the server: undefined method `to_sym' for nil:NilClass
/ Try 3/5: checking status of hammer ping                                       
Couldn't connect to the server: undefined method `to_sym' for nil:NilClass
\ Try 4/5: checking status of hammer ping                                       
Couldn't connect to the server: undefined method `to_sym' for nil:NilClass
| Try 5/5: checking status of hammer ping                             [FAIL]    
Server response check failed!
--------------------------------------------------------------------------------
Rerunning the check after fix procedure
Check whether all services are running using the ping call:           [FAIL]
Couldn't connect to the server: undefined method `to_sym' for nil:NilClass



Expected results:

Candlpin should restart, possiblly even not stop


Additional info:

==> /var/log/foreman/production.log <==
2020-06-29T15:56:24 [D|dyn|] Executor heartbeat
2020-06-29T15:56:25 [E|app|37a00a40] Error occurred while starting Katello::CandlepinEventListener
2020-06-29T15:56:25 [E|app|37a00a40] Permission denied - connect(2) for "localhost" port 61613
2020-06-29T15:56:25 [E|app|37a00a40] /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/netio.rb:461:in `initialize'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/netio.rb:461:in `open'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/netio.rb:461:in `block in open_ssl_socket'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:76:in `timeout'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/netio.rb:460:in `open_ssl_socket'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/netio.rb:520:in `open_socket'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/utils.rb:116:in `block in socket'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/utils.rb:109:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/connection/utils.rb:109:in `socket'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/stomp/connection.rb:173:in `initialize'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/stomp/client.rb:134:in `new'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/stomp/client.rb:134:in `create_connection'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/stomp/client.rb:101:in `block in initialize'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:93:in `block in timeout'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:33:in `block in catch'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:33:in `catch'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:33:in `catch'
/opt/rh/rh-ruby25/root/usr/share/ruby/timeout.rb:108:in `timeout'
/opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/stomp/client.rb:99:in `initialize'

Comment 2 Jonathon Turel 2020-06-29 14:39:38 UTC

Connecting redmine issue https://projects.theforeman.org/issues/30088 from this bug

Comment 3 Bryan Kearney 2020-06-29 16:06:23 UTC

Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/30088 has been resolved.

Comment 4 Jonathon Turel 2020-06-29 20:44:56 UTC

*** Bug 1851180 has been marked as a duplicate of this bug. ***

Comment 5 Brad Buckingham 2020-06-30 12:54:54 UTC

*** Bug 1851806 has been marked as a duplicate of this bug. ***

Comment 6 Lukáš Hellebrandt 2020-07-14 13:37:37 UTC

Verified with Sat 6.8 snap 8.0.

The error messages are not shown in the log anymore and the issues caused by this (ie. bug 1851176, bug 1851167) have been fixed.

Comment 9 errata-xmlrpc 2020-10-27 13:03:46 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366