Bug 1193762 - `service qpidd status` returns 1 - hidden error is "ConnectionError: connection-forced: Connection must be encrypted.(320)"
Summary: `service qpidd status` returns 1 - hidden error is "ConnectionError: connecti...
Keywords:
Status: CLOSED DUPLICATE of bug 1246152
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Installation
Version: 6.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: Unspecified
Assignee: Alan Conway
QA Contact: Tazim Kolhar
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-02-18 06:32 UTC by Jan Hutař
Modified: 2019-10-10 09:37 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-03-07 07:39:27 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Jan Hutař 2015-02-18 06:32:28 UTC
Description of problem:
`service qpidd status` returns 1 even when the service is running:

# ps ax | grep qpid | grep -v grep
29123 ?        Ssl    2:39 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf --daemon --data-dir=/var/lib/qpidd --close-fd 9

Hidden error is "ConnectionError: connection-forced: Connection must be encrypted.(320)". To display it change following in /etc/init.d/qpidd in function qpid_ping:

-    $QPID_HA $QPID_HA_OPTIONS ping >/dev/null 2>&1
+    $QPID_HA $QPID_HA_OPTIONS ping


Version-Release number of selected component (if applicable):
Satellite-6.1.0-RHEL-6-20150210.0-Satellite-x86_64


How reproducible:
always


Steps to Reproduce:
1. Install Satellite without Capsule:
   # katello-installer --foreman-admin-email 'root@localhost' \
     --foreman-admin-username 'admin' --foreman-admin-password '<pass>'
2. # service qpidd status; echo $?


Actual results:
Exit code is 1, no error (or other output) is printed


Expected results:
Status check of the qpidd service should return 0 and should print "is running..."

Comment 1 RHEL Program Management 2015-02-18 20:06:37 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 3 Stephen Benjamin 2015-04-27 10:12:07 UTC
Can this go 6.1.0, or at least Z-stream after GA?  katello-service is broken....

Comment 5 Alan Conway 2015-05-20 13:02:40 UTC
Fixed by the following two upstream commits. Sorry about the delay.

------------------------------------------------------------------------
r1680552 | aconway | 2015-05-20 08:59:52 -0400 (Wed, 20 May 2015) | 8 lines

QPID-6549: `service qpidd status` returns 1 - hidden error is "ConnectionError: connection-forced: Connection must be encrypted.(320)"

The qpidd init script uses qpid-ha to probe the state of the broker.
In the bug reported security configuration on the broker was preventing qpid-ha from connecting.

The qpid-ha checks are only necessary when HA is configured, so this commit disables those
checks if it is not configured.

------------------------------------------------------------------------
r1680550 | aconway | 2015-05-20 08:59:46 -0400 (Wed, 20 May 2015) | 10 lines

QPID-6548: SYSV init scripts do not work properly wiht SSL-only broker.

Previously the broker was writing a PID file with the port number as a suffix.
This was confusing the tools when using SSL and no explicit port, as the actual
listening port is 5671 but qpidd -c was looking for 5672.

This commit introduces a simple --pidfile option which writes the pid exactly
where you tell it with no frills. The original port-pidfile setup is over
complex and not really necessary, it can be deprecated at some future time.

------------------------------------------------------------------------

Comment 6 Mike McCune 2015-06-23 15:55:14 UTC
This is resolved in qpid-cpp-0.30-9 which we are currently using

Comment 7 Tazim Kolhar 2015-07-08 09:07:28 UTC
VERIFIED:

# rpm -qa | grep foreman
ruby193-rubygem-foreman-tasks-0.6.12.8-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.9-1.el7sat.noarch
foreman-debug-1.7.2.29-1.el7sat.noarch
foreman-postgresql-1.7.2.29-1.el7sat.noarch
foreman-vmware-1.7.2.29-1.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
foreman-selinux-1.7.2.13-1.el7sat.noarch
foreman-1.7.2.29-1.el7sat.noarch
foreman-ovirt-1.7.2.29-1.el7sat.noarch
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
foreman-proxy-1.7.2.5-1.el7sat.noarch
ibm-x3655-03.ovirt.rhts.eng.bos.redhat.com-foreman-proxy-1.0-2.noarch
foreman-compute-1.7.2.29-1.el7sat.noarch
foreman-gce-1.7.2.29-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.0-8.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch
foreman-libvirt-1.7.2.29-1.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch
ibm-x3655-03.ovirt.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch
ibm-x3655-03.ovirt.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.18-1.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.4-1.el7sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.15-1.el7sat.noarch


steps:
Install Satellite without Capsule:
   # katello-installer --foreman-admin-email 'root@localhost' --foreman-admin-username 'admin' --foreman-admin-password *****

# service qpidd status; echo $?
Redirecting to /bin/systemctl status  qpidd.service
qpidd.service - An AMQP message broker daemon.
   Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled)
   Active: active (running) since Fri 2015-07-03 03:58:41 EDT; 5 days ago
     Docs: man:qpidd(1)
           http://qpid.apache.org/
 Main PID: 11925 (qpidd)
   CGroup: /system.slice/qpidd.service
           └─11925 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf

Jul 03 03:58:41 ibm-x3655-03.ovirt.rhts.eng.bos.redhat.com systemd[1]: Starte...
Jul 06 10:07:09 ibm-x3655-03.ovirt.rhts.eng.bos.redhat.com systemd[1]: Starte...
Hint: Some lines were ellipsized, use -l to show in full.
0

it prints running and exit code is 0

Comment 8 Bryan Kearney 2015-08-12 16:02:26 UTC
This bug was fixed in Satellite 6.1.1 which was delivered on 12 August, 2015.

Comment 9 Adriano Oliveira 2015-08-13 01:12:13 UTC
I got the same error in the Satellite 6.1.1

[root@v141p030 ~]# ps ax | grep qpid | grep -v grep
 1013 ?        Ssl    0:14 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf
 1123 ?        Ssl    0:01 /usr/sbin/qdrouterd -c /etc/qpid-dispatch/qdrouterd.conf
[root@v141p030 ~]# service qpidd status; echo $?
Redirecting to /bin/systemctl status  qpidd.service
qpidd.service - An AMQP message broker daemon.
   Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled)
   Active: active (running) since Wed 2015-08-12 21:27:28 BRT; 43min ago
     Docs: man:qpidd(1)
           http://qpid.apache.org/
 Main PID: 1013 (qpidd)
   CGroup: /system.slice/qpidd.service
           └─1013 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf

Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error Rejected un-encrypted connection.
Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error Connection qpid.10.0.204.253:5672-10.0.204.253:57518 closed by error: connection-forced: Connection must b...crypted.(320)
Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error Rejected un-encrypted connection.
Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error Rejected un-encrypted connection.
Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error Connection qpid.10.0.204.253:5672-10.0.204.253:57519 closed by error: connection-forced: Connection must b...crypted.(320)
Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error Connection qpid.10.0.204.253:5672-10.0.204.253:57519 closed by error: connection-forced: Connection must b...crypted.(320)
Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Security] error Rejected un-encrypted connection.
Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Protocol] error Connection qpid.127.0.0.1:5672-127.0.0.1:51787 closed by error: connection-forced: Connection must be encrypted.(320)
Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Security] error Rejected un-encrypted connection.
Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Protocol] error Connection qpid.127.0.0.1:5672-127.0.0.1:51787 closed by error: connection-forced: Connection must be encrypted.(320)
Hint: Some lines were ellipsized, use -l to show in full.
0

Comment 10 Adriano Oliveira 2015-08-13 01:18:31 UTC
(In reply to Adriano Oliveira from comment #9)
> I got the same error in the Satellite 6.1.1
> 
> [root@v141p030 ~]# ps ax | grep qpid | grep -v grep
>  1013 ?        Ssl    0:14 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf
>  1123 ?        Ssl    0:01 /usr/sbin/qdrouterd -c
> /etc/qpid-dispatch/qdrouterd.conf
> [root@v141p030 ~]# service qpidd status; echo $?
> Redirecting to /bin/systemctl status  qpidd.service
> qpidd.service - An AMQP message broker daemon.
>    Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled)
>    Active: active (running) since Wed 2015-08-12 21:27:28 BRT; 43min ago
>      Docs: man:qpidd(1)
>            http://qpid.apache.org/
>  Main PID: 1013 (qpidd)
>    CGroup: /system.slice/qpidd.service
>            └─1013 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf
> 
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error
> Connection qpid.10.0.204.253:5672-10.0.204.253:57518 closed by error:
> connection-forced: Connection must b...crypted.(320)
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error
> Connection qpid.10.0.204.253:5672-10.0.204.253:57519 closed by error:
> connection-forced: Connection must b...crypted.(320)
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error
> Connection qpid.10.0.204.253:5672-10.0.204.253:57519 closed by error:
> connection-forced: Connection must b...crypted.(320)
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Protocol] error
> Connection qpid.127.0.0.1:5672-127.0.0.1:51787 closed by error:
> connection-forced: Connection must be encrypted.(320)
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Protocol] error
> Connection qpid.127.0.0.1:5672-127.0.0.1:51787 closed by error:
> connection-forced: Connection must be encrypted.(320)
> Hint: Some lines were ellipsized, use -l to show in full.
> 0

For more detais, see the CASE 01491754 open in Support Cases at Red Hat portal.

Comment 12 Pavel Moravec 2015-10-17 07:39:11 UTC
(In reply to Adriano Oliveira from comment #9)
> I got the same error in the Satellite 6.1.1
> 
> [root@v141p030 ~]# ps ax | grep qpid | grep -v grep
>  1013 ?        Ssl    0:14 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf
>  1123 ?        Ssl    0:01 /usr/sbin/qdrouterd -c
> /etc/qpid-dispatch/qdrouterd.conf
> [root@v141p030 ~]# service qpidd status; echo $?
> Redirecting to /bin/systemctl status  qpidd.service
> qpidd.service - An AMQP message broker daemon.
>    Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled)
>    Active: active (running) since Wed 2015-08-12 21:27:28 BRT; 43min ago
>      Docs: man:qpidd(1)
>            http://qpid.apache.org/
>  Main PID: 1013 (qpidd)
>    CGroup: /system.slice/qpidd.service
>            └─1013 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf
> 
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error
> Connection qpid.10.0.204.253:5672-10.0.204.253:57518 closed by error:
> connection-forced: Connection must b...crypted.(320)
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error
> Connection qpid.10.0.204.253:5672-10.0.204.253:57519 closed by error:
> connection-forced: Connection must b...crypted.(320)
> Aug 12 21:41:07 v141p030 qpidd[1013]: 2015-08-12 21:41:07 [Protocol] error
> Connection qpid.10.0.204.253:5672-10.0.204.253:57519 closed by error:
> connection-forced: Connection must b...crypted.(320)
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Protocol] error
> Connection qpid.127.0.0.1:5672-127.0.0.1:51787 closed by error:
> connection-forced: Connection must be encrypted.(320)
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Security] error
> Rejected un-encrypted connection.
> Aug 12 21:41:08 v141p030 qpidd[1013]: 2015-08-12 21:41:08 [Protocol] error
> Connection qpid.127.0.0.1:5672-127.0.0.1:51787 closed by error:
> connection-forced: Connection must be encrypted.(320)
> Hint: Some lines were ellipsized, use -l to show in full.
> 0

These "errors" are false alarms triggered by running sosreport - see https://access.redhat.com/solutions/1587333 . Check that the timestamp matches when sosreport was running: from sos_logs/sos.log: 

2015-08-12 21:41:06,094 INFO: [plugin:qpid] collecting path '/etc/sasl2/qpidd.conf'
2015-08-12 21:41:06,095 INFO: [plugin:qpid] collecting path '/etc/qpid/qpidd.conf'
2015-08-12 21:41:08,048 INFO: [plugin:qpid] collecting output of 'ls -lanR /var/lib/qpidd'

I.e. qpidd plugin was - at that time - collecting some config files and running some qpid-stat / qpid-config commands on unencrypted connections.

Also notice zero at the end - "service qpidd status" returned zero, that is expected.

So my understanding is the bug has been fixed properly (since qpid-cpp-0.30-9).


(In reply to Bryan Kearney from comment #11)
> wpinheir is also seeing this still.

Waldirio, could you please provide more details? Or check if your experience isnt the same like above (sosrport or qpid-config / qpid-stat run before "service qpidd status")?

Comment 13 Waldirio M Pinheiro 2015-10-19 14:15:03 UTC
Pavel, good morning

Just checking in my environment, the latest version is working fine.

About the case opened by Adriano, the main issue was caused by proxy configuration (customer environment).

In another customer, same version, works fine.

We can close this BZ.

Thank You.
Waldirio

Comment 14 Pradeep Kumar Surisetty 2016-03-06 03:34:45 UTC
i noticed this on 6.1.7  and 6.2 as well


root@satserver ~]# service qpidd restart
Redirecting to /bin/systemctl restart  qpidd.service

[root@satserver ~]# service qpidd status
Redirecting to /bin/systemctl status  qpidd.service
● qpidd.service - An AMQP message broker daemon.
   Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2016-03-05 22:26:43 EST; 6s ago
     Docs: man:qpidd(1)
           http://qpid.apache.org/
 Main PID: 12991 (qpidd)
   CGroup: /system.slice/qpidd.service
           └─12991 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf

Mar 05 22:26:43 satserver systemd[1]: Started An AMQP message broker daemon..
Mar 05 22:26:43 satserver systemd[1]: Starting An AMQP message broker daemon...


[root@satserver ~]# qpid-queue-stats
Queue Name                                     Sec       Depth     Enq Rate     Deq Rate
========================================================================================
*** Error: Exception during connection setup: ConnectionFailed - (None, 'connection-forced: Connection must be encrypted.'), retrying...

^C
[root@satserver ~]# service qpidd status
Redirecting to /bin/systemctl status  qpidd.service
● qpidd.service - An AMQP message broker daemon.
   Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2016-03-05 22:26:43 EST; 2min 44s ago
     Docs: man:qpidd(1)
           http://qpid.apache.org/
 Main PID: 12991 (qpidd)
   CGroup: /system.slice/qpidd.service
           └─12991 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf

Mar 05 22:27:26 satserver qpidd[12991]: 2016-03-05 22:27:26 [Security] error Rejected un-encrypted connection.
Mar 05 22:27:26 satserver qpidd[12991]: 2016-03-05 22:27:26 [Protocol] error Connection qpid.[::1]:5672-[::1]:38011 closed by error: connecti...ted.(320)
Mar 05 22:27:58 satserver qpidd[12991]: 2016-03-05 22:27:58 [Security] error Rejected un-encrypted connection.
Mar 05 22:27:58 satserver qpidd[12991]: 2016-03-05 22:27:58 [Security] error Rejected un-encrypted connection.
Mar 05 22:27:58 satserver qpidd[12991]: 2016-03-05 22:27:58 [Protocol] error Connection qpid.[::1]:5672-[::1]:38012 closed by error: connecti...ted.(320)
Mar 05 22:27:58 satserver qpidd[12991]: 2016-03-05 22:27:58 [Protocol] error Connection qpid.[::1]:5672-[::1]:38012 closed by error: connecti...ted.(320)
Mar 05 22:29:02 satserver qpidd[12991]: 2016-03-05 22:29:02 [Security] error Rejected un-encrypted connection.
Mar 05 22:29:02 satserver qpidd[12991]: 2016-03-05 22:29:02 [Protocol] error Connection qpid.[::1]:5672-[::1]:38013 closed by error: connecti...ted.(320)
Mar 05 22:29:02 satserver qpidd[12991]: 2016-03-05 22:29:02 [Security] error Rejected un-encrypted connection.
Mar 05 22:29:02 satserver qpidd[12991]: 2016-03-05 22:29:02 [Protocol] error Connection qpid.[::1]:5672-[::1]:38013 closed by error: connecti...ted.(320)
Hint: Some lines were ellipsized, use -l to show in full.
[root@satserver ~]#

Comment 15 Pavel Moravec 2016-03-07 07:39:27 UTC

*** This bug has been marked as a duplicate of bug 1246152 ***


Note You need to log in before you can comment on or make changes to this bug.