Bug 509450

Summary: OSAD not connecting through RHN Proxy in x86_64
Product: Red Hat Satellite Proxy 5 Reporter: John Matthews <jmatthew>
Component: ServerAssignee: Miroslav Suchý <msuchy>
Status: CLOSED CURRENTRELEASE QA Contact: Tomas Lestach <tlestach>
Severity: high Docs Contact:
Priority: urgent    
Version: 530CC: bperkins, cperry, jpazdziora, mzazrivec, tlestach, whayutin
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: sat530 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-09-10 14:39:23 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 456999    

Description John Matthews 2009-07-02 20:26:01 UTC
Description of problem:

I am seeing connection errors with OSAD through a x86_64 proxy.
I tried 2 scenarios, OSAD installed and running on 64bit proxy, and OSAD not installed on 64 bit proxy.

1st Scenario:  OSAD installed on 64bit proxy
1) install OSAD on 64bit proxy, this works and ping through sat-proxy is fine
2) register client(rhel5.3-i386) to 64bit proxy through https, ensure ssl-cert is configured
3) on client, /etc/init.d/osad restart

Server does not support TLS - <starttls /> not in <features /> stanza
--> <presence type='unavailable' />

2009-07-02 15:15:37 jabber_lib.print_message: SSLError
2009-07-02 15:15:37


2nd Scenario: OSAD is _not_ installed on 64bit proxy
1) setup 64bit proxy, do _not_ install OSAD
2) register client (rhel5.3-i386) to 64bit proxy through https, ensure ssl-cert is configured
3) on client, /etc/init.d/osad restart

2009-07-02 16:03:23 jabber_lib.connect: 
2009-07-02 16:03:23 jabber_lib.connect: Attempting to connect
2009-07-02 16:03:23 jabber_lib.print_message: socket error
2009-07-02 16:03:23 jabber_lib.print_message: Could not connect to jabber server test09-64.rhndev.redhat.com
2009-07-02 16:03:23 jabber_lib.setup_connection: Could not connect to any jabber server



We took the same client and connected it through a 32bit proxy and OSAD connected fine.
Also took the same client and connected it directly to the satellite and OSAD connected fine.

The issue has only show up on x86_64 bit proxy.
We also reprovisioned the 64 bit guest and retried the install of the RHN proxy to see if it was a fluke in the install.



Version-Release number of selected component (if applicable):
# rpm -qa | grep proxy
rhn-proxy-branding-5.3.0.24-1.el5sat
spacewalk-proxy-docs-0.4.1-2.el5sat
spacewalk-proxy-redirect-0.5.7-7.el5sat
spacewalk-proxy-installer-0.5.25-13.el5sat
spacewalk-proxy-common-0.5.7-7.el5sat
spacewalk-proxy-package-manager-0.5.7-7.el5sat
spacewalk-proxy-selinux-0.5.2-6.el5sat
spacewalk-proxy-monitoring-0.4.4-3.el5sat
spacewalk-proxy-broker-0.5.7-7.el5sat
spacewalk-proxy-management-0.5.7-7.el5sat



  
Actual results:

From 1st Scenario full output of osad restart
[root@rlx-0-24 rhn]# /etc/init.d/osad restart
Shutting down osad:                                        [  OK  ]
Starting osad: 2009-07-02 15:15:37 osad._setup_config: Updating configuration
2009-07-02 15:15:37 osad._setup_config: Time drift 3
2009-07-02 15:15:37 osad._setup_config: Client name 78d6ccbf19525bf7
2009-07-02 15:15:37 osad._setup_config: Shared key b93eb1ec06c98fa2c46f8224304e9917fc98bad5
2009-07-02 15:15:37 jabber_lib.setup_connection: Connecting to test09-64.rhndev.redhat.com
2009-07-02 15:15:37 jabber_lib._get_jabber_client: 
2009-07-02 15:15:37 jabber_lib._get_jabber_client: Connecting to test09-64.rhndev.redhat.com
2009-07-02 15:15:37 jabber_lib.__init__: 
2009-07-02 15:15:37 jabber_lib.__init__: 
2009-07-02 15:15:37 jabber_lib.check_cert: Loading cert <X509Name object '/C=US/ST=NC/L=Raleigh/O=Red Hat/OU=RHEN/CN=Red Hat Test'>
2009-07-02 15:15:37 jabber_lib.connect: 
2009-07-02 15:15:37 jabber_lib.connect: Attempting to connect
--> <?xml version='1.0' encoding='UTF-8'?><stream:stream to='test09-64.rhndev.redhat.com' xmlns='jabber:client' xmlns:stream='http://etherx.jabber.org/streams' version='1.0'>

2009-07-02 15:15:37 jabber_lib.process: 300
2009-07-02 15:15:37 jabber_lib.process: before select(); timeout 299.999993801
2009-07-02 15:15:37 jabber_lib.process: select() returned
2009-07-02 15:15:37 jabber_lib._auth_dispatch: <features><mechanisms xmlns = 'urn:ietf:params:xml:ns:xmpp-sasl' ><mechanism>DIGEST-MD5</mechanism><mechanism>PLAIN</mechanism></mechanisms></features>
<-- <features><mechanisms xmlns = 'urn:ietf:params:xml:ns:xmpp-sasl' ><mechanism>DIGEST-MD5</mechanism><mechanism>PLAIN</mechanism></mechanisms></features>

2009-07-02 15:15:37 jabber_lib.connect: Connected
2009-07-02 15:15:37 jabber_lib.connect: Expecting features stanza, got: <features><mechanisms xmlns = 'urn:ietf:params:xml:ns:xmpp-sasl' ><mechanism>DIGEST-MD5</mechanism><mechanism>PLAIN</mechanism></mechanisms></features>
2009-07-02 15:15:37 jabber_lib.connect: starttls node None
Server does not support TLS - <starttls /> not in <features /> stanza
--> <presence type='unavailable' />

2009-07-02 15:15:37 jabber_lib.print_message: SSLError
2009-07-02 15:15:37 jabber_lib.print_message: Could not connect to jabber server test09-64.rhndev.redhat.com
2009-07-02 15:15:37 jabber_lib.setup_connection: Could not connect to any jabber server
2009-07-02 15:15:37 jabber_lib.push_to_background: Pushing process into background




From 2nd Scenario full output of osad restart:
[root@rlx-0-24 rhn]# /etc/init.d/osad restartShutting down osad:                                        [  OK  ]
Starting osad: 2009-07-02 16:03:23 osad._setup_config: Updating configuration
2009-07-02 16:03:23 osad._setup_config: Time drift 3
2009-07-02 16:03:23 osad._setup_config: Client name 7c368a0f04976093
2009-07-02 16:03:23 osad._setup_config: Shared key 294ea2d4b0e15968d2f6b0e2cd1c1e5f9288a13c
2009-07-02 16:03:23 jabber_lib.setup_connection: Connecting to test09-64.rhndev.redhat.com
2009-07-02 16:03:23 jabber_lib._get_jabber_client: 
2009-07-02 16:03:23 jabber_lib._get_jabber_client: Connecting to test09-64.rhndev.redhat.com
2009-07-02 16:03:23 jabber_lib.__init__: 
2009-07-02 16:03:23 jabber_lib.__init__: 
2009-07-02 16:03:23 jabber_lib.check_cert: Loading cert <X509Name object '/C=US/ST=NC/L=Raleigh/O=Red Hat/OU=RHEN/CN=Red Hat Test'>
2009-07-02 16:03:23 jabber_lib.connect: 
2009-07-02 16:03:23 jabber_lib.connect: Attempting to connect
2009-07-02 16:03:23 jabber_lib.print_message: socket error
2009-07-02 16:03:23 jabber_lib.print_message: Could not connect to jabber server test09-64.rhndev.redhat.com
2009-07-02 16:03:23 jabber_lib.setup_connection: Could not connect to any jabber server
2009-07-02 16:03:23 jabber_lib.push_to_background: Pushing process into background

Comment 4 Miroslav Suchý 2009-07-07 14:54:28 UTC
[16:51] <msuchy> jmatthews: I think that the second scenario is not a bug. but I dunno how it is supposed to work :(
[16:52] <jmatthews> msuchy, don't worry about 2nd scenario, it was just exploration and more info, I wasn't sure if it would be 

I'm reducing this bug to first scenario only.
This bug happens on on our sputnik-stage too - that is 5.2 satellite. Therefore it is not regression.

Comment 8 Miroslav Suchý 2009-07-09 14:22:43 UTC
It happens even with jabberd-2.0s10-3.42.el5.x86_64.rpm from sat520

Comment 9 Miroslav Suchý 2009-07-09 15:38:33 UTC
Adelton noticed that those 64bit proxy do not have jabber cert:
[root@xen68 jabberd]# grep pemfile c2s.xml
    <pemfile>/etc/jabberd/server.pem</pemfile>
    <pemfile>/etc/jabberd/server.pem</pemfile>
[root@xen68 jabberd]# ls -l /etc/jabberd/server.pem
ls: /etc/jabberd/server.pem: No such file or directory

I have this file on 32 bit proxy.

On 64bits however exist:
/etc/pki/spacewalk/jabberd/server.pem
It looked sane so I tried to copy it to /etc/jabberd/server.pem, but it doesnot  help.
If I removed /etc/jabberd/server.pem on those 32bit proxy and restarted jabberd on proxy and osad on client I got that familiar output:
 Starting osad: Server does not support TLS - <starttls /> not in <features /> stanza

I will try to find why the file is missing there.

Comment 10 Miroslav Suchý 2009-07-10 12:15:03 UTC
Got it. The problem is not in architecture. 
WebUI correctly generate /etc/jabberd/server.pem, but command line installer not. Therefore if you run on i386 installed from webUI and even later upgraded using CLI - it runs.
But if you install for the first time using CLI, jabberd ssl will not work.

Comment 11 Miroslav Suchý 2009-07-10 13:39:59 UTC
Hmm, situation is as follows:
i386 - webui - ok
x86_64 - webui - ok
i386 - cli - fail
     - when run cp cp /etc/pki/spacewalk/jabberd/server.pem /etc/jabberd/server.pem - it works
x86_64 - cli - fail
     - when I copy server.pem on propper place, fail too :(

Comment 12 Miroslav Suchý 2009-07-13 15:00:46 UTC
Fixed by commit: ed7ca1abd6520909519d7a0b17a0081babc1f274
It works even on x86_64 and I tested update of jabberd from one 2.2.x to another 2.2.x and it works too.

Comment 13 Miroslav Suchý 2009-07-15 13:07:59 UTC
iso 20090714
moving ON_QA

Comment 14 Tomas Lestach 2009-07-20 14:03:31 UTC
Following scenario#1 according to #Description.
(According to Comment#11, the problem is not arch depended, but cli/webui installer depended, so testinf i386 proxy)

******************************************************************************************
* when using spacewalk-proxy-installer-0.5.25-15.el5sat:
on client:
# service osad restart
Shutting down osad:                                        [  OK  ]
Starting osad: Server does not support TLS - <starttls /> not in <features /> stanza
                                                           [  OK  ]
[problem reproduced]
******************************************************************************************
* when using spacewalk-proxy-installer-0.5.25-16.el5sat:
on client:
# service osad restart
Shutting down osad:                                        [  OK  ]
Starting osad:                                             [  OK  ]

on WEBUI:
-----------------------------------------------------
OSA Status:	 online as of 7/20/09 9:58:43 AM EDT
Last pinged: 7/20/09 9:58:44 AM EDT 
Ping System
-----------------------------------------------------

[problem didn't show up]
******************************************************************************************

Verified with Satellite-5.3.0-RHEL5-re20090714.0!

Comment 15 Milan Zázrivec 2009-08-10 17:36:25 UTC
Verified in stage -> RELEASE_PENDING

Comment 16 Brandon Perkins 2009-09-10 14:39:23 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2009-1433.html