Bug 1222912

Summary: katello-agent doesn't work when custom certs are used
Product: Red Hat Satellite Reporter: Ivan Necas <inecas>
Component: InstallationAssignee: Ivan Necas <inecas>
Status: CLOSED CURRENTRELEASE QA Contact: Tazim Kolhar <tkolhar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: andrew.schofield, bbuckingham, bkearney, cwelton, mmccune, tkolhar
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
URL: http://projects.theforeman.org/issues/10670
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-08-12 13:59:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ivan Necas 2015-05-19 12:32:41 UTC 
Description of problem:
When using custom certificates (issued by commercial or user custom ca), the katello-agent is not able to authorize against the qpid.

Version-Release number of selected component (if applicable):
6.1.0

How reproducible:
Always

Steps to Reproduce:
1. issue custom certificates outside of the installer (https://github.com/iNecas/ownca can be used to do so)
2. configure the katello to use the certificates https://github.com/Katello/katello-installer#custom-server-certificates
3. register a client
4. install the katello-agent

Actual results:

The logs complain about not being able to connect to the qpid. The installation tasks from katello time-out

Expected results:

everything works

Additional info:

The issue was introduced by https://github.com/Katello/puppet-certs/pull/44, with incorrect assumption that the server_ca and candlepin-local ca are always the same (which is not true, when the commercial CA is used as a server_ca). Therefore, we can't use the rhsm settings for using in the agent https://github.com/Katello/katello-agent/pull/20, as that's different use-case and different CA to be used: the rhsm needs a CA to verify that the sat6 server is valid, while the agent uses it for verifying the client certs of the qpid broker.

I also ask for automating this workflow to avoid regressions
Comment 2 Ivan Necas 2015-06-01 16:19:20 UTC 
Created redmine issue http://projects.theforeman.org/issues/10670 from this bug
Comment 3 Ivan Necas 2015-06-01 16:28:42 UTC 
PRs against the agent 	https://github.com/Katello/katello-agent/pull/23 and puppet-certs https://github.com/Katello/puppet-certs/pull/62 opened
Comment 4 Ivan Necas 2015-06-02 15:53:01 UTC 
Small PR to address some PR comments that came after the merge https://github.com/Katello/katello-agent/pull/24
Comment 6 Tazim Kolhar 2015-06-05 10:48:25 UTC 
FAILEDQA:
# rpm -qa | grep foreman
foreman-compute-1.7.2.26-1.el6_6sat.noarch
ruby193-rubygem-foreman-redhat_access-0.1.0-1.el6_6sat.noarch
rubygem-hammer_cli_foreman-0.1.4.13-1.el6_6sat.noarch
foreman-vmware-1.7.2.26-1.el6_6sat.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el6_6sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el6_6sat.noarch
foreman-postgresql-1.7.2.26-1.el6_6sat.noarch
foreman-libvirt-1.7.2.26-1.el6_6sat.noarch
ruby193-rubygem-foreman-tasks-0.6.12.7-1.el6_6sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el6_6sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.6-1.el6_6sat.noarch
foreman-selinux-1.7.2.13-1.el6_6sat.noarch
foreman-debug-1.7.2.26-1.el6_6sat.noarch
foreman-ovirt-1.7.2.26-1.el6_6sat.noarch
foreman-gce-1.7.2.26-1.el6_6sat.noarch
ruby193-rubygem-foreman_discovery-2.0.0.15-1.el6_6sat.noarch
foreman-proxy-1.7.2.5-1.el6_6sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.4-1.el6_6sat.noarch
intel-s3e3432-01.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch
intel-s3e3432-01.rhts.eng.bos.redhat.com-foreman-proxy-1.0-2.noarch
ruby193-rubygem-foreman_docker-1.2.0.14-1.el6_6sat.noarch
intel-s3e3432-01.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch
foreman-1.7.2.26-1.el6_6sat.noarch
ruby193-rubygem-foreman_hooks-0.3.7-2.el6_6sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el6_6sat.noarch

steps:
# katello-installer --certs-server-cert /root/ownca/intel-s3e3432-01.rhts.eng.bos.redhat.com/intel-s3e3432-01.rhts.eng.bos.redhat.com.crt --certs-server-cert-req /root/ownca/intel-s3e3432-01.rhts.eng.bos.redhat.com/intel-s3e3432-01.rhts.eng.bos.redhat.com.crt.req --certs-server-key /root/ownca/intel-s3e3432-01.rhts.eng.bos.redhat.com/intel-s3e3432-01.rhts.eng.bos.redhat.com.key --certs-server-ca-cert /root/ownca/cacert.crt --certs-update-server --certs-update-server-ca
Marking certificate /root/ssl-build/intel-s3e3432-01.rhts.eng.bos.redhat.com/intel-s3e3432-01.rhts.eng.bos.redhat.com-apache for update
Marking certificate /root/ssl-build/intel-s3e3432-01.rhts.eng.bos.redhat.com/intel-s3e3432-01.rhts.eng.bos.redhat.com-foreman-proxy for update
Marking certificate /root/ssl-build/katello-server-ca for update
 /Stage[main]/Apache::Service/Service[httpd]: Failed to call refresh: Could not start Service[httpd]: Execution of '/sbin/service httpd start' returned 1: Starting httpd: [FAILED]
 /Stage[main]/Apache::Service/Service[httpd]: Could not start Service[httpd]: Execution of '/sbin/service httpd start' returned 1: Starting httpd: [FAILED]
 /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[intel-s3e3432-01.rhts.eng.bos.redhat.com]: Could not evaluate: Connection refused - connect(2)
 /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[intel-s3e3432-01.rhts.eng.bos.redhat.com]: Failed to call refresh: Connection refused - connect(2)
 /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[intel-s3e3432-01.rhts.eng.bos.redhat.com]: Connection refused - connect(2)
Installing             Done                                               [100%] [..................................................................]
Comment 7 Ivan Necas 2015-06-10 07:41:33 UTC 
It failed due to different bug, that was switched to modified, putting back to modified this as well, to turn to ON_QA at once with https://bugzilla.redhat.com/show_bug.cgi?id=1227757
Comment 9 Mike McCune 2015-06-10 21:17:38 UTC 
WORKAROUND:


1) Copy /etc/pki/katello/certs/katello-default-ca.crt from your Satellite to your capsule and place in: /etc/rhsm/ca/

2) On the capsule, hand edit /usr/lib/gofer/plugins/katelloplugin.py

3) Go to line 157:

plugin.cfg.messaging.cacert = rhsm_conf['rhsm']['repo_ca_cert'] % rhsm_conf['rhsm']

4) Comment that line out and add a line below:

    #plugin.cfg.messaging.cacert = rhsm_conf['rhsm']['repo_ca_cert'] % rhsm_conf['rhsm']
    plugin.cfg.messaging.cacert = rhsm_conf['rhsm']['ca_cert_dir'] + 'katello-default-ca.crt'

5) restart the 'goferd' service on your Capsule.  Should see connection success in /var/log/messages.
Comment 10 Andrew Schofield 2015-06-11 18:23:59 UTC 
*** Bug 1219961 has been marked as a duplicate of this bug. ***
Comment 11 Tazim Kolhar 2015-06-12 08:59:43 UTC 
VERIFIED:
# rpm -qa | grep foreman
ruby193-rubygem-foreman_discovery-2.0.0.15-1.el7sat.noarch
foreman-libvirt-1.7.2.27-1.el7sat.noarch
ruby193-rubygem-foreman_gutterball-0.0.1.9-1.el7sat.noarch
ruby193-rubygem-foreman_docker-1.2.0.14-1.el7sat.noarch
rubygem-hammer_cli_foreman_discovery-0.0.1.10-1.el7sat.noarch
foreman-selinux-1.7.2.13-1.el7sat.noarch
dell-pe1955-02.rhts.eng.bos.redhat.com-foreman-proxy-1.0-2.noarch
foreman-compute-1.7.2.27-1.el7sat.noarch
foreman-gce-1.7.2.27-1.el7sat.noarch
ruby193-rubygem-foreman-redhat_access-0.2.0-8.el7sat.noarch
rubygem-hammer_cli_foreman-0.1.4.14-1.el7sat.noarch
foreman-debug-1.7.2.27-1.el7sat.noarch
foreman-vmware-1.7.2.27-1.el7sat.noarch
ruby193-rubygem-foreman-tasks-0.6.12.8-1.el7sat.noarch
rubygem-hammer_cli_foreman_tasks-0.0.3.4-1.el7sat.noarch
rubygem-hammer_cli_foreman_docker-0.0.3.7-1.el7sat.noarch
foreman-proxy-1.7.2.5-1.el7sat.noarch
dell-pe1955-02.rhts.eng.bos.redhat.com-foreman-client-1.0-1.noarch
ruby193-rubygem-foreman_bootdisk-4.0.2.13-1.el7sat.noarch
dell-pe1955-02.rhts.eng.bos.redhat.com-foreman-proxy-client-1.0-1.noarch
foreman-ovirt-1.7.2.27-1.el7sat.noarch
rubygem-hammer_cli_foreman_bootdisk-0.1.2.7-1.el7sat.noarch
foreman-1.7.2.27-1.el7sat.noarch
ruby193-rubygem-foreman_hooks-0.3.7-2.el7sat.noarch
foreman-postgresql-1.7.2.27-1.el7sat.noarch

steps:
1. issue custom certificates outside of the installer (https://github.com/iNecas/ownca can be used to do so)
2. configure the katello to use the certificates https://github.com/Katello/katello-installer#custom-server-certificates
# katello-installer --certs-server-cert /root/ownca/dell-pe1955-02.rhts.eng.bos.redhat.com/dell-pe1955-02.rhts.eng.bos.redhat.com.crt --certs-server-cert-req /root/ownca/dell-pe1955-02.rhts.eng.bos.redhat.com/dell-pe1955-02.rhts.eng.bos.redhat.com.crt.req --certs-server-key /root/ownca/dell-pe1955-02.rhts.eng.bos.redhat.com/dell-pe1955-02.rhts.eng.bos.redhat.com.key --certs-server-ca-cert /root/ownca/cacert.crt --certs-update-server --certs-update-server-ca
3. register a client
4. install the katello-agent
# yum install -y katello-agent
Installed:
  katello-agent.noarch 0:2.2.5-1.el6_6sat                                       

Complete!
Comment 12 Bryan Kearney 2015-08-11 13:25:09 UTC 
This bug is slated to be released with Satellite 6.1.
Comment 13 Bryan Kearney 2015-08-12 13:59:56 UTC 
This bug was fixed in version 6.1.1 of Satellite which was released on 12 August, 2015.