Description of problem: Seems that latest SAM code has compatibility issue if java-1.7.0-openjdk is already installed in the system. After successfully installing katello-headpin-all and running katello-configuration, katello-jobs fails to start Version-Release number of selected component (if applicable): * apr-util-ldap-1.3.9-3.el6_0.1.x86_64 * candlepin-0.8.26-1.el6sam.noarch * candlepin-scl-1-5.el6_4.noarch * candlepin-scl-quartz-2.1.5-5.el6_4.noarch * candlepin-scl-rhino-1.7R3-1.el6_4.noarch * candlepin-scl-runtime-1-5.el6_4.noarch * candlepin-selinux-0.8.26-1.el6sam.noarch * candlepin-tomcat6-0.8.26-1.el6sam.noarch * elasticsearch-0.19.9-8.el6sat.noarch * katello-candlepin-cert-key-pair-1.0-1.noarch * katello-certs-tools-1.4.2-2.el6sat.noarch * katello-cli-1.4.3-11.el6sat.noarch * katello-cli-common-1.4.3-11.el6sat.noarch * katello-common-1.4.3-14.el6sam_splice.noarch * katello-configure-1.4.4-4.el6sat.noarch * katello-glue-candlepin-1.4.3-14.el6sam_splice.noarch * katello-glue-elasticsearch-1.4.3-14.el6sam_splice.noarch * katello-headpin-1.4.3-14.el6sam_splice.noarch * katello-headpin-all-1.4.3-14.el6sam_splice.noarch * Katello-Katello-Sanity-ImportKeys-1.2-1.noarch * katello-selinux-1.4.4-2.el6sat.noarch * openldap-2.4.23-32.el6_4.1.x86_64 * openldap-devel-2.4.23-32.el6_4.1.x86_64 * ruby193-rubygem-ldap_fluff-0.2.2-1.el6sat.noarch * ruby193-rubygem-net-ldap-0.3.1-2.el6sat.noarch * thumbslug-0.0.34-1.el6sam.noarch * thumbslug-selinux-0.0.34-1.el6sam.noarch How reproducible: Steps to Reproduce: 1. Provision a new RHEL system 2. Install java-1.7.0-openjdk 3. Install and configure katello-headpin-all Actual results: katello-jobs fails to start Expected results: Additional info: [root@ibm-x3550m3-11 ~]# yum install -y java-1.7.0-openjdk [root@ibm-x3550m3-11 ~]# rpm -qa | grep openjdk java-1.7.0-openjdk-1.7.0.40-2.4.2.1.el6.x86_64 [root@ibm-x3550m3-11 ~]# katello-service status tomcat6 (pid 19716) is running...[ OK ] httpd (pid 19834) is running... thumbslug (pid 19883) is running... elasticsearch (pid 19926) is running... katello (20077) is running. katello (20096) is running. katello (20115) is running. katello (20134) is running. katello (20153) is running. katello (20172) is running. katello (20191) is running. katello (20210) is running. katello (20237) is running. katello (20264) is running. katello (20291) is running. katello (20318) is running. katello (20350) is running. katello (20379) is running. katello (20407) is running. katello (20435) is running. katello (20464) is running. katello (20495) is running. katello (20518) is running. katello (20548) is running. katello (20583) is running. katello (20621) is running. katello (20673) is running. katello (20700) is running. katello (20736) is running. delayed_job is not running. delayed_job_monitor is not running. [root@ibm-x3550m3-11 ~]# headpin -u admin -p admin ping ------------------------------------------------------------------------------------------------------------------------------------------------ Katello Status Status Service Result Duration Message ------------------------------------------------------------------------------------------------------------------------------------------------ FAIL candlepin FAIL SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: tlsv1 alert internal error candlepin_auth FAIL SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: tlsv1 alert internal error elasticsearch ok 38ms katello_jobs FAIL katello-jobs service not running thumbslug ok 148ms [root@ibm-x3550m3-11 ~]# rpm -qa | grep openjdk java-1.7.0-openjdk-1.7.0.40-2.4.2.1.el6.x86_64 [root@ibm-x3550m3-11 ~]# yum install -y java-1.6.0-openjdk [root@ibm-x3550m3-11 ~]# katello-service stop [root@ibm-x3550m3-11 ~]# katello-configure --deployment=sam --user-pass=admin root@ibm-x3550m3-11 ~]# headpin -u admin -p admin ping Service unavailable or restarting, try later [root@ibm-x3550m3-11 ~]# katello-service status tomcat6 (pid 22970) is running...[ OK ] httpd (pid 23288) is running... thumbslug (pid 23176) is running... elasticsearch (pid 22721) is running... katello is not running. delayed_job is not running. delayed_job_monitor is not running. [root@ibm-x3550m3-11 ~]# rpm -qa | grep openjdk java-1.7.0-openjdk-1.7.0.40-2.4.2.1.el6.x86_64 java-1.6.0-openjdk-1.6.0.0-1.62.1.11.11.90.el6_4.x86_64 [root@ibm-x3550m3-11 ~]# rpm -e java-1.7.0-openjdk.x86_64 [root@ibm-x3550m3-11 ~]# rpm -qa | grep openjdk java-1.6.0-openjdk-1.6.0.0-1.62.1.11.11.90.el6_4.x86_64 [root@ibm-x3550m3-11 ~]# katello-service stop [root@ibm-x3550m3-11 ~]# katello-configure --deployment=sam --user-pass=admin [root@ibm-x3550m3-11 ~]# headpin -u admin -p admin ping ------------------------------------------------------------------------------------------------------------------------------------------------ Katello Status Status Service Result Duration Message ------------------------------------------------------------------------------------------------------------------------------------------------ ok candlepin ok 50ms candlepin_auth ok 33ms elasticsearch ok 27ms katello_jobs ok 34ms thumbslug ok 212ms [root@ibm-x3550m3-11 ~]# katello-service status tomcat6 (pid 25090) is running...[ OK ] httpd (pid 26165) is running... thumbslug (pid 25295) is running... elasticsearch (pid 24843) is running... katello (25482) is running. katello (25501) is running. katello (25520) is running. katello (25539) is running. katello (25558) is running. katello (25577) is running. katello (25596) is running. katello (25615) is running. katello (25642) is running. katello (25669) is running. katello (25696) is running. katello (25723) is running. katello (25750) is running. katello (25785) is running. katello (25804) is running. katello (25840) is running. katello (25860) is running. katello (25891) is running. katello (25912) is running. katello (25949) is running. katello (25976) is running. katello (26006) is running. katello (26051) is running. katello (26080) is running. katello (26121) is running. delayed_job is running. delayed_job_monitor is running.
If anyone can reproduce this, could we get the "security.provider.*" lines from java.security file for the JDK on that system?
Ok Chris Duryee hit this and got in touch with me. His java.security featured: security.provider.1=sun.security.pkcs11.SunPKCS11 ${java.home}/lib/security/nss.cfg Which is the FIPS compliant JCE provider, a new default in some JDK rpms in RHEL. This FIPS compliant provider provides an implementation for several anonymous ECDH ciphers you will see in /etc/tomcat6/server.xml: TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA TLS_ECDH_anon_WITH_AES_128_CBC_SHA TLS_ECDH_anon_WITH_AES_256_CBC_SHA However the stock JDK JCE provider does *not* provide an implementation for these: http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#SUNProvider And critically, these ciphers should not be used and cannot work with Candlepin. See https://github.com/candlepin/candlepin/commit/be8c72995e29912319bb6e622b8e648e3cbab411 for more details. TLDR Candlepin has been shipping config that included some ciphers which cannot work with the application. The problem was hidden because those ciphers are not provided by the default JCE provider. The SunPKCS11 however does implement them, and recent JDK's have been switching to this as the default, exposing the problem. To work around, edit your server.xml and remove the anon ciphers above. Cipher config should be corrected as of candlepin-0.9.5-1
Verified. Steps: 1. Provisioned rhel 6.5 2. Removed older openjdk and installed java-1.7.0-openjdk: yum install -y java-1.7.0-openjdk 3. Install SAM 4. Run katello-configure Result: # rpm -qa | grep openjdk java-1.7.0-openjdk-1.7.0.55-2.4.7.1.el6_5.x86_64 # katello-service status tomcat6 (pid 18909) is running...[ OK ] httpd (pid 19371) is running... thumbslug (pid 19119) is running... elasticsearch (pid 18333) is running... katello (19307) is running. katello (19324) is running. katello (19341) is running. delayed_job is running. delayed_job_monitor is running. Version tested: * apr-util-ldap-1.3.9-3.el6_0.1.x86_64 * candlepin-0.9.6-1.el6_5.noarch * candlepin-scl-1-5.el6_4.noarch * candlepin-scl-quartz-2.1.5-5.el6_4.noarch * candlepin-scl-rhino-1.7R3-1.el6_4.noarch * candlepin-scl-runtime-1-5.el6_4.noarch * candlepin-selinux-0.9.6-1.el6_5.noarch * candlepin-tomcat6-0.9.6-1.el6_5.noarch * elasticsearch-0.19.9-8.el6sat.noarch * katello-candlepin-cert-key-pair-1.0-1.noarch * katello-certs-tools-1.4.2-2.el6sat.noarch * katello-cli-1.4.3.1-1.el6sam.noarch * katello-cli-common-1.4.3.1-1.el6sam.noarch * katello-common-1.4.3.26-1.el6sam_splice.noarch * katello-configure-1.4.5-1.el6sam.noarch * katello-glue-candlepin-1.4.3.26-1.el6sam_splice.noarch * katello-glue-elasticsearch-1.4.3.26-1.el6sam_splice.noarch * katello-headpin-1.4.3.26-1.el6sam_splice.noarch * katello-headpin-all-1.4.3.26-1.el6sam_splice.noarch * katello-selinux-1.4.4-2.el6sat.noarch * openldap-2.4.23-32.el6_4.1.x86_64 * openldap-devel-2.4.23-32.el6_4.1.x86_64 * ruby193-rubygem-ldap_fluff-0.2.2-1.el6sat.noarch * ruby193-rubygem-net-ldap-0.3.1-2.el6sat.noarch * thumbslug-0.0.39-1.el6sam.noarch * thumbslug-selinux-0.0.39-1.el6sam.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-0677.html