Bug 1172261

Summary: [puppet] Cannot create activemq java keystore if fqdn != certname
Product: OpenShift Container Platform Reporter: Scott Dodson <sdodson>
Component: InstallerAssignee: Scott Dodson <sdodson>
Status: CLOSED WONTFIX QA Contact: Ma xiaoqiang <xiama>
Severity: low Docs Contact:
Priority: unspecified    
Version: 2.2.0CC: jokerman, libra-bugs, libra-onpremise-devel, mmccomas, xtian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-12-10 20:34:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Scott Dodson 2014-12-09 17:47:07 UTC
Description of problem:
If the certname is different from the fqdn fact then the activemq java keystore cannot be created using the defaults. This happens when you've set a domain or other items that modify the FQDN during the puppet run.

ie: when generating a cert and signing it with a hostname of 'sdodson-test1.redhat.com' but having domain => 'ose22-test.com.cn' as in the following config.

node 'sdodson-test1.os1.phx2.redhat.com' {
  class { 'openshift_origin':
    roles                           => ["msgserver"],
    msgserver_tls_enabled           => 'enabled',
    msgserver_tls_ca                => '/etc/puppetlabs/puppet/ssl/certs/ca.pem',
    msgserver_tls_cert              => inline_template('<%= "/etc/puppetlabs/puppet/ssl/certs/#{fqdn.downcase}.pem" %>'),
    msgserver_tls_key               => inline_template('<%= "/etc/puppetlabs/puppet/ssl/private_keys/#{fqdn.downcase}.pem" %>'),
    domain                          => 'ose22-test.com.cn',
    bind_key                        => '4q9vJH2UPhtwHfQJzl8zu8XXtCYzExBMevU5SLG7VuGsLRi4hk8AuPVozb2fx3BJ4y1ejFZvpPJk1rtaOGEmqg==',
    install_method                  => 'none',
    node_ip_addr                    => '10.66.79.131',
    nameserver_ip_addr              => '10.66.79.136',
    ose_version                     => '2.2',
    conf_nameserver_upstream_dns    => ['10.66.127.17'],
#    register_host_with_nameserver   => true,
  }
}

Info: Applying configuration version '1418146838'
Error: /Stage[main]/Openshift_origin::Activemq_keystores/File[/etc/activemq/ssl_credentials/activemq_certificate.pem]: Could not evaluate: Could not retrieve information from environment production source(s) file:/etc/puppetlabs/puppet/ssl/certs/sdodson-test1.ose22-test.com.cn.pem
Error: /Stage[main]/Openshift_origin::Activemq_keystores/File[/etc/activemq/ssl_credentials/activemq_private.pem]: Could not evaluate: Could not retrieve information from environment production source(s) file:/etc/puppetlabs/puppet/ssl/private_keys/sdodson-test1.ose22-test.com.cn.pem

Comment 1 Scott Dodson 2014-12-09 17:51:40 UTC
The defaults are really a bit of convenience and the user should ensure that the SSL CA, Cert, and Key exist prior to running puppet.

Workaround : ensure that your certname matches the ultimate FQDN for this host and regenerate your puppet SSL certs.

[root@sdodson-test1 ~]# rm -rf /etc/puppetlabs/puppet/ssl/
[root@sdodson-test1 ~]# vi /etc/puppetlabs/puppet/puppet.conf 
[root@sdodson-test1 ~]# puppet agent -t
Info: Creating a new SSL key for sdodson-test1.ose22-test.com.cn
Info: Caching certificate for ca
Info: csr_attributes file loading from /etc/puppetlabs/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for sdodson-test1.ose22-test.com.cn
Info: Certificate Request fingerprint (SHA256): 1A:6A:9C:D0:3E:71:80:94:B5:31:1C:F5:71:23:F2:C4:04:10:51:D5:A8:13:67:B7:38:03:E0:36:82:F1:33:DB
Info: Caching certificate for ca

** sign the cert on your master, then re-run puppet **

Comment 3 Ma xiaoqiang 2014-12-10 04:53:48 UTC
I didn't meet this error. After installation, I can find the three keys files
[root@msg ssl_credentials]# pwd
/etc/activemq/ssl_credentials
[root@msg ssl_credentials]# ls
activemq_certificate.pem  activemq_private.pem  ca.pem
[root@msg ssl_credentials]#

but can not find keystore files
[root@msg activemq]# pwd
/etc/activemq
[root@msg activemq]# ls
activemq.xml  broker-localhost.cert  client.ks  credentials-enc.properties  groups.properties       jetty.xml   jmx.password      logging.properties  ssl_credentials   wrapper.conf
broker.ks     broker.ts              client.ts  credentials.properties      jetty-realm.properties  jmx.access  log4j.properties  login.config        users.properties
[root@msg activemq]# 

log file:
http://ci.qe.openshift.nay.redhat.com/view/OpenShift_Enterprise/job/Puppet_Install/172/consoleFull

Comment 4 Scott Dodson 2014-12-10 17:05:09 UTC
I'm completely baffled, your logs show it creating the files, but it doesn't attempt to create the keystores. I'm grasping at straws here, but can you try this on your puppet master :

puppet module upgrade puppetlabs-java_ks  

I've tested with version 1.2.4 and 1.2.6 and can't reproduce this.

Relevant logs from your jenkins job that show Openshift_origin::Activemq_keystores class being called but not creating keystore and truststores :

Notice: /Stage[main]/Openshift_origin::Update_conf_files/File[/etc/resolv.conf]/content: content changed '{md5}4932df750095b3e044ac7e1661cdc4aa' to '{md5}add67da47552676c820c73ea523eb75e'
Notice: /Stage[main]/Openshift_origin::Msgserver/Package[activemq]/ensure: created
Notice: /Stage[main]/Openshift_origin::Msgserver/Package[activemq-client]/ensure: created
Notice: /Stage[main]/Openshift_origin::Msgserver/File[/var/run/activemq/]/mode: mode changed '0755' to '0750'
Notice: /Stage[main]/Openshift_origin::Activemq_keystores/File[/etc/activemq/ssl_credentials]/ensure: created
Notice: /Stage[main]/Openshift_origin::Activemq_keystores/File[/etc/activemq/ssl_credentials/activemq_certificate.pem]/ensure: defined content as '{md5}a6b18f41ecc962da5b286b9c005aa6e4'
Notice: /Stage[main]/Openshift_origin::Activemq_keystores/File[/etc/activemq/ssl_credentials/activemq_private.pem]/ensure: defined content as '{md5}80492a6c6c03a0eae01d4de604151eae'
Notice: /Stage[main]/Openshift_origin::Activemq_keystores/File[/etc/activemq/ssl_credentials/ca.pem]/ensure: defined content as '{md5}e03744707a7309f6435dfd3f496b5a48'
Notice: /Stage[main]/Openshift_origin::Msgserver/File[activemq.xml config]/content:

Comment 5 Scott Dodson 2014-12-10 20:34:27 UTC
I believe comment 4 is due to pluginsync timeouts. 

I can't think of any solution to fqdn != certname issue so I'm gong to close wontfix this. I was hoping there was a fact/variable avaiable within puppet that stored the certname and we could use that rather than fqdn but that doesn't appear to be the case. For now we'll simply have to rely on the user specifying the proper path to CA, Cert, and Key if the defaults aren't sufficient.

Comment 6 Scott Dodson 2014-12-10 21:06:48 UTC
https://tickets.puppetlabs.com/browse/PUP-1581 would be useful should they implement that.