Bug 826426 - Puppet can't connect to puppetserver (certificate failure)
Summary: Puppet can't connect to puppetserver (certificate failure)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: puppet
Version: 17
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jeroen van Meeuwen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-05-30 07:43 UTC by Pablo Iranzo Gómez
Modified: 2013-03-19 21:30 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-03-19 21:30:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Pablo Iranzo Gómez 2012-05-30 07:43:39 UTC
Description of problem:
Hi,
After upgrade to F17, puppet is no longer able to connect to puppetmaster but, other puppet clients (running F16, EL6, etc can)

Version-Release number of selected component (if applicable):

puppet-2.7.13-1.fc17.noarch

How reproducible:
puppetd -vt  --server=yourpuppetserver


Steps to Reproduce:
1.puppetd -vt  --server=yourpuppetserver
2.puppet will fail
3.
  
Actual results:

on Client:
/usr/share/rubygems/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead.
err: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client
Exiting; failed to retrieve certificate and waitforcert is disabled


On server:
[2012-05-30 09:42:30] ERROR OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1 alert unknown ca
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `accept'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `listen'
        /usr/lib/ruby/1.8/webrick/server.rb:173:in `call'
        /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
        /usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
        /usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
        /usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `synchronize'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:127:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:142:in `start'
        /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:124:in `start'
        /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:114:in `main'
        /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:46:in `run_command'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:305:in `run'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:420:in `hook'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:305:in `run'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:411:in `exit_on_fail'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:305:in `run'
        /usr/sbin/puppetmasterd:4


Expected results:
Puppet should work :)

Comment 1 Pablo Iranzo Gómez 2012-05-30 07:47:35 UTC
Of course, I've tried deleting and recreating certificates on both client and server and it failed.

I've also tried to setup puppet on a new machine (fully wiped) and same problem.

On F16, EL6 or EL5 it works as expected even when recreating certificates for both server and client

Comment 2 Clayton Louden 2012-05-30 10:49:04 UTC
Same here :(

Comment 3 Todd Zullinger 2012-05-30 17:49:45 UTC
This is a known issue.  You cannot generally use a newer puppet client to talk to an older puppet master.  This is an unfortunate compatibility issue upstream.

Our hands were tied with Fedora 17 as it moved to ruby-1.9 and puppet-2.6 has far too many failures with ruby-1.9 to be usable.  Moving all the other releases to puppet-2.7 also has significant costs and would burden folks who are not running Fedora 17.  So it's a bit of a lose/lose situation for us as maintainers.

See bug #822014 for a request to update EPEL for puppet-2.7.

Comment 4 Pablo Iranzo Gómez 2012-05-30 18:31:33 UTC
Todd, I knew of that problem, but I even tried upgrading my server puppet to the one provided by the yum repo at puppetlabs which his 'newer' than the one provided by Fedora

puppet-2.7.14-1.el6.noarch
puppet-server-2.7.14-1.el6.noarch
at baseurl=http://yum.puppetlabs.com/el/$releasever/products/$basearch/


And had that problem too happens:

master:
==> /var/log/puppet/masterhttp.log <==
[2012-05-30 20:24:44] ERROR OpenSSL::SSL::SSLError: SSL_accept returned=1 errno=0 state=SSLv3 read client certificate A: tlsv1 alert unknown ca
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `accept'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:44:in `listen'
        /usr/lib/ruby/1.8/webrick/server.rb:173:in `call'
        /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
        /usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
        /usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
        /usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
        /usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:42:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `initialize'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `new'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:41:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `synchronize'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:38:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:126:in `listen'
        /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:141:in `start'
        /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:124:in `start'
        /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:202:in `main'
        /usr/lib/ruby/site_ruby/1.8/puppet/application/master.rb:144:in `run_command'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:416:in `hook'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:407:in `exit_on_fail'
        /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:309:in `run'
        /usr/sbin/puppetmasterd:4



And client (F17):
[root@x201 ~]# puppetd -vt --server=merak.no-ip.org
/usr/share/rubygems/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead.
err: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client
Exiting; failed to retrieve certificate and waitforcert is disabled


If its all because of ruby, this is a serious problem... and as F17 had to move to ruby 1.9.. it's a bigger one.

I remember having several versions of 'unison$VERSIONUMBER" because of some incompatibilities between them... ¿would it be possible to have a ruby1.X required by puppet26" or something like that?

Thanks
Pablo

Comment 5 Clayton Louden 2012-05-31 11:34:37 UTC
It seem like ruby 1.9 is completely unsopported by puppet:

http://docs.puppetlabs.com/guides/faq.html#which-versions-of-ruby-does-puppet-support

Comment 6 Clayton Louden 2012-05-31 14:59:36 UTC
Just fyi, 

F16 client to F17 master

[root@PuppetClient ~]# puppet agent --server=PuppetMaster.example.com --no-daemonize --verbose
info: Creating a new SSL key for puppetclient.example.com
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for ca
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Creating a new SSL certificate request for puppetclient.example.com
info: Certificate Request fingerprint (md5): 6E:9D:6A:44:BD:E0:FF:3F:FB:BC:4D:4F:AF:94:A0:10
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session

After signing the certificate on the master:

[root@PuppetClient ~]# puppet agent --server=PuppetMaster.example.com --no-daemonize --verbose
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for puppetclient.example.com
notice: Starting Puppet client version 2.6.16
info: Caching certificate_revocation_list for ca
err: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
notice: Using cached catalog
err: Could not retrieve catalog; skipping run



F17 client to F16 master and F17 client to F17 master

[root@PuppetClient ~]# puppet agent --server=PuppetMaster.example.com --no-daemonize --verbose
/usr/share/rubygems/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead.
info: Creating a new SSL key for puppetclient.example.com
err: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed.  This is often because the time is out of sync on the server or client

Certificate request never shows up on the master (puppet cert --list is empty)


version being 2.6.16 for F16 and 2.7.13 for F17 respectively.

Comment 7 Clayton Louden 2012-06-04 08:49:48 UTC
Based on http://projects.puppetlabs.com/issues/9084, here's a workaround:

- Copy /var/lib/puppet/ssl/certs/ca.pem from master to client.
- Create the following symlink on the client:

ln -s /var/lib/puppet/ssl/certs/ca.pem $(openssl version -d|cut -d\" -f2)/certs/$(openssl x509 -hash -noout -in /var/lib/puppet/ssl/certs/ca.pem).0

Comment 8 John Florian 2012-08-11 20:00:20 UTC
(In reply to comment #7)
> Based on http://projects.puppetlabs.com/issues/9084, here's a workaround:
> 
> - Copy /var/lib/puppet/ssl/certs/ca.pem from master to client.
> - Create the following symlink on the client:
> 
> ln -s /var/lib/puppet/ssl/certs/ca.pem $(openssl version -d|cut -d\"
> -f2)/certs/$(openssl x509 -hash -noout -in
> /var/lib/puppet/ssl/certs/ca.pem).0


This work around was very helpful in the early days of Fedora 17, but it no longer seems necessary.  I'm not sure what exactly changed, but I just successfully brought up a new F17 node (against an F17 master) and the process was same as the old days once again.  Yeah!

Comment 9 Clayton Louden 2012-09-03 00:59:50 UTC
(In reply to comment #8)
> This work around was very helpful in the early days of Fedora 17, but it no
> longer seems necessary.  I'm not sure what exactly changed, but I just
> successfully brought up a new F17 node (against an F17 master) and the
> process was same as the old days once again.  Yeah!

Here too! Working fine again with the latest puppet version from updates repo.


Note You need to log in before you can comment on or make changes to this bug.