Bug 1258501 - bkr returning SSL: CERTIFICATE_VERIFY_FAILED
Product: Beaker
Classification: Community
Component: command line (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified (vote)
: ---
: ---
Assigned To: beaker-dev-list
Depends On:
  Show dependency treegraph
Reported: 2015-08-31 09:58 EDT by Jaroslav Škarvada
Modified: 2015-08-31 21:36 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-08-31 21:36:22 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jaroslav Škarvada 2015-08-31 09:58:37 EDT
Description of problem:
bkr command returns SSL: CERTIFICATE_VERIFY_FAILED. It worked few weeks ago, but suddenly stopped working.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. bkr task-list

Actual results:
XML-RPC connection to beaker.engineering.redhat.com failed: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590), 5 retries left

Expected results:
No error

Additional info:
Certificate (/etc/beaker/RedHatInternalCA.pem) is provided by beaker-redhat package. This certificate seems OK:
    Not Before: Sep 16 18:45:25 2009 GMT
    Not After : Sep 14 18:45:25 2019 GMT
My beaker-redhat package seems to be a bit outdated, but unfortunately it seems there isn't newer package in the repo (http://download.lab.bos.redhat.com/beakerrepos/client/Fedora$releasever/)

There is CA_CERT set in my /etc/beaker/client.conf:
CA_CERT = "/etc/beaker/RedHatInternalCA.pem"

But there is none CA_CERT set in the ~/.beaker-client/config. From the code (/usr/lib/python2.7/site-packages/bkr/__init__.py):

if not config_file:
    user_conf = os.path.expanduser('~/.beaker_client/config')
    old_conf = os.path.expanduser('~/.beaker')
    if os.path.exists(user_conf):
        config_file = user_conf
    elif os.path.exists(old_conf):
        config_file = old_conf
        sys.stderr.write("%s is deprecated for config, please use %s instead\n" % (old_conf, user_conf))
    elif os.path.exists('/etc/beaker/client.conf'):
        config_file = "/etc/beaker/client.conf"

It seems it uses ~/.beaker_client/config and ignores /etc/beaker/client.conf, thus the CA_CERT settings is not in effect. By removing ~/.beaker_client/config or adding CA_CERT to ~/.beaker_client/config it started working again.

I haven't changed anything. It's my default configuration created by beaker tools. Also I haven't added or removed RedHatInternalCA.pem from default CA bundle, so if it was initially there, some update had to remove it.

# rpm -qV beaker-client
# rpm -qV beaker-redhat

So the packages seems OK and it should work out of the box without tweaking.
Comment 1 Jaroslav Škarvada 2015-08-31 10:01:58 EDT
I am running Fedora 22 x86_64.
Comment 2 Jaroslav Škarvada 2015-08-31 10:05:46 EDT
It could be also change of behaviour of httplib:
Comment 3 Dan Callaghan 2015-08-31 21:36:22 EDT
Yes, the change you are seeing is due to new SSL cert checking backported to Python 2.7.10.

(In reply to Jaroslav Škarvada from comment #0)
> It seems it uses ~/.beaker_client/config and ignores
> /etc/beaker/client.conf, thus the CA_CERT settings is not in effect. By
> removing ~/.beaker_client/config or adding CA_CERT to
> ~/.beaker_client/config it started working again.

Setting CA_CERT in ~/.beaker_client/config is the right solution, for now. beaker-redhat configures everything system-wide but it cannot touch your user config in your home directory of course.

The real bug is that the CA_CERT setting in /etc/beaker/client.conf has no effect when ~/.beaker_client/config exists. That is covered by bug 844364.

Note You need to log in before you can comment on or make changes to this bug.