Bug 760280

Summary: katello-configure fails with ssl key creation error
Product: Red Hat Satellite Reporter: Mike McCune <mmccune>
Component: InfrastructureAssignee: Tomas Lestach <tlestach>
Status: CLOSED CURRENTRELEASE QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.0.0CC: cperry, kbidarka, lzap
Target Milestone: UnspecifiedKeywords: Regression, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-22 18:10:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 747354    
Attachments:
Description Flags
install log for failed attempt none

Description Mike McCune 2011-12-05 18:30:23 UTC
Fresh install of Katello on EL6 results in this error from katello-configure:

[root@dhcp77-228]# katello-configure 
Starting Katello configuration
The top-level log file is [/var/log/katello/katello-configure-20111205-130038/main.log]
err: /Stage[main]/Certs::Config/Exec[generate-ssl-keystore]/returns: change from notrun to 0 failed: openssl pkcs12 -export -in /usr/share/katello/KATELLO-TRUSTED-SSL-CERT -inkey /root/ssl-build/KATELLO-PRIVATE-SSL-KEY -out /etc/pki/katello/keystore -name tomcat -CAfile /usr/share/katello/KATELLO-TRUSTED-SSL-CERT -caname root -chain -passin "file:/etc/katello/ssl_ca_password-file" -password "file:/etc/katello/keystore_password-file" returned 1 instead of one of [0] at /usr/share/katello/install/puppet/modules/certs/manifests/config.pp:102


looking a bit deeper by running manually:

[root@dhcp77-228 yum.repos.d]#  openssl pkcs12 -export -in /usr/share/katello/KATELLO-TRUSTED-SSL-CERT -inkey /root/ssl-build/KATELLO-PRIVATE-SSL-KEY -out /etc/pki/katello/keystore -name tomcat -CAfile /usr/share/katello/KATELLO-TRUSTED-SSL-CERT -caname root -chain -passin "file:/etc/katello/ssl_ca_password-file" -password "file:/etc/katello/keystore_password-file" 
Error opening private key /root/ssl-build/KATELLO-PRIVATE-SSL-KEY
139922836670280:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen('/root/ssl-build/KATELLO-PRIVATE-SSL-KEY','r')
139922836670280:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:
unable to load private key

Comment 1 Mike McCune 2011-12-05 18:31:06 UTC
# rpm -qa |grep katello | sort
katello-0.1.123-1.el6.x86_64
katello-all-0.1.123-1.el6.x86_64
katello-certs-tools-1.0.1-1.el6.noarch
katello-cli-0.1.20-1.el6.noarch
katello-cli-common-0.1.20-1.el6.noarch
katello-common-0.1.123-1.el6.noarch
katello-configure-0.1.24-1.el6.noarch
katello-glue-candlepin-0.1.123-1.el6.x86_64
katello-glue-foreman-0.1.123-1.el6.x86_64
katello-glue-pulp-0.1.123-1.el6.x86_64
katello-httpd-ssl-key-pair-1.0-1.noarch
katello-qpid-broker-key-pair-1.0-1.noarch
katello-repos-0.1.4-1.el6.noarch
katello-trusted-ssl-cert-1.0-1.noarch

Comment 2 Clifford Perry 2011-12-05 18:34:19 UTC
Tomas, 
Lukas, 

Looks like SSL commit has at least one bug - can we investigate and resolve within the next 2 days. Hopefully it is a quick / easy fix. 

Cliff

Comment 3 Mike McCune 2011-12-05 19:23:22 UTC
quick workaround:

katello-configure --skip-ssl-ca-generation=True

Comment 4 Clifford Perry 2011-12-05 20:49:55 UTC
My test install completed without error. 

[root@dhcp77-206 ~]# katello-configure --help
Usage: /usr/sbin/katello-configure [options]
        --answer-file=ANSWER_FILE    Path of the answer file
        --user-name=USER_NAME        Katello administrative user (default: admin)
        --user-pass=USER_PASS        Katello user's password (default: admin)
        --user-email=USER_EMAIL      Katello user's email (default: root@localhost)
        --org-name=ORG_NAME          Katello initial Organization (default: ACME_Corporation)
        --proxy-url=PROXY_URL        HTTP Proxy URL (example: http://172.31.1.1)
        --proxy-port=PROXY_PORT      HTTP Proxy port (default: 3128)
        --proxy-user=PROXY_USER      HTTP Proxy user (proxy username, if auth is required)
        --proxy-pass=PROXY_PASS      HTTP Proxy pass (proxy password, if auth is required)
        --db-name=DB_NAME            Katello database name
        --db-user=DB_USER            Katello database user
        --db-password=DB_PASSWORD    Katello database password
        --deployment=DEPLOYMENT      Deployment Type
        --non-interactive=NON_INTERACTIVE
                                     Non-interactive installer mode
        --skip-ssl-ca-generation=SKIP_SSL_CA_GENERATION
                                     skip SSL CA generation
        --ssl-ca-password=SSL_CA_PASSWORD
                                     SSL CA password
        --ssl-ca-country=SSL_CA_COUNTRY
                                     SSL CA country
        --ssl-ca-state=SSL_CA_STATE  SSL CA state
        --ssl-ca-city=SSL_CA_CITY    SSL CA city
        --ssl-ca-org=SSL_CA_ORG      SSL CA organization
        --ssl-ca-org-unit=SSL_CA_ORG_UNIT
                                     SSL CA organization unit
        --ssl-ca-cn=SSL_CA_CN        SSL CA common name
        --ssl-ca-email=SSL_CA_EMAIL  SSL CA e-mail address
        --ssl-cert-expiration=SSL_CERT_EXPIRATION
                                     SSL certificate expiration (in days)
        --ssl-ca-password-file=SSL_CA_PASSWORD_FILE
                                     SSL CA password file path
        --keystore-password-file=KEYSTORE_PASSWORD_FILE
                                     Keystore password file path
        --nss-db-password-file=NSS_DB_PASSWORD_FILE
                                     NSS DB password file path
        --only-show-config           Print the resulting configuration and exit
    -h, --help                       Show this short summary


[root@dhcp77-206 ~]# hostname 
dhcp77-206.rhndev.redhat.com
[root@dhcp77-206 ~]# katello-configure 
Starting Katello configuration
The top-level log file is [/var/log/katello/katello-configure-20111205-152404/main.log]
[root@dhcp77-206 ~]# echo $?
0
[root@dhcp77-206 ~]# rpm -q katello-configure
katello-configure-0.1.24-1.el6.noarch
[root@dhcp77-206 ~]#

Comment 5 Clifford Perry 2011-12-05 20:53:32 UTC
Potentially - this is an ordering issue and we need to be more strict with requires/dep to force puppet order. Mike, can we have the install log to review. I can compare my good install puppet ordering to your bad install puppet ordering.

Cliff

Comment 6 Mike McCune 2011-12-05 21:05:27 UTC
Created attachment 541082 [details]
install log for failed attempt

Attaching the logfile from the install run as posted in the first summary

Comment 7 Tomas Lestach 2011-12-05 22:28:19 UTC
Hey Mike,

katello-configure (within katello-configure-0.1.24-1.el6.noarch) finished also without any issues on my newly installed 64-bit RHEL6.1. I do the installation strictly according to https://fedorahosted.org/katello/wiki/Install

How did you do your installation? Can you reproduce it on another machine?

According to the #Description, CA private key (/root/ssl-build/KATELLO-PRIVATE-SSL-KEY) cannot be read. But according to the posted main.log, that private key will be used also earlier.

Could you check /root/ssl-build/KATELLO-PRIVATE-SSL-KEY - whether you can access the file, its permissions, selinux context, if the content look meaningful, sm. like:

-----BEGIN RSA PRIVATE KEY-----
<base64 encoded key>
-----END RSA PRIVATE KEY-----

?

The dependencies look good to me:
generate-ssl-keystore depends on generate-keystore-password, that on deploy-ssl-ca-certificate, that on generate-ssl-ca-certificate - and that generates apart from the other stuff - the CA private key.

Comment 8 Tomas Lestach 2011-12-05 22:32:37 UTC
What version of openssl do you have installed?
On my machine: openssl-1.0.0-10.el6.x86_64

Comment 9 Kedar Bidarkar 2011-12-06 11:12:13 UTC
I face the same issue even with freshly installed f15 machines.

[root@scalpel dev]# katello-configure 
Starting Katello configuration
The top-level log file is [/var/log/katello/katello-configure-20111206-102153/main.log]
err: /Stage[main]/Certs::Config/Exec[generate-ssl-keystore]/returns: change from notrun to 0 failed: openssl pkcs12 -export -in /usr/share/katello/KATELLO-TRUSTED-SSL-CERT -inkey /root/ssl-build/KATELLO-PRIVATE-SSL-KEY -out /etc/pki/katello/keystore -name tomcat -CAfile /usr/share/katello/KATELLO-TRUSTED-SSL-CERT -caname root -chain -passin "file:/etc/katello/ssl_ca_password-file" -password "file:/etc/katello/keystore_password-file" returned 1 instead of one of [0] at /usr/share/katello/install/puppet/modules/certs/manifests/config.pp:102

[root@scalpel dev]# vi /var/log/katello/katello-configure-20111206-102153/main.log

[root@scalpel dev]# rpm -q katello-configure
katello-configure-0.1.24-1.fc15.noarch

[root@scalpel dev]# rpm -qav | grep -i openssl 
openssl-1.0.0d-1.fc15.x86_64

[root@scalpel dev]# rpm -qav | grep -i katello
katello-cli-0.1.21-1.fc15.noarch
katello-certs-tools-1.0.1-1.fc15.noarch
katello-glue-pulp-0.1.124-1.fc15.x86_64
katello-httpd-ssl-key-pair-1.0-1.noarch
katello-cli-common-0.1.21-1.fc15.noarch
katello-repos-0.1.4-1.fc16.noarch
katello-trusted-ssl-cert-1.0-1.noarch
katello-qpid-broker-key-pair-1.0-1.noarch
katello-configure-0.1.24-1.fc15.noarch
katello-glue-candlepin-0.1.124-1.fc15.x86_64
katello-glue-foreman-0.1.124-1.fc15.x86_64
katello-common-0.1.124-1.fc15.noarch
katello-0.1.124-1.fc15.x86_64
katello-all-0.1.124-1.fc15.x86_64


[root@scalpel ~]# cat /etc/fedora-release 
Fedora release 15 (Lovelock)
[root@scalpel ~]# ll /root/ssl-build/KATELLO-PRIVATE-SSL-KEY
ls: cannot access /root/ssl-build/KATELLO-PRIVATE-SSL-KEY: No such file or directory

Comment 10 Lukas Zapletal 2011-12-06 14:41:24 UTC
I dont think its a ordering issue, since in my case (beaker, F15) I also have the same result as Mike, but from the log I can see the task that should generate that missing file (KATELLO-PRIVATE-SSL-KEY) successfuly executed:

rhn-ssl-tool --gen-ca -p "$(cat /etc/katello/ssl_ca_password-file)" --set-country 'US' --set-state 'North Carolina' --set-city 'Raleigh' --set-org 'Red Hat' --set-org-unit 'Cloud BU' --set-common-name 'Katello machine' --set-email 'root@localhost' --ca-key 'KATELLO-PRIVATE-SSL-KEY' --ca-cert 'KATELLO-TRUSTED-SSL-CERT' --ca-cert-rpm katello-trusted-ssl-cert

I noticed from it's output it is creating all files in the CURRENT directory, so a simple find showed:

# find / -name KATELLO-PRIVATE-SSL-KEY
/mnt/tests/Kalpana/Installation/PuppetSystemTest/ssl-build/KATELLO-PRIVATE-SSL-KEY

My wild guess is it got executed in a different directory for Beaker. I guess we should direct the tool to save everything in the /root/ssl-build folder rather than current pwd.

Comment 11 Lukas Zapletal 2011-12-06 15:02:13 UTC
I change current working dir to /root and check if we are running as root before starting configuration. This won't hurt us for sure.

647dc20 760280 - katello-configure fails with ssl key creation error

Comment 12 Lukas Zapletal 2011-12-06 15:56:01 UTC
Not goot, jlaska just run into the issue again:

http://fpaste.org/d3q3/ 

But it seems this time is hostname issue:

jlaska: the problem seems to be that ssl-build/katello.rdu.redhat.com/ doesn't exist ... but ssl-build/katello/ does

I will fix this one immediately fyi:

https://bugzilla.redhat.com/show_bug.cgi?id=760265

Comment 13 Lukas Zapletal 2011-12-06 22:13:24 UTC
Ok it seems that problem James reported is a different one. And it has been also fixed.

Comment 14 Kedar Bidarkar 2011-12-07 09:48:45 UTC
[root@yyyy ~]# katello-configure 
Starting Katello configuration
The top-level log file is [/var/log/katello/katello-configure-20111207-091236/main.log]
[root@yyyyy ~]# ls
anaconda-ks.cfg  install.log  install.log.syslog  ssl-build
[root@yyyy ~]# cd ssl-build/
[root@yyyy ssl-build]# ls 
index.txt
index.txt.attr
index.txt.attr.old
KATELLO-PRIVATE-SSL-KEY
KATELLO-TRUSTED-SSL-CERT
katello-trusted-ssl-cert-1.0-1.noarch.rpm
katello-trusted-ssl-cert-1.0-1.src.rpm
latest.txt
rhn-ca-openssl.cnf
rhn-ca-openssl.cnf.1
yyyy.redhat.com
serial

This issue is no longer faced.

Comment 15 Kedar Bidarkar 2011-12-07 09:49:50 UTC
[root@yyyy ssl-build]# rpm -qav | grep -i katello 
katello-cli-0.1.22-1.fc15.noarch
katello-glue-candlepin-0.1.128-1.fc15.x86_64
katello-qpid-broker-key-pair-1.0-1.noarch
katello-cli-common-0.1.22-1.fc15.noarch
katello-certs-tools-1.0.1-1.fc15.noarch
katello-repos-0.1.4-1.fc16.noarch
katello-httpd-ssl-key-pair-1.0-1.noarch
katello-trusted-ssl-cert-1.0-1.noarch
katello-configure-0.1.26-1.fc15.noarch
katello-glue-pulp-0.1.128-1.fc15.x86_64
katello-glue-foreman-0.1.128-1.fc15.x86_64
katello-common-0.1.128-1.fc15.noarch
katello-0.1.128-1.fc15.x86_64
katello-all-0.1.128-1.fc15.x86_64