Bug 752414 - Aviary doesn't run over SSL with only one server/cert
Summary: Aviary doesn't run over SSL with only one server/cert
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: condor-aviary
Version: Development
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: 2.1
: ---
Assignee: Pete MacKinnon
QA Contact: MRG Quality Engineering
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-11-09 13:03 UTC by Stanislav Graf
Modified: 2013-11-13 21:28 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-11-09 16:53:13 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 733677 0 high CLOSED Integration of aviary for job control, submission, and job/submission queries [RFE] 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 791197 0 medium CLOSED aviary cannot load valid OpenSSL certificates 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 973047 1 None None None 2021-01-20 06:05:38 UTC

Internal Links: 733677 791197 973047

Description Stanislav Graf 2011-11-09 13:03:11 UTC
Description of problem:
Generate CA certificate and then generate server/client certificates signed by CA ( Bug 746251 ).
Use those certificate for aviary SSL setup and condor_schedd ends with an exception.

11/09/11 13:58:12 (pid:3508) axis2_ssl_utils_initialize_ctx failed
11/09/11 13:58:12 (pid:3508) SSL/TLS requested but configuration failed
11/09/11 13:58:12 (pid:3508) ERROR "Unable to configure AviaryProvider. Exiting..." at line 57 in file /builddir/build/BUILD/condor-7.6.4/src/condor_contrib/aviary/src/AviaryScheddPlugin.cpp
Stack dump for process 3508 at timestamp 1320843492 (11 frames)
condor_schedd(dprintf_dump_stack+0x4a)[0x81a378a]
condor_schedd[0x81d8be6]
[0x2e3420]
/lib/libc.so.6(abort+0x221)[0x87e821]
condor_schedd(_EXCEPT_+0xa6)[0x81b32d6]
/usr/lib/condor/plugins/AviaryScheddPlugin-plugin.so(_ZN6aviary3job18AviaryScheddPlugin15earlyInitializeEv+0x25f)[0x59f76f]
condor_schedd(_ZN19ScheddPluginManager15EarlyInitializeEv+0x33)[0x80bdf73]
condor_schedd(_Z9main_initiPPc+0xb5)[0x8116185]
condor_schedd(main+0x1171)[0x8147971]
/lib/libc.so.6(__libc_start_main+0xdc)[0x869e9c]
condor_schedd[0x80bdb41]


Version-Release number of selected component (if applicable):
condor-7.6.5-0.6
condor-aviary-7.6.5-0.6

How reproducible:
100%

Steps to Reproduce:
[Generate certificates]
Use script from Bug 746251
~]# mrg_gen_ssl_certs.sh

[List all private keys]
~]# certutil -K -d ~/CA_db/

[Export certificates from database]
~]# cd ~/CA_db/
~/CA_db]# pk12util -d . -k passwordfile -o ca.p12     -n "NSS Certificate
DB:CAnick"
~/CA_db]# pk12util -d . -k passwordfile -o client.p12 -n "NSS Certificate
DB:client_..."
~/CA_db]# pk12util -d . -k passwordfile -o serv.p12   -n "NSS Certificate
DB:serv_..."

[Change certificate format]
~/CA_db]# openssl pkcs12 -in client.p12 -out client.pem -nodes
~/CA_db]# openssl pkcs12 -in serv.p12   -out serv.pem   -nodes
~/CA_db]# openssl pkcs12 -in ca.p12     -out ca.pem     -nodes

[Verify certificates]
~/CA_db]# mkdir /tmp/ssl
~/CA_db]# cp *.pem /tmp/ssl
~/CA_db]# cd /tmp/ssl
/tmp/ssl]# openssl verify -CAfile                    ./ca.pem serv.pem client.pem
/tmp/ssl]# openssl verify -purpose sslclient -CAfile ./ca.pem serv.pem client.pem
/tmp/ssl]# openssl verify -purpose sslserver -CAfile ./ca.pem serv.pem client.pem
/tmp/ssl]# openssl verify -purpose any       -CAfile ./ca.pem serv.pem client.pem

/tmp/ssl]# for i in *.pem ; do openssl x509 -noout -in $i -hash; done
5be5959f
5be5959f
5be5959f

/tmp/ssl]# grep CN= *
ca.pem:subject=/CN=CAcert
ca.pem:issuer=/CN=CAcert
client.pem:subject=/CN=CAcert
client.pem:issuer=/CN=CAcert
client.pem:subject=/CN=client_dhcp...
client.pem:issuer=/CN=CAcert
serv.pem:subject=/CN=CAcert
serv.pem:issuer=/CN=CAcert
serv.pem:subject=/CN=serv_dhcp...
serv.pem:issuer=/CN=CAcert

/tmp/ssl]# ls 
ca.pem  client.pem  serv.pem

/tmp/ssl]# condor_config_val -dump | grep SSL
QUERY_SERVER.AVIARY_SSL = True
QUERY_SERVER.AVIARY_SSL_CA_DIR = /tmp/ssl/
QUERY_SERVER.AVIARY_SSL_CA_FILE = /tmp/ssl/ca.pem
QUERY_SERVER.AVIARY_SSL_SERVER_CERT = /tmp/ssl/serv.pem
QUERY_SERVER.AVIARY_SSL_SERVER_KEY = /tmp/ssl/serv.pem
SCHEDD.AVIARY_SSL = True
SCHEDD.AVIARY_SSL_CA_DIR = /tmp/ssl/
SCHEDD.AVIARY_SSL_CA_FILE = /tmp/ssl/ca.pem
SCHEDD.AVIARY_SSL_SERVER_CERT = /tmp/ssl/serv.pem
SCHEDD.AVIARY_SSL_SERVER_KEY = /tmp/ssl/serv.pem

  
Actual results:
condor_schedd created exception

Expected results:
condor_schedd runs

Additional info:

Comment 2 Pete MacKinnon 2011-11-09 16:53:13 UTC
After examining the serv.pem file generated by the QE procedure and reviewing the OpenSSL docs I believe that the problem is that the server's certificate must be the FIRST cert listed after the private key. Thus, serv.pem should appear (be generated) as:

Bag Attributes
    friendlyName: serv_grid1.lab.bos.redhat.com
    localKeyID: 5B D1 2E 09 23 7F 73 03 A1 0B B0 57 43 8D A1 42 47 A4 9A B0 
Key Attributes: <No Attributes>
-----BEGIN PRIVATE KEY-----
MIICdwIBADANBgkqhkiG9w0BAQEFAASCAmEwggJdAgEAAoGBALG25YqqmQI8Ztpr
1E9cAfJWU8p602Gf0v4ezpllbugssJETNsB44uhgKZbRu+kr1Lg9mE1bL3XNKrdm
ueXOcoXf+qZnfRZSCnxZGeHxZgr9sCNB6RMlDO/difPhMhGxQp9zoZ+MraABj+uk
JJDvW9tBlEEbgeP3ljnyf5h5Ji+rAgMBAAECgYB0JWS/y99UZsWR2wdXFnrvNxxs
KPodw6bSDrBX8MMbWpnLNxOBl4A0/yQvPcPiEJNmFluDsyTBiOVyF+FHIDYX0gyE
XGxhpCd3OppUoDhDBNj4kwCeTfZtHFgvIkq5xP1q6TxlIu/TExFFX8vlbSLgU5f/
Oxry1e6yO2t7jJVfgQJBANXSaxyPtC2QXEaxvheteXKEgE9i2AI1bL8Eg8qw2zUs
2mU3BIkwY0JJNwpG+m7J5OX2XT+G1jhMj6S0MlB77jUCQQDUxR8HUBWiv8VnIApB
HXAvrVNP5jwyTje/JaToiy6MlgJpIAY33U8XOrOK8JZByaJCzIpU5B1s2JUb/Dlb
JuJfAkAk0Y/aIjCfa+1HyxF1mEyCWKiTguy5LKPmHIvpGh0VAf01Uoz4Zpmg72SH
44L6Es/UfWC/MKOwnBZcAR9s1npFAkEAgShz8HV93MC67SH295YOLvLxOvRRIFx4
3LDWTU+H31GEfxowjCLsyvYYXUQ0ghULSa9uXZ/n+NX0lftjOeD90wJBAJ/MqeYg
cIDdhsnnz3HmJQLUZe6ejMNWzxZ4KMLQst/h1p0sLg6YGgPhR/r/ultxVwC/97AU
q55O9IbC037MoAg=
-----END PRIVATE KEY-----
Bag Attributes
    friendlyName: serv_grid1.lab.bos.redhat.com
    localKeyID: 5B D1 2E 09 23 7F 73 03 A1 0B B0 57 43 8D A1 42 47 A4 9A B0 
subject=/CN=grid1.lab.bos.redhat.com
issuer=/CN=CAcert
-----BEGIN CERTIFICATE-----
MIIBqTCCARKgAwIBAgICA+kwDQYJKoZIhvcNAQEFBQAwETEPMA0GA1UEAxMGQ0Fj
ZXJ0MB4XDTExMTEwOTE0NDU1MVoXDTIxMTEwOTE0NDU1MVowIzEhMB8GA1UEAxMY
Z3JpZDEubGFiLmJvcy5yZWRoYXQuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCB
iQKBgQCxtuWKqpkCPGbaa9RPXAHyVlPKetNhn9L+Hs6ZZW7oLLCREzbAeOLoYCmW
0bvpK9S4PZhNWy91zSq3ZrnlznKF3/qmZ30WUgp8WRnh8WYK/bAjQekTJQzv3Ynz
4TIRsUKfc6GfjK2gAY/rpCSQ71vbQZRBG4Hj95Y58n+YeSYvqwIDAQABMA0GCSqG
SIb3DQEBBQUAA4GBAECIhCFns4USFLocZ1fgC/v2NXhgt8rTvJboV3uJNsSIFYH/
44vpIBiKEniqTVk/7/125NUPlls+IaDaFBV+5Z71CpfHsp87AdrfMAnre4b0IAvk
ItmuGLigIWbF8uBSoVdDhL4s44c5CblyS57c6/y9YJBNulFN2rSGsCw/BKOK
-----END CERTIFICATE-----
Bag Attributes
    friendlyName: CAnick
subject=/CN=CAcert
issuer=/CN=CAcert
-----BEGIN CERTIFICATE-----
MIIBlzCCAQCgAwIBAgICA+gwDQYJKoZIhvcNAQEFBQAwETEPMA0GA1UEAxMGQ0Fj
ZXJ0MB4XDTExMTEwOTE0NDU1MFoXDTIxMTEwOTE0NDU1MFowETEPMA0GA1UEAxMG
Q0FjZXJ0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDLjaTM7v+QNl9//p9u
GSKpcAgmktW/ywRONlYevUGC/gz+Vrj+tHxBZdVimFdczGQ7oHp75kTNWfyUHOD1
1fSgyLT5B3ZMhpo75049551wy8mtFi0oOmpcld1DEwasWVifAw6U2qfDHiQnh7Ow
UhZbhOYRbpsy2KA75Nv7KTs+6QIDAQABMA0GCSqGSIb3DQEBBQUAA4GBADFwDgnM
OB3XRMhxpFlWuYD/yXLPho7ds2FJF/etM6zBKJCTgTPBgLcmnagr4wbaDeSdsrjg
M1Uxaqqyhoy1CuVC2FSxxobUHx43+rBnTr1QhT6WIzr3tVfdhgibx0EyMgkCJaE6
wKxv+KrR0ty9JBwI/4VpBt+vaGuZ1Zo4gqDV
-----END CERTIFICATE-----

This rearranged PEM file works in my testing.


Note You need to log in before you can comment on or make changes to this bug.