Description of problem: Unable to create a cert by nova x509-create-cert Version-Release number of selected component (if applicable): rpm -qa |grep nova python-nova-2012.2.4-4.el6ost.noarch openstack-nova-compute-2012.2.4-4.el6ost.noarch openstack-nova-network-2012.2.4-4.el6ost.noarch openstack-nova-volume-2012.2.4-4.el6ost.noarch openstack-nova-objectstore-2012.2.4-4.el6ost.noarch openstack-nova-cert-2012.2.4-4.el6ost.noarch python-novaclient-2.10.0-11.el6ost.noarch openstack-nova-api-2012.2.4-4.el6ost.noarch openstack-nova-common-2012.2.4-4.el6ost.noarch openstack-nova-scheduler-2012.2.4-4.el6ost.noarch openstack-nova-console-2012.2.4-4.el6ost.noarch How reproducible: Steps to Reproduce: 1. nova x509-create-cert 2. 3. Actual results: ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-48298254-7eca-4eb4-b60a-514214794eb0) Expected results: [root@other-stack-0 ~(keystone_admin)]# nova x509-create-cert Wrote private key to pk.pem Wrote x509 certificate to cert.pem [root@other-stack-0 ~(keystone_admin)]# ls -l pk.pem cert.pem -rw-r--r--. 1 root root 2547 May 6 03:38 cert.pem -r--------. 1 root root 887 May 6 03:38 pk.pem Additional info: tail -n 50 /var/log/nova/cert.log 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd)) 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command. 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp Command: openssl ca -batch -out /tmp/tmpErThJD/outbound.csr -config ./openssl.cnf -infiles /tmp/tmpErThJD/inbound.csr 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp Exit code: 1 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp Stdout: '' 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp Stderr: "Using configuration from ./openssl.cnf\n./index.txt: No such file or directory\nunable to open './index.txt'\n140722827712328:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen('./index.txt','r')\n140722827712328:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:\n" 2013-05-06 01:22:51 1380 TRACE nova.openstack.common.rpc.amqp 2013-05-06 01:22:51 1380 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.34.69.214:5672 2013-05-06 01:22:51 1380 ERROR nova.openstack.common.rpc.common [-] Returning exception Unexpected error while running command. Command: openssl ca -batch -out /tmp/tmpErThJD/outbound.csr -config ./openssl.cnf -infiles /tmp/tmpErThJD/inbound.csr Exit code: 1 Stdout: '' Stderr: "Using configuration from ./openssl.cnf\n./index.txt: No such file or directory\nunable to open './index.txt'\n140722827712328:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen('./index.txt','r')\n140722827712328:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:\n" to caller 2013-05-06 01:22:51 1380 ERROR nova.openstack.common.rpc.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/cert/manager.py", line 58, in generate_x509_cert\n return crypto.generate_x509_cert(user_id, project_id)\n', ' File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 243, in generate_x509_cert\n (serial, signed_csr) = sign_csr(csr, project_id)\n', ' File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 287, in sign_csr\n return _sign_csr(csr_text, ca_folder())\n', ' File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 308, in _sign_csr\n \'./openssl.cnf\', \'-infiles\', inbound)\n', ' File "/usr/lib/python2.6/site-packages/nova/utils.py", line 210, in execute\n cmd=\' \'.join(cmd))\n', 'ProcessExecutionError: Unexpected error while running command.\nCommand: openssl ca -batch -out /tmp/tmpErThJD/outbound.csr -config ./openssl.cnf -infiles /tmp/tmpErThJD/inbound.csr\nExit code: 1\nStdout: \'\'\nStderr: "Using configuration from ./openssl.cnf\\n./index.txt: No such file or directory\\nunable to open \'./index.txt\'\\n140722827712328:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen(\'./index.txt\',\'r\')\\n140722827712328:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:\\n"\n'] 2013-05-06 01:27:11 1380 INFO nova.service [-] Caught SIGTERM, exiting 2013-05-06 01:27:11 1380 CRITICAL nova [-] need more than 0 values to unpack 2013-05-06 01:35:26 1528 AUDIT nova.service [-] Starting cert node (version 2012.2.4-4.el6ost) 2013-05-06 01:35:26 1528 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.34.69.214:5672 2013-05-06 01:35:28 1528 INFO nova.service [-] Caught SIGTERM, exiting 2013-05-06 01:35:28 1548 AUDIT nova.service [-] Starting cert node (version 2012.2.4-4.el6ost) 2013-05-06 01:35:28 1548 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.34.69.214:5672 2013-05-06 01:35:56 1548 ERROR nova.openstack.common.rpc.amqp [-] Exception during message handling 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/cert/manager.py", line 58, in generate_x509_cert 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp return crypto.generate_x509_cert(user_id, project_id) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 243, in generate_x509_cert 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp (serial, signed_csr) = sign_csr(csr, project_id) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 287, in sign_csr 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp return _sign_csr(csr_text, ca_folder()) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 308, in _sign_csr 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp './openssl.cnf', '-infiles', inbound) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/nova/utils.py", line 210, in execute 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp cmd=' '.join(cmd)) 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp ProcessExecutionError: Unexpected error while running command. 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp Command: openssl ca -batch -out /tmp/tmpwcRI8v/outbound.csr -config ./openssl.cnf -infiles /tmp/tmpwcRI8v/inbound.csr 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp Exit code: 1 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp Stdout: '' 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp Stderr: "Using configuration from ./openssl.cnf\n./index.txt: No such file or directory\nunable to open './index.txt'\n139731457177416:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen('./index.txt','r')\n139731457177416:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:\n" 2013-05-06 01:35:56 1548 TRACE nova.openstack.common.rpc.amqp 2013-05-06 01:35:56 1548 INFO nova.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.34.69.214:5672 2013-05-06 01:35:56 1548 ERROR nova.openstack.common.rpc.common [-] Returning exception Unexpected error while running command. Command: openssl ca -batch -out /tmp/tmpwcRI8v/outbound.csr -config ./openssl.cnf -infiles /tmp/tmpwcRI8v/inbound.csr Exit code: 1 Stdout: '' Stderr: "Using configuration from ./openssl.cnf\n./index.txt: No such file or directory\nunable to open './index.txt'\n139731457177416:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen('./index.txt','r')\n139731457177416:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:\n" to caller 2013-05-06 01:35:56 1548 ERROR nova.openstack.common.rpc.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/cert/manager.py", line 58, in generate_x509_cert\n return crypto.generate_x509_cert(user_id, project_id)\n', ' File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 243, in generate_x509_cert\n (serial, signed_csr) = sign_csr(csr, project_id)\n', ' File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 287, in sign_csr\n return _sign_csr(csr_text, ca_folder())\n', ' File "/usr/lib/python2.6/site-packages/nova/crypto.py", line 308, in _sign_csr\n \'./openssl.cnf\', \'-infiles\', inbound)\n', ' File "/usr/lib/python2.6/site-packages/nova/utils.py", line 210, in execute\n cmd=\' \'.join(cmd))\n', 'ProcessExecutionError: Unexpected error while running command.\nCommand: openssl ca -batch -out /tmp/tmpwcRI8v/outbound.csr -config ./openssl.cnf -infiles /tmp/tmpwcRI8v/inbound.csr\nExit code: 1\nStdout: \'\'\nStderr: "Using configuration from ./openssl.cnf\\n./index.txt: No such file or directory\\nunable to open \'./index.txt\'\\n139731457177416:error:02001002:system library:fopen:No such file or directory:bss_file.c:355:fopen(\'./index.txt\',\'r\')\\n139731457177416:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:357:\\n"\n']
I was unable to reproduce the bug locally, but some code review leads me to believe that we may be passing a local rather than absolute path to openssl. Upstream bug https://bugs.launchpad.net/nova/+bug/1185052 I have a patch in review, which might fix this.
Possible fix upstream, in Havana: http://github.com/openstack/nova/commit/2cd1783bd5604937d5fbfa23ebd87b9009e95138 backport review for rh-grizzly-rhel-6-patches: https://code.engineering.redhat.com/gerrit/8181 The fix should be very safe, but I'm not sure if it really fixes the bug, since I can't reproduce it.
Attila, please retest and let me know if it's fixed now.
It is fixed now. (openstack-nova-cert-2013.1.2-4.el6ost.noarch) /var/lib/nova/CA/index.txt was there as empty file, before the first x509-create-cert request.
Thanks Attila for confirming that it's fixed. Putting in POST.
verified: # nova x509-create-cert Wrote private key to pk.pem Wrote x509 certificate to cert.pem # ls -l pk.pem cert.pem -rw-r--r--. 1 root root 2551 Nov 26 22:56 cert.pem -r--------. 1 root root 887 Nov 26 22:56 pk.pem # rpm -qa |grep openstack-nova openstack-nova-api-2013.2-5.el6ost.noarch openstack-nova-console-2013.2-5.el6ost.noarch openstack-nova-compute-2013.2-5.el6ost.noarch openstack-nova-scheduler-2013.2-5.el6ost.noarch openstack-nova-common-2013.2-5.el6ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2013-1859.html
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days