Bug 2083485 - [RHOS 16.2][TLS] Failed to launch a nova instance with "Cannot load certificate ..."
Summary: [RHOS 16.2][TLS] Failed to launch a nova instance with "Cannot load certifica...
Keywords:
Status: CLOSED DUPLICATE of bug 2079767
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 16.2 (Train)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Adriano Petrich
QA Contact: David Rosenfeld
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-10 08:30 UTC by lkuchlan
Modified: 2022-05-18 09:50 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-18 09:50:36 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker OSP-15101 0 None None None 2022-05-10 08:45:25 UTC

Description lkuchlan 2022-05-10 08:30:37 UTC
Description of problem:
Creating a nova instance in TLS environment is failed with
"Cannot load certificate '/etc/pki/libvirt-vnc/server-cert.pem' & key '/etc/pki/libvirt-vnc/serv
er-key.pem': Error while reading file"

Version-Release number of selected component (if applicable):
RHOS-16.2-RHEL-8-20220427.n.3
openstack-tripleo-common-containers-11.7.1-2.20220318011205.b5ef9a5.el8ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy TLS environment
2. Try to create a nova instance

Actual results:
Instance creation is failed with:
Cannot load certificate '/etc/pki/libvirt-vnc/server-cert.pem' & key '/etc/pki/libvirt-vnc/serv
er-key.pem': Error while reading file

Expected results:
Creating an instance should be successful.

Additional info:

From compute.log
=================
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [req-e80d2b13-1ac8-480a-b36c-60d1c21ed379 6a47f16d7eb84c32906f7dfbca7de9fd 485a8c5de5314bc087cd68ec1d96cf50 - default default] [instance: d0f97193-7400-42f
c-83c9-01c0995d7daa] Failed to build and run instance: libvirt.libvirtError: internal error: process exited while connecting to monitor: 2022-05-09T13:20:55.121551Z qemu-kvm: -object tls-creds-x509,id=vnc-tls
-creds0,dir=/etc/pki/libvirt-vnc,endpoint=server,verify-peer=yes: Cannot load certificate '/etc/pki/libvirt-vnc/server-cert.pem' & key '/etc/pki/libvirt-vnc/server-key.pem': Error while reading file.
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa] Traceback (most recent call last):
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 2485, in _build_and_run_instance
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     block_device_info=block_device_info)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3780, in spawn
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     cleanup_instance_disks=created_disks)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6683, in _create_domain_and_ne
twork
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     cleanup_instance_disks=cleanup_instance_disks)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     self.force_reraise()
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     six.reraise(self.type_, self.value, self.tb)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     raise value
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6649, in _create_domain_and_ne
twork
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     post_xml_callback=post_xml_callback)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 6578, in _create_domain
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     guest.launch(pause=pause)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/guest.py", line 149, in launch
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     self._encoded_xml, errors='ignore')
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     self.force_reraise()
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     six.reraise(self.type_, self.value, self.tb)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     raise value
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/guest.py", line 144, in launch
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     return self._domain.createWithFlags(flags)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     rv = execute(f, *args, **kwargs)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     six.reraise(c, e, tb)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     raise value
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 83, in tworker
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     rv = meth(*args, **kwargs)
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]   File "/usr/lib64/python3.6/site-packages/libvirt.py", line 1385, in createWithFlags
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]     raise libvirtError('virDomainCreateWithFlags() failed')
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa] libvirt.libvirtError: internal error: process exited while connecting to monitor: 2022-05-09T13:20:55.1215
51Z qemu-kvm: -object tls-creds-x509,id=vnc-tls-creds0,dir=/etc/pki/libvirt-vnc,endpoint=server,verify-peer=yes: Cannot load certificate '/etc/pki/libvirt-vnc/server-cert.pem' & key '/etc/pki/libvirt-vnc/serv
er-key.pem': Error while reading file.
2022-05-09 13:20:56.723 7 ERROR nova.compute.manager [instance: d0f97193-7400-42fc-83c9-01c0995d7daa]

Comment 3 Artom Lifshitz 2022-05-16 14:54:47 UTC
Indeed, can you retry with puppet-tripleo-11.7.0-2.20220405015037.el8ost and report back if the same problem continues?

Comment 4 Bogdan Dobrelya 2022-05-17 14:00:15 UTC
Please let us know if the aforementioned fix helped

Comment 5 lkuchlan 2022-05-18 06:13:55 UTC
(In reply to Artom Lifshitz from comment #3)
> Indeed, can you retry with puppet-tripleo-11.7.0-2.20220405015037.el8ost and
> report back if the same problem continues?

The latest compose (RHOS-16.2-RHEL-8-20220513.n.2) uses puppet-tripleo-11.7.0-2.20220405015037.el8ost.noarch
and according to the CI results the problem was fixed.

Comment 6 Artom Lifshitz 2022-05-18 09:50:36 UTC

*** This bug has been marked as a duplicate of bug 2079767 ***


Note You need to log in before you can comment on or make changes to this bug.