Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1479578

Summary:	openstack-nova: unable to launch an instance "libvirtError: Secret not found: no secret with matching uuid '1c50eddf-56fb-470f-a72d-47 a960f520a0'"
Product:	Red Hat OpenStack	Reporter:	Alexander Chuzhoy <sasha>
Component:	openstack-tripleo-heat-templates	Assignee:	Giulio Fidente <gfidente>
Status:	CLOSED ERRATA	QA Contact:	Alexander Chuzhoy <sasha>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	12.0 (Pike)	CC:	berrange, dasmith, eglynn, gfidente, hbrock, jschluet, kchamart, lyarwood, mburns, mcornea, mprivozn, owalsh, rhel-osp-director-maint, sbauza, sclewis, sferdjao, sgordon, srevivo, tvignaud, vromanso
Target Milestone:	beta	Keywords:	AutomationBlocker, Triaged
Target Release:	12.0 (Pike)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-7.0.0-0.20170805163046.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-12-13 21:51:30 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2017-08-08 21:00:34 UTC

openstack-niova: unable to launch an instance "libvirtError: Secret not found: no secret with matching uuid '1c50eddf-56fb-470f-a72d-47 a960f520a0'"

Environment:
Undercloud:
openstack-tripleo-heat-templates-7.0.0-0.20170805163045.el7ost.noarch
instack-undercloud-7.2.1-0.20170729010705.el7ost.noarch
openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch

on compute node:
libvirt-daemon-driver-storage-rbd-3.2.0-14.el7.x86_64
openstack-nova-scheduler-16.0.0-0.20170805120344.5971dde.el7ost.noarch
libvirt-daemon-driver-nwfilter-3.2.0-14.el7.x86_64
libvirt-daemon-driver-lxc-3.2.0-14.el7.x86_64
libvirt-daemon-driver-qemu-3.2.0-14.el7.x86_64
libvirt-daemon-driver-storage-iscsi-3.2.0-14.el7.x86_64
python-nova-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-console-16.0.0-0.20170805120344.5971dde.el7ost.noarch
libvirt-libs-3.2.0-14.el7.x86_64
libvirt-daemon-driver-interface-3.2.0-14.el7.x86_64
libvirt-client-3.2.0-14.el7.x86_64
libvirt-daemon-driver-nodedev-3.2.0-14.el7.x86_64
puppet-nova-11.3.0-0.20170805105252.30a205c.el7ost.noarch
libvirt-daemon-driver-storage-core-3.2.0-14.el7.x86_64
libvirt-daemon-driver-storage-logical-3.2.0-14.el7.x86_64
libvirt-daemon-driver-storage-disk-3.2.0-14.el7.x86_64
openstack-nova-conductor-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-novncproxy-16.0.0-0.20170805120344.5971dde.el7ost.noarch
libvirt-daemon-driver-network-3.2.0-14.el7.x86_64
libvirt-python-3.2.0-3.el7.x86_64
libvirt-daemon-config-nwfilter-3.2.0-14.el7.x86_64
libvirt-daemon-driver-storage-gluster-3.2.0-14.el7.x86_64
libvirt-daemon-driver-storage-3.2.0-14.el7.x86_64
openstack-nova-compute-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-migration-16.0.0-0.20170805120344.5971dde.el7ost.noarch
openstack-nova-api-16.0.0-0.20170805120344.5971dde.el7ost.noarch
libvirt-3.2.0-14.el7.x86_64
libvirt-daemon-driver-storage-scsi-3.2.0-14.el7.x86_64
libvirt-daemon-kvm-3.2.0-14.el7.x86_64
openstack-nova-common-16.0.0-0.20170805120344.5971dde.el7ost.noarch
libvirt-daemon-3.2.0-14.el7.x86_64
libvirt-daemon-driver-secret-3.2.0-14.el7.x86_64
python-novaclient-9.1.0-0.20170804194758.0a53d19.el7ost.noarch
libvirt-daemon-driver-storage-mpath-3.2.0-14.el7.x86_64
openstack-nova-placement-api-16.0.0-0.20170805120344.5971dde.el7ost.noarch
libvirt-daemon-config-network-3.2.0-14.el7.x86_64


Steps to reproduce:
1. Deploy OC (with external ceph)
2. Try to launch an instance.


Result:
The instance gets status "ERROR"
+--------------------------------------+------+--------+------------+-------------+----------+
| ID                                   | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------+--------+------------+-------------+----------+
| bb2518e8-0961-4495-84b4-a8d081f5e753 | bb   | ERROR  | -          | NOSTATE     |          |
+--------------------------------------+------+--------+------------+-------------+----------+


Checking /var/log/containers/nova/nova-compute.log for errors:

2017-08-08 20:42:25.805 1 ERROR nova.virt.libvirt.guest [req-676cd827-8a51-44c3-b0fa-b68dc5f42db9 d9b11d963e0843438d08f71f636bebd8 a87d53801e224cb78396aae057687997 - default default] Error launching a defined domain with XML: <domain type='kvm'>
2017-08-08 20:42:25.806 1 ERROR nova.virt.libvirt.driver [req-676cd827-8a51-44c3-b0fa-b68dc5f42db9 d9b11d963e0843438d08f71f636bebd8 a87d53801e224cb78396aae057687997 - default default] [instance: bb2518e8-0961-4495-84b4-a8d081f5e753] Failed to start libvirt guest: libvirtError: Secret not found: no secret with matching uuid '1c50eddf-56fb-470f-a72d-47a960f520a0'
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [req-676cd827-8a51-44c3-b0fa-b68dc5f42db9 d9b11d963e0843438d08f71f636bebd8 a87d53801e224cb78396aae057687997 - default default] [instance: bb2518e8-0961-4495-84b4-a8d081f5e753] Instance failed to spawn: libvirtError: Secret not found: no secret with matching uuid '1c50eddf-56fb-470f-a72d-47a960f520a0'
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753] Traceback (most recent call last):
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     yield resources
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _build_and_run_instance
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     block_device_info=block_device_info)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2805, in spawn
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     destroy_disks_on_failure=True)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5274, in _create_domain_and_network
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     destroy_disks_on_failure)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     self.force_reraise()
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     six.reraise(self.type_, self.value, self.tb)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5244, in _create_domain_and_network
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     post_xml_callback=post_xml_callback)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5162, in _create_domain
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     guest.launch(pause=pause)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     self._encoded_xml, errors='ignore')
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     self.force_reraise()
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     six.reraise(self.type_, self.value, self.tb)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     return self._domain.createWithFlags(flags)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     rv = execute(f, *args, **kwargs)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     six.reraise(c, e, tb)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     rv = meth(*args, **kwargs)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1069, in createWithFlags
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753] libvirtError: Secret not found: no secret with matching uuid '1c50eddf-56fb-470f-a72d-47a960f520a0'
2017-08-08 20:42:27.024 1 ERROR nova.compute.manager [instance: bb2518e8-0961-4495-84b4-a8d081f5e753]

Comment 4 Alexander Chuzhoy 2017-08-18 15:47:27 UTC

Verified

Environment:
openstack-tripleo-heat-templates-7.0.0-0.20170805163048.el7ost.noarch

The reported issue doesn't reproduce.

Comment 8 errata-xmlrpc 2017-12-13 21:51:30 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462