Hide Forgot
2016-09-14 14:44:28.447 29392 ERROR nova.virt.libvirt.guest [req-ee39700f-b5a6-4aed-9bb6-37ab09256b24 c7e08bf99b1f46f587f849c8e7603b8e cd166989ce974c6181b5b456ecadde65 - - -] Error launching a defined domain with XML: <domain type='kvm'> <name>instance-0000000b</name> <uuid>8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="14.0.0-0.20160823074012.21312bf.el7ost"/> <nova:name>vm1</nova:name> <nova:creationTime>2016-09-14 14:44:26</nova:creationTime> <nova:flavor name="m1.tiny"> <nova:memory>512</nova:memory> <nova:disk>1</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>1</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="c7e08bf99b1f46f587f849c8e7603b8e">master</nova:user> <nova:project uuid="cd166989ce974c6181b5b456ecadde65">master</nova:project> </nova:owner> <nova:root type="image" uuid="eb717993-317b-4853-8139-65bf9ba939d5"/> </nova:instance> </metadata> <memory unit='KiB'>524288</memory> <currentMemory unit='KiB'>524288</currentMemory> <vcpu placement='static'>1</vcpu> <cputune> <shares>1024</shares> </cputune> <sysinfo type='smbios'> <system> <entry name='manufacturer'>Red Hat</entry> <entry name='product'>OpenStack Compute</entry> <entry name='version'>14.0.0-0.20160823074012.21312bf.el7ost</entry> <entry name='serial'>1a713901-c563-4931-9910-0b380166f1dc</entry> <entry name='uuid'>8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.3.0'>hvm</type> <boot dev='hd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='1' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='network' device='disk'> <driver name='qemu' type='raw' cache='writeback' discard='unmap'/> <auth username='openstack'> <secret type='ceph' uuid='74c2353a-7a1a-11e6-90da-5254003ec994'/> </auth> <source protocol='rbd' name='vms/8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6_disk'> <host name='10.19.95.15' port='6789'/> <host name='10.19.95.19' port='6789'/> <host name='10.19.95.26' port='6789'/> </source> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='usb' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'/> <interface type='bridge'> <mac address='fa:16:3e:35:a1:a7'/> <source bridge='qbr4db8b2f0-96'/> <target dev='tap4db8b2f0-96'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='file'> <source path='/var/lib/nova/instances/8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6/console.log'/> <target port='0'/> </serial> <serial type='pty'> <target port='1'/> </serial> <console type='file'> <source path='/var/lib/nova/instances/8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6/console.log'/> <target type='serial' port='0'/> </console> <input type='tablet' bus='usb'> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0' keymap='en-us'> <listen type='address' address='0.0.0.0'/> </graphics> <video> <model type='cirrus' vram='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <stats period='10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> </domain> 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [req-ee39700f-b5a6-4aed-9bb6-37ab09256b24 c7e08bf99b1f46f587f849c8e7603b8e cd166989ce974c6181b5b456ecadde65 - - -] [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] Instance failed to spawn 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] Traceback (most recent call last): 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2075, in _build_resources 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] yield resources 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1919, in _build_and_run_instance 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] block_device_info=block_device_info) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2583, in spawn 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] post_xml_callback=gen_confdrive) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4812, in _create_domain_and_network 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] post_xml_callback=post_xml_callback) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4741, in _create_domain 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] guest.launch(pause=pause) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 144, in launch 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] self._encoded_xml, errors='ignore') 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] self.force_reraise() 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] six.reraise(self.type_, self.value, self.tb) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 139, in launch 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] return self._domain.createWithFlags(flags) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 183, in doit 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] result = proxy_call(self._autowrap, f, *args, **kwargs) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 141, in proxy_call 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] rv = execute(f, *args, **kwargs) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 122, in execute 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] six.reraise(c, e, tb) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 80, in tworker rhel-osp-director: Unable to launch instance Environment: openstack-nova-compute-14.0.0-0.20160823074012.21312bf.el7ost.noarch openstack-puppet-modules-9.0.0-0.20160802183056.8c758d6.el7ost.noarch instack-undercloud-5.0.0-0.20160818065636.41ef775.el7ost.noarch openstack-tripleo-heat-templates-5.0.0-0.20160823140311.72404b.1.el7ost.noarch Steps to reproduce: 1. Deploy overcloud with: openstack overcloud deploy --templates --control-scale 3 --control-flavor control --compute-scale 2 --compute-flavor compute --ceph-storage-scale 3 --ceph-storage-flavor ceph-storage --neutron-network-type vxlan --neutron-tunnel-types vxlan -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e network-environment.yaml --ntp-server clock.redhat.com -e ceph-layout.yaml 2. Populate the overcloud with networks, image for vm. 3. Attempt to launch a vm. Result. The launched VM ends up with status Error. Looking in /var/log/nova/nova-compute.log, I see this error: 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] rv = meth(*args, **kwargs) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1065, in createWithFlags 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] libvirtError: internal error: qemu unexpectedly closed the monitor: -vnc 0.0.0.0:0 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] 2016-09-14T14:44:28.230977Z qemu-kvm: -drive file=rbd:vms/8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6_disk:id=openstack:auth_supported=cephx\;none:mon_host=10.19.95.15\:6789\;10.19.95.19\:6789\;10.19.95.26\:6789,password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=writeback,discard=unmap: Block format 'raw' does not support the option 'password-secret' 2016-09-14 14:44:28.448 29392 ERROR nova.compute.manager [instance: 8ea0ec40-cdf0-453b-99ed-3c5950bcfbc6] Expected result: Successfully launched instance.
No info on livirt version is shown, but I'm guessing the reporter is using an outdated build from 7.3, as this issue is fixed in libvirt-2.0.0-6.el7 as per https://bugzilla.redhat.com/show_bug.cgi?id=1182074 comment 24
Was able to launch instance after upgrading the libvirt* on compute to libvirt-2.0.0-9
This is fixed in libvirt as sasha and danpb noted. We're going to carry the newer libvirt for our beta. Since this is fixed in RHEL, I'm going to mark it a duplicate. *** This bug has been marked as a duplicate of bug 1182074 ***