openstack-nova: unable to attach cinder volume to instance: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1 Environment: libvirt-daemon-driver-storage-rbd-3.2.0-14.el7.x86_64 openstack-nova-scheduler-16.0.0-0.20170805120344.5971dde.el7ost.noarch libvirt-daemon-driver-nwfilter-3.2.0-14.el7.x86_64 libvirt-daemon-driver-lxc-3.2.0-14.el7.x86_64 libvirt-daemon-driver-qemu-3.2.0-14.el7.x86_64 libvirt-daemon-driver-storage-iscsi-3.2.0-14.el7.x86_64 python-nova-16.0.0-0.20170805120344.5971dde.el7ost.noarch openstack-nova-console-16.0.0-0.20170805120344.5971dde.el7ost.noarch libvirt-libs-3.2.0-14.el7.x86_64 libvirt-daemon-driver-interface-3.2.0-14.el7.x86_64 libvirt-client-3.2.0-14.el7.x86_64 libvirt-daemon-driver-nodedev-3.2.0-14.el7.x86_64 puppet-nova-11.3.0-0.20170805105252.30a205c.el7ost.noarch libvirt-daemon-driver-storage-core-3.2.0-14.el7.x86_64 libvirt-daemon-driver-storage-logical-3.2.0-14.el7.x86_64 libvirt-daemon-driver-storage-disk-3.2.0-14.el7.x86_64 openstack-nova-conductor-16.0.0-0.20170805120344.5971dde.el7ost.noarch openstack-nova-novncproxy-16.0.0-0.20170805120344.5971dde.el7ost.noarch libvirt-daemon-driver-network-3.2.0-14.el7.x86_64 libvirt-python-3.2.0-3.el7.x86_64 libvirt-daemon-config-nwfilter-3.2.0-14.el7.x86_64 libvirt-daemon-driver-storage-gluster-3.2.0-14.el7.x86_64 libvirt-daemon-driver-storage-3.2.0-14.el7.x86_64 openstack-nova-compute-16.0.0-0.20170805120344.5971dde.el7ost.noarch openstack-nova-migration-16.0.0-0.20170805120344.5971dde.el7ost.noarch openstack-nova-api-16.0.0-0.20170805120344.5971dde.el7ost.noarch libvirt-3.2.0-14.el7.x86_64 libvirt-daemon-driver-storage-scsi-3.2.0-14.el7.x86_64 libvirt-daemon-kvm-3.2.0-14.el7.x86_64 openstack-nova-common-16.0.0-0.20170805120344.5971dde.el7ost.noarch libvirt-daemon-3.2.0-14.el7.x86_64 libvirt-daemon-driver-secret-3.2.0-14.el7.x86_64 python-novaclient-9.1.0-0.20170804194758.0a53d19.el7ost.noarch libvirt-daemon-driver-storage-mpath-3.2.0-14.el7.x86_64 openstack-nova-placement-api-16.0.0-0.20170805120344.5971dde.el7ost.noarch libvirt-daemon-config-network-3.2.0-14.el7.x86_64 openstack-tripleo-heat-templates-7.0.0-0.20170805163045.el7ost.noarch instack-undercloud-7.2.1-0.20170729010705.el7ost.noarch openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch Steps to reproduce: 1. Deploy OC 2. Launch instance (tried with cirros and rhel image). 3. Create cinder volume 4. Try to attach the cinder volume to the instance Result: Although no error is shown in the console: (overcloud) [stack@undercloud-0 ~]$ nova volume-attach rhel 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d | | serverId | 281edd5a-83de-4b4f-a301-329658ecf4be | | volumeId | 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d | Checking the instance - there's no /dev/vdb. Looking in nova-compute.log: [root@overcloud-compute-0 ~]# grep -i error /var/log/containers/nova/nova-compute.log 2017-08-10 19:31:22.214 1 ERROR nova.virt.libvirt.imagebackend [req-3530c749-84d1-4b13-8b74-80155fd5978f efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] /var/lib/nova/instances/_base/c62e48c870a2f6983224417418c841eca33de8ff virtual size 10737418240 larger than flavor root disk size 5368709120 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [req-3530c749-84d1-4b13-8b74-80155fd5978f efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] Instance failed to spawn: FlavorDiskSmallerThanImage: Flavor's disk is too small for requested image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes. 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] Traceback (most recent call last): 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] yield resources 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _build_and_run_instance 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] block_device_info=block_device_info) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2793, in spawn 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] block_device_info=block_device_info) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3231, in _create_image 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] fallback_from_host) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3322, in _create_and_inject_local_root 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] instance, size, fallback_from_host) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6968, in _try_fetch_image_cache 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] size=size) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 241, in cache 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] *args, **kwargs) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 893, in create_image 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] self.verify_base_size(base, size) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 297, in verify_base_size 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] flavor_size=size, image_size=base_size) 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] FlavorDiskSmallerThanImage: Flavor's disk is too small for requested image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes. 2017-08-10 19:31:22.216 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [req-3530c749-84d1-4b13-8b74-80155fd5978f efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] Build of instance feae9666-9b8d-4d07-ab05-781ffca090e4 aborted: Flavor's disk is too small for requested image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes.: BuildAbortException: Build of instance feae9666-9b8d-4d07-ab05-781ffca090e4 aborted: Flavor's disk is too small for requested image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes. 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] Traceback (most recent call last): 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1789, in _do_build_and_run_instance 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] filter_properties) 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2011, in _build_and_run_instance 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] reason=e.format_message()) 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] BuildAbortException: Build of instance feae9666-9b8d-4d07-ab05-781ffca090e4 aborted: Flavor's disk is too small for requested image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes. 2017-08-10 19:31:24.441 1 ERROR nova.compute.manager [instance: feae9666-9b8d-4d07-ab05-781ffca090e4] 2017-08-10 19:34:43.812 1 ERROR nova.virt.libvirt.imagebackend [req-a58a8e5f-4aa6-44a3-a529-360d8ece8939 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] /var/lib/nova/instances/_base/c62e48c870a2f6983224417418c841eca33de8ff virtual size 10737418240 larger than flavor root disk size 6442450944 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [req-a58a8e5f-4aa6-44a3-a529-360d8ece8939 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] Instance failed to spawn: FlavorDiskSmallerThanImage: Flavor's disk is too small for requested image. Flavor disk is 6442450944 bytes, image is 10737418240 bytes. 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] Traceback (most recent call last): 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2125, in _build_resources 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] yield resources 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1940, in _build_and_run_instance 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] block_device_info=block_device_info) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2793, in spawn 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] block_device_info=block_device_info) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3231, in _create_image 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] fallback_from_host) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3322, in _create_and_inject_local_root 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] instance, size, fallback_from_host) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6968, in _try_fetch_image_cache 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] size=size) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 241, in cache 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] *args, **kwargs) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 893, in create_image 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] self.verify_base_size(base, size) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 297, in verify_base_size 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] flavor_size=size, image_size=base_size) 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] FlavorDiskSmallerThanImage: Flavor's disk is too small for requested image. Flavor disk is 6442450944 bytes, image is 10737418240 bytes. 2017-08-10 19:34:43.814 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [req-a58a8e5f-4aa6-44a3-a529-360d8ece8939 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] Build of instance b8903e8f-68ae-4cf7-9a28-7f2245de60d0 aborted: Flavor's disk is too small for requested image. Flavor disk is 6442450944 bytes, image is 10737418240 bytes.: BuildAbortException: Build of instance b8903e8f-68ae-4cf7-9a28-7f2245de60d0 aborted: Flavor's disk is too small for requested image. Flavor disk is 6442450944 bytes, image is 10737418240 bytes. 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] Traceback (most recent call last): 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1789, in _do_build_and_run_instance 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] filter_properties) 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2011, in _build_and_run_instance 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] reason=e.format_message()) 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] BuildAbortException: Build of instance b8903e8f-68ae-4cf7-9a28-7f2245de60d0 aborted: Flavor's disk is too small for requested image. Flavor disk is 6442450944 bytes, image is 10737418240 bytes. 2017-08-10 19:34:45.034 1 ERROR nova.compute.manager [instance: b8903e8f-68ae-4cf7-9a28-7f2245de60d0] 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [req-a4455de3-c161-4ce0-8891-5aee8950d169 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Failed to attach volume at mountpoint: /dev/vdb: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Traceback (most recent call last): 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = execute(f, *args, **kwargs) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(c, e, tb) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = meth(*args, **kwargs) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [req-a4455de3-c161-4ce0-8891-5aee8950d169 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Driver failed to attach volume 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d at /dev/vdb: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Traceback (most recent call last): 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 389, in attach 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] device_type=self['device_type'], encryption=encryption) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1266, in attach_volume 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._disconnect_volume(connection_info, disk_dev, instance) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self.force_reraise() 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = execute(f, *args, **kwargs) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(c, e, tb) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = meth(*args, **kwargs) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.308 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] 2017-08-10 19:38:13.310 1 WARNING nova.volume.cinder [req-a4455de3-c161-4ce0-8891-5aee8950d169 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3.: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [req-a4455de3-c161-4ce0-8891-5aee8950d169 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Failed to attach 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d at /dev/vdb: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Traceback (most recent call last): 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4767, in _attach_volume 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] do_driver_attach=True) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 45, in wrapped 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] ret_val = method(obj, context, *args, **kwargs) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 398, in attach 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] connector) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self.force_reraise() 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 389, in attach 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] device_type=self['device_type'], encryption=encryption) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1266, in attach_volume 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._disconnect_volume(connection_info, disk_dev, instance) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self.force_reraise() 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = execute(f, *args, **kwargs) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(c, e, tb) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = meth(*args, **kwargs) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:13.839 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] 2017-08-10 19:38:13.841 1 WARNING nova.volume.cinder [req-a4455de3-c161-4ce0-8891-5aee8950d169 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3.: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server [req-a4455de3-c161-4ce0-8891-5aee8950d169 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] Exception during message handling: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 211, in decorated_function 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 199, in decorated_function 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4752, in attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server do_attach_volume(context, instance, driver_bdm) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4750, in do_attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server bdm.destroy() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4747, in do_attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server return self._attach_volume(context, instance, driver_bdm) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4781, in _attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server volume_id=bdm.volume_id) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4767, in _attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server do_driver_attach=True) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 45, in wrapped 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server ret_val = method(obj, context, *args, **kwargs) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 398, in attach 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server connector) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 389, in attach 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server device_type=self['device_type'], encryption=encryption) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1266, in attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self._disconnect_volume(connection_info, disk_dev, instance) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:38:14.220 1 ERROR oslo_messaging.rpc.server 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [req-8dd1e465-c2f6-49ba-b496-099536fc6149 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Failed to attach volume at mountpoint: /dev/vdb: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Traceback (most recent call last): 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = execute(f, *args, **kwargs) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(c, e, tb) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = meth(*args, **kwargs) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.303 1 ERROR nova.virt.libvirt.driver [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [req-8dd1e465-c2f6-49ba-b496-099536fc6149 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Driver failed to attach volume 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d at /dev/vdb: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Traceback (most recent call last): 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 389, in attach 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] device_type=self['device_type'], encryption=encryption) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1266, in attach_volume 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._disconnect_volume(connection_info, disk_dev, instance) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self.force_reraise() 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = execute(f, *args, **kwargs) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(c, e, tb) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = meth(*args, **kwargs) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.304 1 ERROR nova.virt.block_device [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] 2017-08-10 19:51:17.305 1 WARNING nova.volume.cinder [req-8dd1e465-c2f6-49ba-b496-099536fc6149 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3.: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [req-8dd1e465-c2f6-49ba-b496-099536fc6149 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Failed to attach 2cf5dbe2-bbc3-42c8-b7ff-ea95c3dadc3d at /dev/vdb: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] Traceback (most recent call last): 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4767, in _attach_volume 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] do_driver_attach=True) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 45, in wrapped 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] ret_val = method(obj, context, *args, **kwargs) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 398, in attach 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] connector) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self.force_reraise() 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 389, in attach 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] device_type=self['device_type'], encryption=encryption) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1266, in attach_volume 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._disconnect_volume(connection_info, disk_dev, instance) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self.force_reraise() 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = execute(f, *args, **kwargs) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] six.reraise(c, e, tb) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] rv = meth(*args, **kwargs) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.766 1 ERROR nova.compute.manager [instance: 281edd5a-83de-4b4f-a301-329658ecf4be] 2017-08-10 19:51:17.768 1 WARNING nova.volume.cinder [req-8dd1e465-c2f6-49ba-b496-099536fc6149 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3.: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server [req-8dd1e465-c2f6-49ba-b496-099536fc6149 efa8cb8e2d9e438784edc732713d54f5 74b09ed902344e229f620c1dde332906 - default default] Exception during message handling: libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 160, in _process_incoming 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 213, in dispatch 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _do_dispatch 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 211, in decorated_function 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 199, in decorated_function 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4752, in attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server do_attach_volume(context, instance, driver_bdm) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4750, in do_attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server bdm.destroy() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4747, in do_attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server return self._attach_volume(context, instance, driver_bdm) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4781, in _attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server volume_id=bdm.volume_id) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4767, in _attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server do_driver_attach=True) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 45, in wrapped 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server ret_val = method(obj, context, *args, **kwargs) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 398, in attach 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server connector) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 389, in attach 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server device_type=self['device_type'], encryption=encryption) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1266, in attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self._disconnect_volume(connection_info, disk_dev, instance) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1250, in attach_volume 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server guest.attach_device(conf, persistent=True, live=live) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 308, in attach_device 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/libvirt.py", line 564, in attachDeviceFlags 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1' 2017-08-10 19:51:17.897 1 ERROR oslo_messaging.rpc.server
Created attachment 1311907 [details] /var/log/containers/nova/nova-compute.log
This seems to be trying to store a 10GiB image into a 5GiB disk: FlavorDiskSmallerThanImage: Flavor's disk is too small for requested image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes.
(In reply to Gorka Eguileor from comment #2) > This seems to be trying to store a 10GiB image into a 5GiB disk: > > FlavorDiskSmallerThanImage: Flavor's disk is too small for requested > image. Flavor disk is 5368709120 bytes, image is 10737418240 bytes. We saw that error initially, but then updated the flavor to 12G and still failed. Beforehand, we tried with cirros image (very small and used to work in previous releases)- also failed.
Just want to clarify: this is not about fitting images into disks at boot. That happened earlier so it will be in the logs but we got past that. What this bug is about is: Attaching an existing volume to an already running instance
Created attachment 1312211 [details] /var/log/containers/nova/nova-compute.log with debug
Was able to attach the volume on deployment with internal ceph. Environment: instack-undercloud-7.2.1-0.20170729010706.el7ost.noarch openstack-tripleo-heat-templates-7.0.0-0.20170805163048.el7ost.noarch openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch ceph-ansible-3.0.0-0.1.rc3.el7cp.noarch ceph-mon-10.2.7-28.el7cp.x86_64 ceph-osd-10.2.7-28.el7cp.x86_64 libcephfs1-10.2.7-28.el7cp.x86_64 puppet-ceph-2.3.1-0.20170805094345.868e6d6.el7ost.noarch ceph-selinux-10.2.7-28.el7cp.x86_64 python-cephfs-10.2.7-28.el7cp.x86_64 ceph-base-10.2.7-28.el7cp.x86_64 ceph-radosgw-10.2.7-28.el7cp.x86_64 ceph-mds-10.2.7-28.el7cp.x86_64 ceph-common-10.2.7-28.el7cp.x86_64 2017-08-18 15:12:14.752 1 INFO nova.compute.manager [req-97485f97-c26e-4cfa-b62c-fe98068c9887 4ea007d1ef484f7c935667ccaea99d74 502bd4c746314367b1b31dde42d8034b - default default] [instance: 1fd7fcdc-4bb6-415f-878e-1b8d1e644d85] Attaching volume f9df1e73-e6e5-4015-b244-d3b0906cb61e to /dev/vdb 2017-08-18 15:12:14.821 1 WARNING nova.volume.cinder [req-97485f97-c26e-4cfa-b62c-fe98068c9887 4ea007d1ef484f7c935667ccaea99d74 502bd4c746314367b1b31dde42d8034b - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3. 2017-08-18 15:12:15.873 1 INFO oslo.privsep.daemon [req-97485f97-c26e-4cfa-b62c-fe98068c9887 4ea007d1ef484f7c935667ccaea99d74 502bd4c746314367b1b31dde42d8034b - default default] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova.conf', '--config-file', '/etc/nova/rootwrap.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmpNRLmpo/privsep.sock'] 2017-08-18 15:12:16.466 1 INFO oslo.privsep.daemon [req-97485f97-c26e-4cfa-b62c-fe98068c9887 4ea007d1ef484f7c935667ccaea99d74 502bd4c746314367b1b31dde42d8034b - default default] Spawned new privsep daemon via rootwrap 2017-08-18 15:12:16.407 1056 INFO oslo.privsep.daemon [-] privsep daemon starting 2017-08-18 15:12:16.411 1056 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 2017-08-18 15:12:16.413 1056 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none 2017-08-18 15:12:16.413 1056 INFO oslo.privsep.daemon [-] privsep daemon running as pid 1056 2017-08-18 15:12:16.646 1 WARNING nova.volume.cinder [req-97485f97-c26e-4cfa-b62c-fe98068c9887 4ea007d1ef484f7c935667ccaea99d74 502bd4c746314367b1b31dde42d8034b - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3. 2017-08-18 15:12:18.346 1 WARNING nova.volume.cinder [req-97485f97-c26e-4cfa-b62c-fe98068c9887 4ea007d1ef484f7c935667ccaea99d74 502bd4c746314367b1b31dde42d8034b - default default] The support for the Cinder API v2 is deprecated, please upgrade to Cinder API v3. 2017-08-18 15:13:05.747 1 INFO nova.compute.resource_tracker [req-95fb4fe9-fd48-41be-abdb-29fbf98d29ad - - - - -] Final resource view: name=overcloud-compute-1.localdomain phys_ram=6143MB used_ram=4608MB phys_disk=118GB used_disk=1GB total_vcpus=4 used_vcpus=1 pci_stats=[] 2017-08-18 15:14:05.738 1 INFO nova.compute.resource_tracker [req-95fb4fe9-fd48-41be-abdb-29fbf98d29ad - - - - -] Final resource view: name=overcloud-compute-1.localdomain phys_ram=6143MB used_ram=4608MB phys_disk=118GB used_disk=1GB total_vcpus=4 used_vcpus=1 pci_stats=[] 2017-08-18 15:15:07.769 1 INFO nova.compute.resource_tracker [req-95fb4fe9-fd48-41be-abdb-29fbf98d29ad - - - - -] Final resource view: name=overcloud-compute-1.localdomain phys_ram=6143MB used_ram=4608MB phys_disk=118GB used_disk=1GB total_vcpus=4 used_vcpus=1 pci_stats=[] 2017-08-18 15:16:09.733 1 INFO nova.compute.resource_tracker [req-95fb4fe9-fd48-41be-abdb-29fbf98d29ad - - - - -] Final resource view: name=overcloud-compute-1.localdomain phys_ram=6143MB used_ram=4608MB phys_disk=118GB used_disk=1GB total_vcpus=4 used_vcpus=1 pci_stats=[] [stack@undercloud-0 ~]$ ssh cirros.0.194 "sudo /sbin/fdisk -l" cirros.0.194's password: Disk /dev/vda: 1073 MB, 1073741824 bytes 255 heads, 63 sectors/track, 130 cylinders, total 2097152 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/vda1 * 16065 2088449 1036192+ 83 Linux Disk /dev/vdb: 1073 MB, 1073741824 bytes 16 heads, 63 sectors/track, 2080 cylinders, total 2097152 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/vdb doesn't contain a valid partition table
I believe this could be due to a mismatch in the volume features supported by the OSP containers (previously using Ceph libs from baseos, 0.94) with the version of Ceph running in the ceph containers (10.2). Could you try, after creating a cinder volume, to run the following from one of the controller nodes: # rbd ls volumes # rbd info -p volumes $name_of_an_image_from_previous_command The output should tell us which features are set for the image. Ideally we should repeat the same with older OSP container images and see if the enabled features are different.
Works for me now.
(In reply to Alexander Chuzhoy from comment #10) > Works for me now. Closing this out as NOTABUG given this feedback, feel free to provide more detail and/or reopen this bug if we are able to reproduce this.