Description of problem: We are seeing the same problem as described in https://bugs.launchpad.net/nova/+bug/1699228 however with already attaching the second volume using the virtio-scsi. 2017-09-22 11:50:42.951 640086 ERROR nova.compute.manager [instance: 79bf1f1f-2bb7-4955-b827-9374d2703c7e] 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server [req-fc6bbe1e-f0e1-489b-a327-100b16344ed3 5c8cf456b70a4de5bf0c890f68be90f3 d2ec148db73c4bfc8181e96b0d05b714 - - -] Exception during message handling 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 75, in wrapped 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/exception_wrapper.py", line 66, in wrapped 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 216, in decorated_function 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 204, in decorated_function 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4675, in attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server do_attach_volume(context, instance, driver_bdm) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 271, in inner 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server return f(*args, **kwargs) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4673, in do_attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server bdm.destroy() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4670, in do_attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server return self._attach_volume(context, instance, driver_bdm) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4693, in _attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.volume_api.unreserve_volume(context, bdm.volume_id) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4685, in _attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server do_check_attach=False, do_driver_attach=True) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 49, in wrapped 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server ret_val = method(obj, context, *args, **kwargs) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 283, in attach 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server connector) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/block_device.py", line 274, in attach 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server device_type=self['device_type'], encryption=encryption) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1166, in attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self._disconnect_volume(connection_info, disk_dev) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self.force_reraise() 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1155, in attach_volume 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server guest.attach_device(conf, persistent=True, live=live) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 294, in attach_device 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server self._domain.attachDeviceFlags(device_xml, flags=flags) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server File "/usr/lib64/python2.7/site-packages/libvirt.py", line 560, in attachDeviceFlags 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainAttachDeviceFlags() failed', dom=self) 2017-09-22 11:50:43.198 640086 ERROR oslo_messaging.rpc.server libvirtError: internal error: unable to execute QEMU command '__com.redhat_drive_add': Duplicate ID 'drive-scsi0-0-0-0' for drive We believe this might be fixed by applying the chain up to https://review.openstack.org/#/c/459741/. This is not yet backported to Newton. Please could you backport the patches to OSP10? Version-Release number of selected component (if applicable): # rpm -qa |grep nova openstack-nova-console-14.0.2-7.el7ost.noarch python-novaclient-6.0.0-1.el7ost.noarch openstack-nova-cert-14.0.2-7.el7ost.noarch python-nova-14.0.2-7.el7ost.noarch openstack-nova-api-14.0.2-7.el7ost.noarch openstack-nova-scheduler-14.0.2-7.el7ost.noarch openstack-nova-conductor-14.0.2-7.el7ost.noarch openstack-nova-compute-14.0.2-7.el7ost.noarch openstack-nova-novncproxy-14.0.2-7.el7ost.noarch openstack-nova-common-14.0.2-7.el7ost.noarch puppet-nova-9.4.0-2.el7ost.noarch How reproducible: Create VM with image using virtio-scsi scsi model and scsi bus: 1. openstack image create --os-image-api-version 1 --public --file CentOS-7-x86_64 --disk-format qcow2 --container-format bare --property os_type=linux --property os_distro=centos-7.2 --property hw_scsi_model=virtio-scsi --property hw_disk_bus=scsi --min-disk 8 CentOS-7-x86_64 2. Attach additional volume to the VM Actual results: This fails with the above libvirtError. Expected results: Being able to attach additional volume. Additional info: This could be most likely fixed by applying chain up to https://review.openstack.org/#/c/459741
It's a valid issue. The chain you have mentioned should fix the issue. I'm going to see whether we could backport it. If not we would have to probably provide a downstream-only fix.
Other Bugzilla bug might be relevant as it fixes the issue in later release https://bugzilla.redhat.com/show_bug.cgi?id=1269577
Verification steps: 1. Create the image and boot a VM [stack@undercloud-0 testingdir]$ openstack image create --os-image-api-version 1 --public --file rhel-guest-image-7.4-latest.x86_64.qcow2 --disk-format qcow2 --container-format bare --property os_type=linux --property os_distro=centos-7.4 --property hw_scsi_model=virtio-scsi --property hw_disk_bus=scsi --min-disk 8 RHEL-7-x86_64 +------------------+------------------------------------------------------------------------------------------+ | Field | Value | +------------------+------------------------------------------------------------------------------------------+ | checksum | 4c762b186f05049da8871f9ee9f47666 | | container_format | bare | | created_at | 2017-11-02T20:34:07.000000 | | deleted | False | | deleted_at | None | | disk_format | qcow2 | | id | 1ff3cdb0-ac6f-4af3-a104-638f44701ee9 | | is_public | True | | min_disk | 8 | | min_ram | 0 | | name | RHEL-7-x86_64 | | owner | 9430d96b274046f7a0b2905343c4ebd0 | | properties | hw_disk_bus='scsi', hw_scsi_model='virtio-scsi', os_distro='centos-7.4', os_type='linux' | | protected | False | | size | 534011392 | | status | active | | updated_at | 2017-11-02T20:34:13.000000 | | virtual_size | None | +------------------+------------------------------------------------------------------------------------------+ [stack@undercloud-0 testingdir]$ nova boot --poll --image 1ff3cdb0-ac6f-4af3-a104-638f44701ee9 --flavor m1.micro --nic net-id=$SID vm1 +--------------------------------------+------------------------------------------------------+ | Property | Value | +--------------------------------------+------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | | | OS-EXT-SRV-ATTR:host | - | | OS-EXT-SRV-ATTR:hostname | vm1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | - | | OS-EXT-SRV-ATTR:instance_name | | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-jah0pt0i | | OS-EXT-SRV-ATTR:root_device_name | - | | OS-EXT-SRV-ATTR:user_data | - | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | scheduling | | OS-EXT-STS:vm_state | building | | OS-SRV-USG:launched_at | - | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | adminPass | 8u52Y3Z2Tx33 | | config_drive | | | created | 2017-11-02T20:40:04Z | | description | - | | flavor | m1.micro (02e0eee5-4a65-4bb9-9b09-40410a865a9e) | | hostId | | | host_status | | | id | 6030f79b-177a-4753-9cb8-00e96539eeb5 | | image | RHEL-7-x86_64 (1ff3cdb0-ac6f-4af3-a104-638f44701ee9) | | key_name | - | | locked | False | | metadata | {} | | name | vm1 | | os-extended-volumes:volumes_attached | [] | | progress | 0 | | security_groups | default | | status | BUILD | | tags | [] | | tenant_id | 9430d96b274046f7a0b2905343c4ebd0 | | updated | 2017-11-02T20:40:04Z | | user_id | d1cfcf10d34045e1ae6bcd950fec4ab7 | +--------------------------------------+------------------------------------------------------+ Server building... 100% complete Finished 2. Create 2 volumes and attach them [stack@undercloud-0 testingdir]$ cinder create --name vol1 1 +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2017-11-02T20:44:07.000000 | | description | None | | encrypted | False | | id | 0ab99417-c56d-4390-818f-e537605e969f | | metadata | {} | | migration_status | None | | multiattach | False | | name | vol1 | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 9430d96b274046f7a0b2905343c4ebd0 | | replication_status | disabled | | size | 1 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | d1cfcf10d34045e1ae6bcd950fec4ab7 | | volume_type | None | +--------------------------------+--------------------------------------+ [stack@undercloud-0 testingdir]$ cinder create --name vol2 1 +--------------------------------+--------------------------------------+ | Property | Value | +--------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2017-11-02T20:44:14.000000 | | description | None | | encrypted | False | | id | ba389b8a-4251-4792-b21c-834dee44f8fa | | metadata | {} | | migration_status | None | | multiattach | False | | name | vol2 | | os-vol-host-attr:host | None | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 9430d96b274046f7a0b2905343c4ebd0 | | replication_status | disabled | | size | 1 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | d1cfcf10d34045e1ae6bcd950fec4ab7 | | volume_type | None | +--------------------------------+--------------------------------------+ [stack@undercloud-0 testingdir]$ nova volume-attach vm1 0ab99417-c56d-4390-818f-e537605e969f /dev/vdf +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/sdb | | id | 0ab99417-c56d-4390-818f-e537605e969f | | serverId | 6030f79b-177a-4753-9cb8-00e96539eeb5 | | volumeId | 0ab99417-c56d-4390-818f-e537605e969f | +----------+--------------------------------------+ [stack@undercloud-0 testingdir]$ nova volume-attach vm1 ba389b8a-4251-4792-b21c-834dee44f8fa /dev/vdg +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/sdc | | id | ba389b8a-4251-4792-b21c-834dee44f8fa | | serverId | 6030f79b-177a-4753-9cb8-00e96539eeb5 | | volumeId | ba389b8a-4251-4792-b21c-834dee44f8fa | +----------+--------------------------------------+
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3229