Bug 1024032

Summary: openstack-nova: volumes are not attached to instances, error: libvirtError: unsupported configuration: disk bus 'ide' cannot be hotplugged
Product: Red Hat OpenStack Reporter: Yogev Rabl <yrabl>
Component: openstack-novaAssignee: Nikola Dipanov <ndipanov>
Status: CLOSED ERRATA QA Contact: Toure Dunnon <tdunnon>
Severity: high Docs Contact:
Priority: medium    
Version: 3.0CC: eglynn, ndipanov, sclewis, sgordon, vpopovic, yeylon, yrabl
Target Milestone: rcKeywords: Triaged, ZStream
Target Release: 5.0 (RHEL 7)   
Hardware: x86_64   
OS: Linux   
Whiteboard: storage
Fixed In Version: openstack-nova-2014.1-3.el7ost Doc Type: Release Note
Doc Text:
In OpenStack Compute, attaching volumes to instances is not supported for device names like '/dev/hd*' which will cause the bus to be defaulted to 'ide', which cannot be hot plugged. Attaching volumes to running instances is only supported for virtio, so the device name needs to be similar to '/dev/vd*'
Story Points: ---
Clone Of:
: 1111700 (view as bug list) Environment:
Last Closed: 2014-07-08 15:26:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1111700    
Attachments:
Description Flags
nova compute logs
none
volume attachment failed. logs in debug level
none
nova compute none

Description Yogev Rabl 2013-10-28 15:59:43 UTC
Created attachment 816849 [details]
nova compute logs

Description of problem:
When trying to attach volumes to an instance, the action fails and the logs of the nova compute logs are attached.

Version-Release number of selected component (if applicable):
the openstack versions are: 

openstack-nova-compute-2013.1.4-1.el6ost.noarch            
openstack-dashboard-2013.1.4-1.el6ost.noarch               
openstack-utils-2013.1-8.1.el6ost.noarch                   
openstack-glance-2013.1.4-1.el6ost.noarch                  
openstack-selinux-0.1.2-10.el6ost.noarch                   
openstack-cinder-2013.1.4-2.el6ost.noarch                  
openstack-nova-api-2013.1.4-1.el6ost.noarch                
openstack-nova-console-2013.1.4-1.el6ost.noarch            
openstack-nova-conductor-2013.1.4-1.el6ost.noarch          
openstack-nova-novncproxy-0.4-6.el6ost.noarch              
openstack-nova-network-2013.1.4-1.el6ost.noarch            
openstack-nova-cert-2013.1.4-1.el6ost.noarch               
python-django-openstack-auth-1.0.6-2.el6ost.noarch         
redhat-access-plugin-openstack-1.2.0-5.el6ost.noarch       
openstack-keystone-2013.1.4-1.el6ost.noarch                
openstack-nova-common-2013.1.4-1.el6ost.noarch             
openstack-nova-scheduler-2013.1.4-1.el6ost.noarch          
openstack-packstack-2013.1.1-0.33.dev695.el6ost.noarch

The OS is: 
Red Hat Enterprise Linux Server release 6.5 Beta

How reproducible:
doesn't happen when booting from volume

Steps to Reproduce:
1. create a volume (any size)
2. create an instance
3. run 
  # nova volume attach <instance id> <volume id> auto / /dev/<whatever>

Actual results:
the volume is starting the attachment process and returning to available

Expected results:
the volume should be attached to the instance.

Additional info:

Comment 3 Solly Ross 2013-11-07 20:42:34 UTC
Could you please clairify the expect vs actual results, as the actual results that you listed do not seem to match up with the bug summary.

Comment 4 Nikola Dipanov 2013-11-12 17:03:51 UTC
Yogev,

Looking at the code - libvirt driver decides the device bus based on the device supplied, so /dev/hd* would default to 'ide' which seems to fail.

Could you confirm that it works for virtio devices (so setting the device to say /dev/vdc).

This was reported against 3.0, but I suspect will still happen in 4.0 - would be good to confirm.

Leaving the needinfo on.

Comment 5 Xavier Queralt 2013-11-15 10:16:50 UTC
Related fedora bug 1001731

Comment 6 Yogev Rabl 2013-11-20 08:48:26 UTC
(In reply to Nikola Dipanov from comment #4)
> Yogev,
> 
> Looking at the code - libvirt driver decides the device bus based on the
> device supplied, so /dev/hd* would default to 'ide' which seems to fail.
> 
> Could you confirm that it works for virtio devices (so setting the device to
> say /dev/vdc).
> 
> This was reported against 3.0, but I suspect will still happen in 4.0 -
> would be good to confirm.
> 
> Leaving the needinfo on.

it does work with virtio, as you said.

Comment 7 Yogev Rabl 2013-11-21 15:42:07 UTC
(In reply to Nikola Dipanov from comment #4)
> Yogev,
> 
> Looking at the code - libvirt driver decides the device bus based on the
> device supplied, so /dev/hd* would default to 'ide' which seems to fail.
> 
> Could you confirm that it works for virtio devices (so setting the device to
> say /dev/vdc).
> 
> This was reported against 3.0, but I suspect will still happen in 4.0 -
> would be good to confirm.
> 
> Leaving the needinfo on.

I've tried the same scenario today, when I have two nova compute nodes and one remote Cinder. 

I ran the command: 

nova volume-attach <instance uuid> <volume uuid> /dev/vdc 

And it failed. 
I have added the logs in debug level to the attachments.

Comment 8 Yogev Rabl 2013-11-21 15:43:21 UTC
Created attachment 827297 [details]
volume attachment failed. logs in debug level

Comment 9 Nikola Dipanov 2013-11-21 16:03:08 UTC
I will move this bug to 5.0 as there is a blueprint upstream to improve this in Icehouse.

https://blueprints.launchpad.net/nova/+spec/use-new-bdm-format-in-attach

Adding a release note to explain the issue.

Comment 11 Yogev Rabl 2013-11-24 15:19:16 UTC
Created attachment 828344 [details]
nova compute

To add my former statement: "and it failed" - my command was: nova volume-attach <sever id> <volume> /dev/vdc , as it was before. 
The output was: 

+----------+--------------------------------------+
| Property | Value                                |
+----------+--------------------------------------+
| device   | /dev/hdc                             |
| serverId | 0d36e5da-5440-4bd1-90e7-528537575a1c |
| id       | f70a7e28-a983-41a9-ae46-9b4261ab4bb4 |
| volumeId | f70a7e28-a983-41a9-ae46-9b4261ab4bb4 |
+----------+--------------------------------------+

thus it tried to add the volume as IDE bus. 
I'm attaching the compute log (in debug mode).

Comment 12 Stephen Gordon 2014-05-27 16:35:17 UTC
Is specifying the device ever supposed to work for Libvirt/KVM anyway? I thought the Xen driver was the only one that actually supported this (the rest end up using auto even if you specify)?

Comment 13 Nikola Dipanov 2014-05-29 09:09:24 UTC
Stephen - yes this is correct - libvirt/KVM should probably ignore it on the Nova side altogether.

However currently Nova uses slightly different code paths to default this when booting an instance and when attaching a volume to it - this is something that needs to get fixed but I never got around to it. Making nova disregard the information for different drivers would probably involve fixing this issue as well.

This will also free us of ugly gate bugs that keep popping up from time to time such as https://bugs.launchpad.net/nova/+bug/1193113

I will comment on there and prioritize this fix since it seems to be causing more pain than needed in different areas.

Comment 14 Vladan Popovic 2014-06-16 09:26:33 UTC
All 4 patches made it in the 2014.1 rebase and got in the build.
Setting appropriate fields and removing rhos-6.0 flag.

Comment 20 errata-xmlrpc 2014-07-08 15:26:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0853.html