Bug 1489001 - nova boot --block-device-mapping wrong order
Summary: nova boot --block-device-mapping wrong order
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-novaclient
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Eoghan Glynn
QA Contact: Joe H. Rahme
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-06 14:57 UTC by Eduardo Minguez
Modified: 2019-09-09 17:18 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-09-08 11:12:11 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Eduardo Minguez 2017-09-06 14:57:02 UTC
Description of problem:
Using nova boot to create instances with attached volumes at boot time makes volume attached to different devices than the specified.

Version-Release number of selected component (if applicable):
nova --version
6.0.0
python-openstackclient-3.2.1-1.el7ost.noarch
python-novaclient-6.0.0-3.el7ost.noarch

How reproducible:
Try to create an instance with 3 precreated cinder volumes of different sizes attached to it.

Steps to Reproduce:

1. domain="control.edu.example.com"
2. blkid1=$(openstack volume show master-0-docker -f value -c id)
3. blkid2=$(openstack volume show master-0-etcd -f value -c id)
4. blkid3=$(openstack volume show master-0-openshift-local -f value -c id)
5. openstack volume list | grep master-1
| b9e4c257-f7c7-4733-acd6-4d9df7ad6da9 | master-1-openshift-local     | available |   30 |             |
| 5f8ca402-7096-49cd-8004-070987675c3b | master-1-etcd                | available |   25 |             |
| 2a7da1ec-b648-4559-8ca6-51167112d400 | master-1-docker              | available |   20 |             |
6. echo $blkid1
2a7da1ec-b648-4559-8ca6-51167112d400
7. echo $blkid2
5f8ca402-7096-49cd-8004-070987675c3b
8. echo $blkid3
b9e4c257-f7c7-4733-acd6-4d9df7ad6da9
9. nova boot --flavor m1.medium --image rhel7 --key-name ocp3kp --user-data ./user-data/master-1.yaml --security-groups master-sg,node-sg --block-device-mapping vdb=$blkid1       --block-device-mapping vdc=$blkid2 --block-device-mapping vdd=$blkid3  --nic net-id=$netid1 master-1.$domain


Actual results:

openstack volume list | grep master-1

| b9e4c257-f7c7-4733-acd6-4d9df7ad6da9 | master-1-openshift-local     | in-use    |   30 | Attached to master-1.control.edu.example.com on /dev/vdd  |
| 5f8ca402-7096-49cd-8004-070987675c3b | master-1-etcd                | in-use    |   25 | Attached to master-1.control.edu.example.com on /dev/vdb  |
| 2a7da1ec-b648-4559-8ca6-51167112d400 | master-1-docker              | in-use    |   20 | Attached to master-1.control.edu.example.com on /dev/vdc  |


Expected results:
vdb attached to master-1-docker
vdc attached to master-1-etcd
vdd attached to master-1-openshift-local

Additional info:
I've tried with openstack server create as well but it same result.

Comment 1 Eduardo Minguez 2017-09-06 14:59:03 UTC
(In reply to Eduardo Minguez from comment #0)
Sorry, copy & paste issues...
When it says "master-0-*" it is "master-1-*"

> 2. blkid1=$(openstack volume show master-0-docker -f value -c id)
> 3. blkid2=$(openstack volume show master-0-etcd -f value -c id)
> 4. blkid3=$(openstack volume show master-0-openshift-local -f value -c id)

Comment 2 Eduardo Minguez 2017-09-06 15:11:23 UTC
Using --block-device to create the volume at boot time seems to work tho:

$ nova boot --flavor m1.medium --image rhel7 --key-name ocp3kp --user-data ./user-data/master-1.yaml --security-groups master-sg,node-sg --block-device source=blank,dest=volume,device=vdb,size=25,shutdown=preserve --block-device source=blank,dest=volume,device=vdc,size=20,shutdown=preserve --block-device source=blank,dest=volume,device=vdd,size=30,shutdown=preserve --nic net-id=$netid1 master-1.$domain
...
$ openstack volume list
+--------------------------------------+------------------------------------------------+--------+------+-----------------------------------------------------------+
| ID                                   | Display Name                                   | Status | Size | Attached to                                               |
+--------------------------------------+------------------------------------------------+--------+------+-----------------------------------------------------------+
| 924dc148-5b57-4bdf-8740-232c832fc81a | 36be1cb7-cbd6-4092-b1fe-fb57e4cbaf0d-blank-vol | in-use |   30 | Attached to master-1.control.edu.example.com on /dev/vdd  |
| a5dc0736-7967-46fe-ac42-25aef97e9066 | 36be1cb7-cbd6-4092-b1fe-fb57e4cbaf0d-blank-vol | in-use |   20 | Attached to master-1.control.edu.example.com on /dev/vdc  |
| ff220008-e07d-4664-ac65-40e4cb917abe | 36be1cb7-cbd6-4092-b1fe-fb57e4cbaf0d-blank-vol | in-use |   25 | Attached to master-1.control.edu.example.com on /dev/vdb  |
+--------------------------------------+------------------------------------------------+--------+------+-----------------------------------------------------------+

Comment 3 Matthew Booth 2017-09-08 11:12:11 UTC
For an array of reasons Nova doesn't make any guarantees about the order that devices are presented to the guest. The order can even change within the lifecycle of the instance, for example across a reboot. Similarly Linux doesn't even make any guarantees about the order devices presented to the guest are presented to the user. Relying on this is never going to be robust.

This problem is fixed in OSP10 onwards with device tagging:

  https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html/networking_guide/use-tagging

It also supports vifs, which have a similar issue.


Note You need to log in before you can comment on or make changes to this bug.