Bug 1200049

Summary: VirtIO-SCSI device order wrong in VM
Product: Red Hat Enterprise Linux 6 Reporter: Evgheni Dereveanchin <ederevea>
Component: qemu-kvmAssignee: Fam Zheng <famz>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.6CC: auditt7708, bsarathy, chayang, guy.baconniere, juzhang, mkenneth, qzhang, rbalakri, sluo, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-11 03:28:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Evgheni Dereveanchin 2015-03-09 15:32:38 UTC
Description of problem:
When using VirtIO-SCSI and multiple disks, the disk order visible in the VM is random, causing issues with VM provisioning (disk names unpredictable) and even unbootable VM (disk added becomes first in VM and renders it unbootable)

Version-Release number of selected component (if applicable):
qemu-img-rhev-0.12.1.2-2.446.el6.x86_64

RHEL 6.6 in VM

How reproducible:
Always

Steps to Reproduce:
1. Create a VM in RHEV with 4 disks:
Disk1 10GB
Disk2 20GB
Disk3 40GB
Disk4 30GB

2) use VirtIO bus, start VM and check disk proper names:
Disk1 /dev/vda
Disk2 /dev/vdb
Disk3 /dev/vdc
Disk4 /dev/vdd

3) attach all disks to the Virtio-SCSI controller
Actual results:
Inconsistent device naming:
Disk1 /dev/sdd
Disk2 /dev/sda
Disk3 /dev/sdb
Disk4 /dev/sdc
(may be in different order on a different VM)

Expected results:
Consistent device naming:
Disk1 /dev/sda
Disk2 /dev/sdb
Disk3 /dev/sdc
Disk4 /dev/sdd

Additional info:

here's what dmesg may look like in a VM:
sd 2:0:0:0: [sda] 20971520 512-byte logical blocks: (10.7 GB/10.0 GiB)
sd 2:0:0:2: [sdb] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
sd 2:0:0:3: [sdc] 83886080 512-byte logical blocks: (42.9 GB/40.0 GiB)
sd 2:0:0:1: [sdd] 41943040 512-byte logical blocks: (21.4 GB/20.0 GiB)

Comment 3 Fam Zheng 2015-03-11 03:28:20 UTC
A sane guest should not rely on the automatically generated /dev/sdX names to address a partition. Even if the order in a single bus is consistent, attaching a different bus (for example, ide/ahci) could as easily break the names.

Alternatively, there are /dev/disk/by-* symbol links that are more robust to system configuration changes. If you really need persistent /dev/sdX names, use udev persistent device name rules.

Fam

Comment 4 Fam Zheng 2015-03-11 03:32:26 UTC
By default, RHEL 6 installation (and almost every Linux distribution) uses UUID to find boot and root filesystem devices, so it will work still after you plug new disks that changes /dev/sdX order.

Comment 5 Guy Baconniere 2015-05-12 12:55:44 UTC
I had the same issue after switching from virtio-blk to virtio-scsi
udev does not cache sda pci map even if you add serial to the disk.

As workaround, I changed the target (on top of the unit) and 
it is working.

BEFORE

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='3'/>

AFTER (workaround)

      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
      <address type='drive' controller='0' bus='0' target='1' unit='1'/>
      <address type='drive' controller='0' bus='0' target='2' unit='2'/>
      <address type='drive' controller='0' bus='0' target='3' unit='3'/>

    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source dev='/var/lib/libvirt/images/boot.raw'/>
      <target dev='sda' bus='scsi'/>
      <serial>42-001</serial>
      <boot order='1'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source dev='/var/lib/libvirt/images/root.raw'/>
      <target dev='sdb' bus='scsi'/>
      <serial>42-002</serial>
      <alias name='scsi0-0-1-1'/>
      <address type='drive' controller='0' bus='0' target='1' unit='1'/>
    </disk>
    ...
    <controller type='scsi' index='0' model='virtio-scsi'>
      <alias name='scsi0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>

The interessting part is also that the ID_SCSI does not change?!

for dev in sd{a..d}; do echo -e "\n[$dev]"; udevadm info --query=all --name=/dev/$dev | grep -E '(ID_SCSI|ID_PATH)'; done

[sda]
E: ID_PATH=pci-0000:00:05.0-virtio-pci-virtio2-scsi-0:0:0:0
E: ID_PATH_TAG=pci-0000_00_05_0-virtio-pci-virtio2-scsi-0_0_0_0
E: ID_SCSI=1
E: ID_SCSI_SERIAL=42-001

[sdb]
E: ID_PATH=pci-0000:00:05.0-virtio-pci-virtio2-scsi-0:0:1:1
E: ID_PATH_TAG=pci-0000_00_05_0-virtio-pci-virtio2-scsi-0_0_1_1
E: ID_SCSI=1
E: ID_SCSI_SERIAL=42-002

[sdc]
E: ID_PATH=pci-0000:00:05.0-virtio-pci-virtio2-scsi-0:0:2:2
E: ID_PATH_TAG=pci-0000_00_05_0-virtio-pci-virtio2-scsi-0_0_2_2
E: ID_SCSI=1
E: ID_SCSI_SERIAL=42-003

[sdd]
E: ID_PATH=pci-0000:00:05.0-virtio-pci-virtio2-scsi-0:0:3:3
E: ID_PATH_TAG=pci-0000_00_05_0-virtio-pci-virtio2-scsi-0_0_3_3
E: ID_SCSI=1
E: ID_SCSI_SERIAL=42-004

Comment 6 jmkim 2015-06-07 16:17:39 UTC
How do the possible need to modify a change?

Virtio-scsi must either modify the config volume may seem normal?