Bug 1498042

Summary: RFE: option to mark virtual block device as rotational/non-rotational
Product: Red Hat Enterprise Linux 7 Reporter: Daniel Berrangé <berrange>
Component: qemu-kvm-rhevAssignee: Daniel Berrangé <berrange>
Status: CLOSED ERRATA QA Contact: CongLi <coli>
Severity: low Docs Contact:
Priority: low    
Version: 7.5CC: aliang, berrange, chayang, coli, eblake, juzhang, kchamart, knoel, lolyu, mailinglists35, michen, mtessun, qzhang, thomas, virt-maint, xuwei
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.10.0-10.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1498955 1583807 (view as bug list) Environment:
Last Closed: 2018-04-11 00:38:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 1498955    

Description Daniel Berrangé 2017-10-03 11:35:04 UTC
Description of problem:
In absence of information to the contrary, Linux guests assume that all block devices are backed by rotational storage. The guest OS admin can override this by setting the 'rotational' flag in sysfs.

It would be desirable to be able to inform the guest OS the correct rotational/non-rotational setting based on what is used as the backing for the virtual block device.

In Linux, if a disk is marked non-rotational disk, it is also removed as a source of random entropy, so if a virtual disk is backed by SSD it is desirable that the guest know this so that it doesn't consider it an entropy source.

Both ATA and SCSI protocols have a way to report this feature.

For SCSI the Block Device Characteristics VPD, page 0xb1 provides a flag that Linux checks:

        if (!buffer ||
            /* Block Device Characteristics VPD */
            scsi_get_vpd_page(sdkp->device, 0xb1, buffer, vpd_len))
                goto out;

        rot = get_unaligned_be16(&buffer[4]);

        if (rot == 1) {
                queue_flag_set_unlocked(QUEUE_FLAG_NONROT, q);
                queue_flag_clear_unlocked(QUEUE_FLAG_ADD_RANDOM, q);
        }


For ATA there is an ID flag with the info that Linux checks

        ATA_ID_ROT_SPEED        = 217,

static inline bool ata_id_is_ssd(const u16 *id)
{
        return id[ATA_ID_ROT_SPEED] == 0x01;
}



virtio-blk has no mechanism for this. One could be added, or just rely on a recommendation use of virtio-scsi instead.

So QEMU would need to have a 'rotational' boolean flag added for its SCSI & ATA disk device backends. Libvirt would in turn expose this setting to mgmt apps, which would set it based on their knowledge of the storage in use.

Comment 3 Eric Blake 2017-10-03 12:02:37 UTC
NBD also supports the advertisement of rotational or not

Comment 4 Daniel Berrangé 2017-10-03 12:05:49 UTC
(In reply to Eric Blake from comment #3)
> NBD also supports the advertisement of rotational or not

Interestingly/strangely the Linux NBD kernel client ignores this feature, and unconditionally sets the block dev as non-rotational.

Comment 5 Daniel Berrangé 2017-10-04 11:52:54 UTC
Posted some possible patches for IDE/SCSI https://lists.gnu.org/archive/html/qemu-devel/2017-10/msg00698.html

Comment 8 Daniel Berrangé 2017-11-29 11:32:43 UTC
Merged upstream in 3 patches


commit 96f43c2b0a663f4789b51ed97297163321e7ba5e
Author: Daniel P. Berrange <berrange@redhat.com>
Date:   Fri Oct 20 10:14:03 2017 +0100

    ide: avoid referencing NULL dev in rotational rate setting
    
    The 'dev' variable can be NULL when the guest OS calls identify on an IDE
    unit that does not have a drive attached to it.
    
    Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
    Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
    Message-id: 20171020091403.1479-1-berrange@redhat.com
    Signed-off-by: John Snow <jsnow@redhat.com>


commit 3b19f4506901ecce25ff36cf62353a2b4bfe4f2b
Author: Daniel P. Berrange <berrange@redhat.com>
Date:   Wed Oct 4 12:40:08 2017 +0100

    ide: support reporting of rotation rate
    
    The Linux kernel will query the ATA IDENTITY DEVICE data, word 217
    to determine the rotations per minute of the disk. If this has
    the value 1, it is taken to be an SSD and so Linux sets the
    'rotational' flag to 0 for the I/O queue and will stop using that
    disk as a source of random entropy. Other operating systems may
    also take into account rotation rate when setting up default
    behaviour.
    
    Mgmt apps should be able to set the rotation rate for virtualized
    block devices, based on characteristics of the host storage in use,
    so that the guest OS gets sensible behaviour out of the box. This
    patch thus adds a 'rotation-rate' parameter for 'ide-hd' device
    types.
    
    Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
    Message-Id: <20171004114008.14849-3-berrange@redhat.com>
    Reviewed-by: John Snow <jsnow@redhat.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

commit 070f80095ad5b1143b50d2faffd2b1a84292e00d
Author: Daniel P. Berrange <berrange@redhat.com>
Date:   Wed Oct 4 12:40:07 2017 +0100

    scsi-disk: support reporting of rotation rate
    
    The Linux kernel will query the SCSI "Block device characteristics"
    VPD to determine the rotations per minute of the disk. If this has
    the value 1, it is taken to be an SSD and so Linux sets the
    'rotational' flag to 0 for the I/O queue and will stop using that
    disk as a source of random entropy. Other operating systems may
    also take into account rotation rate when setting up default
    behaviour.
    
    Mgmt apps should be able to set the rotation rate for virtualized
    block devices, based on characteristics of the host storage in use,
    so that the guest OS gets sensible behaviour out of the box. This
    patch thus adds a 'rotation-rate' parameter for 'scsi-hd' and
    'scsi-block' device types. For the latter, this parameter will be
    ignored unless the host device has TYPE_DISK.
    
    Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
    Message-Id: <20171004114008.14849-2-berrange@redhat.com>
    Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

Comment 10 Miroslav Rezanina 2017-11-30 16:54:18 UTC
Fix included in qemu-kvm-rhev-2.10.0-10.el7

Comment 12 CongLi 2017-12-05 10:51:03 UTC
Verified on:
qemu-kvm-rhev-2.10.0-10.el7.x86_64


1.  scsi-hd:
1.1 # /usr/libexec/qemu-kvm -device scsi-hd,\?
scsi-hd.rotation_rate=uint16

1.2 rotation_rate=1
CML:
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x3 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,rotation_rate=1 \
qtree info:
    rotation_rate = 1 (0x1)
guest:
    # cat /sys/block/sda/queue/rotational
      0

1.3 rotation_rate=0
CML:
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x3 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,rotation_rate=0 \
qtree info:
    rotation_rate = 0 (0x0)
guest:
    # cat /sys/block/sda/queue/rotational
      1


2.  scsi-block:
2.1 # /usr/libexec/qemu-kvm -device scsi-block,\?
scsi-block.rotation_rate=uint16

2.2 rotation_rate=1
CML:
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x3 \
    -drive file=/dev/sdb,if=none,media=disk,format=raw,rerror=stop,werror=stop,cache=none,aio=native,id=scsi-disk0 \
    -device scsi-block,drive=scsi-disk0,id=disk,rotation_rate=1 \
qtree info:
    dev: scsi-block, id "disk"
      drive = "scsi-disk0"
      rotation_rate = 1 (0x1)
guest:
    # cat /sys/block/sdb/queue/rotational
      0

2.3 rotation_rate=0
CML:
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x3 \
    -drive file=/dev/sdb,if=none,media=disk,format=raw,rerror=stop,werror=stop,cache=none,aio=native,id=scsi-disk0 \
    -device scsi-block,drive=scsi-disk0,id=disk,rotation_rate=0 \
qtree info:
    dev: scsi-block, id "disk"
      drive = "scsi-disk0"
      rotation_rate = 0 (0x0)
guest:
    # cat /sys/block/sda/queue/rotational
      1


3.  ide-hd:
3.1 # /usr/libexec/qemu-kvm -device ide-hd,\?
ide-hd.rotation_rate=uint16

3.2 rotation_rate=1
CML:
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio.qcow2 \
    -device ide-hd,id=image1,drive=drive_image1,bus=ide.0,unit=0,rotation_rate=1 \
qtree info:
    rotation_rate = 1 (0x1)
guest:
    # cat /sys/block/sda/queue/rotational
      0

3.3 rotation_rate=0
CML:
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio.qcow2 \
    -device ide-hd,id=image1,drive=drive_image1,bus=ide.0,unit=0,rotation_rate=0 \
qtree info:
    rotation_rate = 0 (0x0)
guest:
    # cat /sys/block/sda/queue/rotational
      1

Comment 13 CongLi 2017-12-05 10:54:45 UTC
Hi Daniel,

Could you please help confirm that ide-drive and scsi-generic will not add rotational flag, right ?

Thanks.

Comment 14 Daniel Berrangé 2017-12-05 11:56:54 UTC
That is correct, only  ide-hd, scsi-block and scsi-hd support this. 

BTW, as well as testing  rotation_rate=1 (non rotational), and rotation_rate=0 (unspecified rotation rate), it is worth testing rotation_rate=15000  (a rotation rate of 15,000 RPM). The latter should result in rotational==1 in the guest, as with rotation_rate=0. 

Also, check

  $ smartctl -i /dev/NNN

reports a "Rotation Rate" value, when using a rotation_rate != 0 in QEMU.

Comment 15 CongLi 2017-12-06 06:19:18 UTC
1. scsi-block:
CML:
    -drive file=/dev/sdb,if=none,media=disk,format=raw,rerror=stop,werror=stop,cache=none,aio=native,id=scsi-disk0 \
    -device scsi-block,drive=scsi-disk0,id=disk,rotation_rate=15000 \
qtree info:
              dev: scsi-block, id "disk"
                rotation_rate = 15000 (0x3a98)
guest:
# cat /sys/block/sdb/queue/rotational
1
# smartctl -i /dev/sdb 
smartctl 6.5 2016-05-07 r4318 [x86_64-linux-3.10.0-800.el7.x86_64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               NETAPP
Product:              LUN
Revision:             7320
User Capacity:        42,858,446,848 bytes [42.8 GB]
Logical block size:   512 bytes
Rotation Rate:        15000 rpm
Logical Unit id:      0x60a9800050334c33424a7856476258640x4a785647625864000a980050334c3342
Serial number:        P3L3BJxVGbXd
Device type:          disk
Transport protocol:   Fibre channel (FCP-2)
Local Time is:        Wed Dec  6 13:16:42 2017 CST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Disabled or Not Supported


2. scsi-hd:
CML:
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/rhel75-64-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,rotation_rate=15000 \
qtree info:
              dev: scsi-hd, id "image1"
                rotation_rate = 15000 (0x3a98)
guest:
# cat /sys/block/sda/queue/rotational
1
# smartctl -i /dev/sda
smartctl 6.5 2016-05-07 r4318 [x86_64-linux-3.10.0-800.el7.x86_64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               QEMU
Product:              QEMU HARDDISK
Revision:             2.5+
User Capacity:        21,474,836,480 bytes [21.4 GB]
Logical block size:   512 bytes
LU is thin provisioned, LBPRZ=0
Rotation Rate:        15000 rpm
Device type:          disk
Local Time is:        Wed Dec  6 13:19:10 2017 CST
SMART support is:     Unavailable - device lacks SMART capability.

3. ide-hd:
CML:

qtree info:
          dev: ide-hd, id "image1"
            rotation_rate = 15000 (0x3a98)
guest:
# cat /sys/block/sda/queue/rotational
1
# smartctl -i /dev/sda
smartctl 6.5 2016-05-07 r4318 [x86_64-linux-3.10.0-789.el7.x86_64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     QEMU HARDDISK
Serial Number:    QM00001
Firmware Version: 2.5+
User Capacity:    21,474,836,480 bytes [21.4 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    15000 rpm
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ATA/ATAPI-7, ATA/ATAPI-5 published, ANSI NCITS 340-2000
Local Time is:    Wed Dec  6 13:59:58 2017 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled


Hi Daniel,

Could you please help confirm the following scenario works as expected?

1. host: 
#modprobe scsi_debug
# lsscsi 
[0:0:0:0]    storage HP       P420i            7.02  -        
[0:1:0:0]    disk    HP       LOGICAL VOLUME   7.02  /dev/sda 
[3:0:0:0]    cd/dvd  hp       DVDRAM GT80N     EA02  /dev/sr0 
[14:0:0:0]   disk    Linux    scsi_debug       0004  /dev/sdb

2. pass through /dev/sdb to guest.
CML:
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x3 \
    -drive file=/dev/sdb,if=none,media=disk,format=raw,rerror=stop,werror=stop,cache=none,aio=native,id=scsi-disk0 \
    -device scsi-block,drive=scsi-disk0,id=disk,rotation_rate=15000 \
qtree info:
              dev: scsi-block, id "disk"
                rotation_rate = 15000 (0x3a98)
guest:
#  cat /sys/block/sda/queue/rotational
0
# smartctl -i /dev/sda 
smartctl 6.5 2016-05-07 r4318 [x86_64-linux-3.10.0-789.el7.x86_64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               Linux
Product:              scsi_debug
Revision:             0004
User Capacity:        8,388,608 bytes [8.38 MB]
Logical block size:   512 bytes
Rotation Rate:        Solid State Device
Form Factor:          < 1.8 inches
Logical Unit id:      0x5333333000007530
Serial number:        30000
Device type:          disk
Transport protocol:   SAS (SPL-3)
Local Time is:        Wed Dec  6 14:18:25 2017 CST
SMART support is:     Available - device has SMART capability.
SMART support is:     Disabled
Temperature Warning:  Disabled or Not Supported


Thanks.

Comment 17 Daniel Berrangé 2017-12-08 09:37:11 UTC
That all looks like it is operating correctly. Note, however, that your very last test with virtio-scsi-pci is not required - that device model was not affected by this work.

Comment 18 CongLi 2017-12-08 09:42:37 UTC
(In reply to Daniel Berrange from comment #17)
> That all looks like it is operating correctly. Note, however, that your very
> last test with virtio-scsi-pci is not required - that device model was not
> affected by this work.

Thanks Daniel.

Based on comment 12, comment 15 and comment 17, set this bug to 'VERIFIED'.

Comment 19 aihua liang 2017-12-13 03:20:57 UTC
Hi, Daniel

  Test with 6.9 guest, no "Rotation Rate" displayed in "smartctl -i /dev/sda".

qemu cmd:
 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=raw,file=/home/kvm_autotest_root/images/rhel69-64-virtio.raw \
-device ide-hd,id=image1,drive=drive_image1,bus=ide.0,unit=0,rotation_rate=15000 \

qtree info:
  rotation_rate = 15000 (0x3a98)

guest info:
  # cat /sys/block/sda/queue/rotational 
     1
  
  # smartctl -i /dev/sda
smartctl 5.43 2016-09-28 r4347 [x86_64-linux-2.6.32-696.el6.x86_64] (local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Device Model:     QEMU HARDDISK
Serial Number:    QM00001
Firmware Version: 2.5+
User Capacity:    21,474,836,480 bytes [21.4 GB]
Sector Size:      512 bytes logical/physical 
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   5
ATA Standard is:  ATA/ATAPI-5 published, ANSI NCITS 340-2000
Local Time is:    Wed Dec 13 06:17:28 2017 CST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Comment 20 Daniel Berrangé 2017-12-13 09:07:34 UTC
I expect that is simply due to RHEL-6 having an older version of smartctl which doesn't report this info.

Comment 21 aihua liang 2017-12-13 09:32:22 UTC
Got it, thank you, Daniel.

Comment 23 errata-xmlrpc 2018-04-11 00:38:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1104

Comment 24 Mai Ling 2018-06-27 19:55:06 UTC
what is the corresponding domain xml file for this feature? i.e. I want to force a vm guest think the storage is ssd

Comment 25 Mai Ling 2018-06-27 20:06:43 UTC
I mean...

[root@gw2 vmstore]# virt-xml gwwin --edit --disk rotation_rate=0
ERROR    Unknown options ['rotation_rate']

Comment 26 Mai Ling 2018-06-29 08:59:58 UTC
oh, libvirt does not know yet about this. filed #1595931