Bug 1008903

Summary: q35: can't attach multiple sata devices
Product: Red Hat Enterprise Linux 7 Reporter: Gerd Hoffmann <kraxel>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: acathrow, bili, chhu, dyuan, juzhang, kraxel, lsu, shyu, sluo, weizhan
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-1.1.1-7.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 10:26:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
domain xml
none
libvirt logfile none

Description Gerd Hoffmann 2013-09-17 10:15:49 UTC
Description of problem:
q35: can't attach multiple sata devices

Version-Release number of selected component (if applicable):
libvirt-1.1.1-5.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
Try attach multiple sata disks/cdroms to the q35 ahci controller, using multiple
<disk ...> [ ... ] <target bus='sata'/></disk> entries in the <device>
section.

Actual results:
qemu fails to start

Expected results:
guest boots fine.

Additional info:
Maybe related:  The single disk which can be attached shows up at port 5 in the guest.  xml says <address ... unit='0'> though, so I'd expect it show up at port 0.

Comment 2 Laine Stump 2013-09-18 12:12:47 UTC
Just for completeness, can you attach the full domain xml and also paste in the qemu commandline and ensuing error messages from /var/log/libvirt/qemu/$guest.log?

Comment 3 Gerd Hoffmann 2013-09-18 12:28:53 UTC
Created attachment 799349 [details]
domain xml

Comment 4 Gerd Hoffmann 2013-09-18 12:32:32 UTC
Created attachment 799351 [details]
libvirt logfile

Comment 5 Laine Stump 2013-09-19 10:06:19 UTC
qemu is saying that the sata bus only supports a single unit. Is this true? (I couldn't find those error messages in my local copy of (upstream) qemu source)

Comment 6 Laine Stump 2013-09-19 10:24:27 UTC
I've also just found that when I add a 2nd sata controller and place both the primary hard disk and the cd drive on this controller, qemu *doesn't* complain.

Is this:

1) a "true-to-life" limitation of the integrated sata controller at 00:1f.2 of the q35 chipset?

2) a bug in qemu's emulation of that controller?

or perhaps 

3) an odd problem (bug?) in qemu caused by us not specifying a controller, instead relying on qemu to place the disks on the default controller?

If (1) then libvirt needs to be taught the limitation so that it only places a single disk on that controller. if (2) or (3) then this bug should be transferred to qemu.

Comment 7 Gerd Hoffmann 2013-09-19 10:29:31 UTC
(In reply to Laine Stump from comment #5)
> qemu is saying that the sata bus only supports a single unit. Is this true?
> (I couldn't find those error messages in my local copy of (upstream) qemu
> source)

Yes.  There are 6 busses though, one for each ahci port.  From 'info qtree':

      dev: ich9-ahci, id ""
        addr = 1f.2
        romfile = <null>
        rombar = 1
        multifunction = on
        command_serr_enable = on
        class SATA controller, addr 00:1f.2, pci id 8086:2922 (sub 1af4:1100)
        bar 4: i/o at 0xffffffffffffffff [0x1e]
        bar 5: mem at 0xffffffffffffffff [0xffe]
        bus: ide.5
          type IDE
        bus: ide.4
          type IDE
        bus: ide.3
          type IDE
        bus: ide.2
          type IDE
        bus: ide.1
          type IDE
        bus: ide.0
          type IDE

Comment 8 Gerd Hoffmann 2013-09-19 10:49:57 UTC
(In reply to Laine Stump from comment #6)
> I've also just found that when I add a 2nd sata controller and place both
> the primary hard disk and the cd drive on this controller, qemu *doesn't*
> complain.

Let me check ...

This happens because libvirt generates different command lines for the two cases:

built-in controller:

-device ide-hd,unit=0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1
-device ide-cd,unit=2,drive=drive-sata0-0-2,id=sata0-0-2,bootindex=3

additional controller:

-device ide-hd,bus=ahci1.0,drive=drive-sata1-0-0,id=sata1-0-0,bootindex=1
-device ide-cd,bus=ahci1.2,drive=drive-sata1-0-2,id=sata1-0-2,bootindex=3

libvirt should use "bus=ide.$nr" instead of "unit=$nr" for the built-in controller.

Comment 9 Laine Stump 2013-09-19 13:14:23 UTC
So it could possibly be (3). I'll try a build that sets the alias for the first sata controller to ide.0 in this case and see if the behavior changes.

Comment 10 Laine Stump 2013-09-19 14:45:14 UTC
Yep, switching to using an explicit bus eliminates the error. I'll post a patch upstream tonight.

Comment 11 Laine Stump 2013-09-20 10:30:54 UTC
Fix posted upstream, waiting for review:

https://www.redhat.com/archives/libvir-list/2013-September/msg01177.html

Comment 12 Laine Stump 2013-09-20 11:07:22 UTC
Pushed upstream:

commit 30bb4c4b54ed3a23adc90c52540dd96b0cbcfbcc
Author: Laine Stump <laine>
Date:   Fri Sep 20 06:00:48 2013 -0400

    qemu: use "ide" as device name for implicit SATA controller on Q35

    The Q35 machinetype has an implicit SATA controller at 00:1F.2 which
    isn't given the "expected" id of ahci0 by qemu when it's created. The
    original suggested solution to this problem was to not specify any
    controller for the disks that use the default controller and just
    specify "unit=n" instead; qemu should then use the first IDE or SATA
    controller for the disk.
    
    Unfortunately, this "solution" is ignorant of the fact that in the
    case of SATA disks, the "unit" attribute in the disk XML is actually
    *not* being used for the unit, but is instead used to specify the
    "bus" number; each SATA controller has 6 buses, and each bus only
    allows a single unit. This makes it nonsensical to specify unit='n'
    where n is anything other than 0. It also means that the only way to
    connect more than a single device to the implicit SATA controller is
    to explicitly give the bus names, which happen to be "ide.$n", where
    $n can be replaced by the disk's "unit" number.

Comment 15 chhu 2013-09-29 02:51:15 UTC
Reproduced with the packages:
libvirt-1.1.1-6.el7.x86_64

Verified with the packages:
libvirt-1.1.1-7.el7.x86_64

Test steps:
1. define a guest with xml:

    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rh7-qcow2.img'/>
      <target dev='sda' bus='sata'/>
      <boot order='1'/>
      <alias name='sata0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='sdc' bus='sata'/>
      <readonly/>
      <shareable/>
      <boot order='3'/>
      <alias name='sata0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='sata' index='0'>
      <alias name='sata0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>

2. virsh start the guest
# virsh start rh7.sata
Domain rh7.sata started
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     rh7.sata                       running

# ps -ef|grep qemu| grep sata
......-drive file=/var/lib/libvirt/images/rh7-qcow2.img,if=none,id=drive-sata0-0-0,format=qcow2 -device ide-hd,bus=ide.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-sata0-0-2,readonly=on,format=raw -device ide-cd,bus=ide.2,drive=drive-sata0-0-2,id=sata0-0-2,bootindex=3
......

Test results:
Guest start with multiple sata disk/cdrom successfully, so change the status to VERIFIED.

Comment 16 Ludek Smid 2014-06-13 10:26:42 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.