Please grab the xml virt-manager is producing from the output of virt-manager --debug

Created attachment 1267527 [details]
virt-manager --debug log for creating and attempting to start a Q35 machine with an IDE controller

Some observations:

- virt-manager doesn't allow adding IDE storage devices even when an IDE controller is present (the "BUS type" dropdown does not include "IDE" regardless of whether an IDE controller has been added).
- However, it's possible to add an IDE device using "virsh edit".
- Adding an IDE device with "virsh edit" will also automatically add an IDE controller, if there isn't one already.
- virt-manager does recognize IDE devices (e.g. an IDE disk drive appears in the list as "IDE CDROM 1" despite there being no obvious way to add one through its UI.
- On http://wiki.qemu-project.org/Features/Q35, the description of Q35 includes "contains an integrated IDE controller". Perhaps libvirt needs to learn how to use Q35's integrated IDE controller instead of trying to add its own?

Okay the XML is:

<domain type="kvm">
  <name>generic-test</name>
  <uuid>a47edf11-2b2b-4b95-b219-52f63e36ba37</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch="x86_64" machine="q35">hvm</type>
    <boot dev="network"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state="off"/>
  </features>
  <cpu mode="custom" match="exact">
    <model>IvyBridge</model>
  </cpu>
  <clock offset="utc">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/sbin/qemu-system-x86_64</emulator>
    <controller type="usb" index="0" model="ich9-ehci1"/>
    <controller type="usb" index="0" model="ich9-uhci1">
      <master startport="0"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci2">
      <master startport="2"/>
    </controller>
    <controller type="usb" index="0" model="ich9-uhci3">
      <master startport="4"/>
    </controller>
    <controller type="ide" index="0"/>
    <interface type="bridge">
      <source bridge="br0"/>
      <mac address="52:54:00:78:91:f7"/>
    </interface>
    <input type="mouse" bus="ps2"/>
    <graphics type="spice" port="-1" tlsPort="-1" autoport="yes">
      <image compression="off"/>
    </graphics>
    <console type="pty"/>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
    </channel>
    <sound model="ich6"/>
    <video>
      <model type="qxl"/>
    </video>
    <redirdev bus="usb" type="spicevmc"/>
    <redirdev bus="usb" type="spicevmc"/>
  </devices>
</domain>


<controller type=ide/> is causing the error, in this case it was explicitly added by the user, so not anything virt-manager is defaulting too.

My understanding is that q35 provides IDE behavior through SATA or something like that. Not sure if this is expected to work or not.

Laine, can you clarify?

> - virt-manager doesn't allow adding IDE storage devices even when an IDE
> controller is present (the "BUS type" dropdown does not include "IDE"
> regardless of whether an IDE controller has been added).

virt-manager shouldn't allow adding an IDE controller to *any* machinetype. On the machinetypes that have an integrated IDE controller, it will show up in the config automatically, but adding in additional IDE controllers beyond the integrated controller isn't supported. On machinetypes with no integrated controller, you also can't add any extra IDE controllers.

A long time ago I made a patch for libvirt to support adding IDE controllers beyond the default integrated IDE (on machines that have it) but then we discussed it and decided that we didn't want to encourage use of a "double old super slow" emulated disk interface, so we didn't add it (I think I didn't even post the patch). Instead we said that the proper way to support IDE on Q35 machinetypes for OSes that really were old enough to not support SATA/AHCI directly was:

1) convince qemu to add the IDE emulation mode to their SATA controller, since this is how a *real* Q35 machine would support it,

or

2) *Don't* support it, and tell people who must run OSes that ancient that they should use a 440fx-based machinetype instead.

> - However, it's possible to add an IDE device using "virsh edit".

You can do *a lot* of inadvisable / non-working things with virsh edit.

>- Adding an IDE device with "virsh edit" will also automatically add an IDE
> controller, if there isn't one already.

That's because, at the level that controllers are being automatically added, libvirt isn't checking whether or not that kind of disk controller is actually available on the machinetype you're using (it probably should).


- On http://wiki.qemu-project.org/Features/Q35, the description of Q35 includes "contains an integrated IDE  controller". Perhaps libvirt needs to learn how to use Q35's integrated IDE controller instead of trying to add its own?

This is the sentence on that page that references an integrated IDE controller:

   "The main bus of the I440FX is PCI and the SuperIO chipset contains an
    integrated IDE controller, USB controller, and PCI-ISA bridge."

So it's not saying that the Q35 machinetype has an integrated IDE controller, it's saying that the old 440fx machinetype has an integrated IDE controller. In the next paragraph (just after the diagram of the ICH9 chipset, which is part of of the Q35 machine) it says this:

    "The ICH9 has an integrated AHCI controller, ..."

The only way to have an IDE controller in a qemu Q35 machinetype is to add one with -device on the qemu commandline, and libvirt doesn't (and won't) support that.

You should just put your disk devices on the SATA bus and use the Q35's integrated SATA (aka AHCI) controller, or if your OS is so old that it doesn't support that, then use a 440fx machinetype.

I'm not sure there's anything to do here - it might be nice to add a check for unsupported bus type when defining a domain, but that just moves the error reporting a bit earlier.

(In reply to Laine Stump from comment #5)
> You should just put your disk devices on the SATA bus and use the Q35's
> integrated SATA (aka AHCI) controller, or if your OS is so old that it
> doesn't support that, then use a 440fx machinetype.

I'm guessing this is the point where I should provide some context:

The use case is installing Mac OS X. The OS apparently only works with the Q35 QEMU machine setting. Although the OS supports SATA storage devices, the installation disks will only work if they are inserted into an IDE-connected optical disk drive. I think this applies all the way to the last version that was available on physical disks (10.6), but newer versions of the OS can be downloaded from the Mac App Store and converted to bootable disk images, which will then have the same IDE requirement when booted from.

As such, none of your suggestions would be applicable. Although, to be fair, this is only necessary during installation, it would still make installing the OS simpler in some circumstances.

(In reply to Laine Stump from comment #5)
> > - However, it's possible to add an IDE device using "virsh edit".
> 
> You can do *a lot* of inadvisable / non-working things with virsh edit.

Actually, it is possible using just virt-manager. Just add a storage device, then, in the "Disk bus" field, type in "ide". It will add an IDE controller automatically.

(In reply to Vladimir Panteleev from comment #6)

> The use case is installing Mac OS X. The OS apparently only works with the
> Q35 QEMU machine setting. Although the OS supports SATA storage devices, the
> installation disks will only work if they are inserted into an IDE-connected
> optical disk drive.

Ugh.

My first reaction is that it would be "more true to actual hardware" to solve this by convincing qemu to add a switch that turns on IDE emulation in the SATA controller. Of course libvirt would also need to support that switch, but this is the way the situation is resolved on real hardware. I've talked to John Snow (QEMU SATA driver maintainer) about this before, but don't remember what his prognosis was. At the time neither of us was too concerned, because the only situation we knew of that required IDE was WinXP and other similarly "old" OSes that could also be supported by using an i440fx machinetype. Since this example is for a "modern" OS that won't run on i440fx yet still requires IDE, maybe it should be revisited.

(In reply to Vladimir Panteleev from comment #6)
> (In reply to Laine Stump from comment #5)
> > You should just put your disk devices on the SATA bus and use the Q35's
> > integrated SATA (aka AHCI) controller, or if your OS is so old that it
> > doesn't support that, then use a 440fx machinetype.
> 
> I'm guessing this is the point where I should provide some context:
> 
> The use case is installing Mac OS X. The OS apparently only works with the
> Q35 QEMU machine setting. Although the OS supports SATA storage devices, the
> installation disks will only work if they are inserted into an IDE-connected
> optical disk drive. I think this applies all the way to the last version
> that was available on physical disks (10.6), but newer versions of the OS
> can be downloaded from the Mac App Store and converted to bootable disk
> images, which will then have the same IDE requirement when booted from.

Have you got any pointers to webpages describing this requirement for IDE over SATA ?  I find it rather surprising that any OS from the last 10 years only works with IDE and not SATA.   Though I guess real BIOS have the setting that lets you make SATA disks look like IDE, it is still odd that OS-X which is cutting edge in its hardware use in many other ways, can't do SATA CDROMs

(In reply to Daniel Berrange from comment #9)
> Have you got any pointers to webpages describing this requirement for IDE
> over SATA ?

Gabriel Somlo's installation guide uses a QEMU command line with the disk drive specified as an IDE device:

http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/

Dhiru Kholia's OSX-KVM repo includes a libvirt XML configuration, which also specifies the installation disk drive as a direct QEMU parameter:

https://github.com/kholia/OSX-KVM/blob/a12802ca8875a53aeb8e650fa8695ef2e1c2a901/macOS-libvirt.xml#L60-L63

Although I don't know of any source which explicitly states that the installation drive MUST be connected by IDE, it is trivially verifiable through experiment.

> I find it rather surprising that any OS from the last 10 years
> only works with IDE and not SATA.

The IDE requirement only applies to the installation. I guess the installer's boot loader (or other component) only supports booting from IDE drives. Once the system is installed and booted from a hard drive, SATA-connected disk drives are supported.

> The use case is installing Mac OS X. The OS apparently only works with the
> Q35 QEMU machine setting. Although the OS supports SATA storage devices, the
> installation disks will only work if they are inserted into an IDE-connected
> optical disk drive. I think this applies all the way to the last version
> that was available on physical disks (10.6),

How exactly do you try to boot?

Booting the 10.6 installer iso directly doesn't fly on non-apple hardware, correct?

> but newer versions of the OS
> can be downloaded from the Mac App Store and converted to bootable disk
> images, which will then have the same IDE requirement when booted from.

How exactly do you create the bootable images?

Again, when using the tool shipped by apple with the os download you'll get images which don't work on non-apple hardware, correct?

(In reply to Gerd Hoffmann from comment #11)
> How exactly do you try to boot?

There are example qemu command lines at the links in my previous comment.

Here is one example from Gabriel Somlo's page:

    bin/qemu-system-x86_64 -machine q35,accel=kvm -bios ~/OVMF.fd -m 4096 \
      -cpu Penryn -smp 4,cores=2 \
      -usb -device usb-kbd -device usb-tablet \
      -device isa-applesmc,osk="insert-real-64-char-OSK-here" \
      -netdev user,id=usr0 -device e1000-82545em,netdev=usr0,id=vnet0 \
      -device ide-drive,bus=ide.0,drive=MacDVD \
      -drive id=MacDVD,if=none,snapshot=on,file=~/DVD/Sierra.10.12.1.iso \
      -device ide-drive,bus=ide.2,drive=MacHDD \
      -drive id=MacHDD,if=none,file=~/mac_hdd.img \
      -monitor stdio

> Booting the 10.6 installer iso directly doesn't fly on non-apple hardware,
> correct?

The OS identifies the machine as Apple hardware using a device apparently called an "Apple SMC, which is present on all Apple hardware. It is implemented in QEMU in hw/misc/applesmc.c, and can be enabled using the QEMU parameter:

      -device isa-applesmc,osk="insert-real-64-char-OSK-here"

The "insert-real-64-char-OSK-here" string must be replaced with the real 64-character OSK value, which can be extracted from a real Apple machine.

> How exactly do you create the bootable images?

Versions before and including OS X 10.6 were available on physical optical disks (dual-layer DVDs). These can be booted from directly.

> Again, when using the tool shipped by apple with the os download you'll get
> images which don't work on non-apple hardware, correct?

Files downloaded from the Mac App Store will not be bootable on any hardware (despite being in .dmg format and writable to optical disks), and require some preparation to be made bootable. Gabriel Somlo's page lists the necessary steps, and Dhiru Kholia's repository (both linked above) includes a script that will perform these steps automatically.

> Although I don't know of any source which explicitly states that the
> installation drive MUST be connected by IDE, it is trivially verifiable
> through experiment.

Tried USB?  I had trouble with SATA cdrom too, even though that was with iBoot (https://www.tonymacx86.com/resources/iboot-3-3-0.38/), i.e. before the macos sata driver even comes to play.  Throws read errors after changing the (virtual) cdrom.  Didn't investigate whenever that was a iBoot or seabios bug though.

  Hi,

> > Booting the 10.6 installer iso directly doesn't fly on non-apple hardware,
> > correct?
> 
> The OS identifies the machine as Apple hardware using a device apparently
> called an "Apple SMC, which is present on all Apple hardware. It is
> implemented in QEMU in hw/misc/applesmc.c, and can be enabled using the QEMU
> parameter:
> 
>       -device isa-applesmc,osk="insert-real-64-char-OSK-here"
> 
> The "insert-real-64-char-OSK-here" string must be replaced with the real
> 64-character OSK value, which can be extracted from a real Apple machine.

Additional issue is lacking firmware support (HFS+ filesystem),
but I see you are using the patched OVMF build.  Didn't try that, used hackintosh community tools instead (clover provides an hfs+ filesystem driver, beside other things needed to run osx on standard intel pc hardware).

> Files downloaded from the Mac App Store will not be bootable on any hardware
> (despite being in .dmg format and writable to optical disks), and require
> some preparation to be made bootable.

Yes (https://support.apple.com/en-us/HT201372).

> Gabriel Somlo's page lists the
> necessary steps, and Dhiru Kholia's repository (both linked above) includes
> a script that will perform these steps automatically.

Why go the extra mile and create an iso?  Just let createinstallmedia do the job of creating a bootable disk image.  Then add clover to it if needed, but when using a OVMF build with HFS+ support you can probably skip that.

Oh, and that doesn't require a usb stick, you can use a sata disk too.

(In reply to Gerd Hoffmann from comment #14)
> Additional issue is lacking firmware support (HFS+ filesystem),
> but I see you are using the patched OVMF build.  Didn't try that, used
> hackintosh community tools instead (clover provides an hfs+ filesystem
> driver, beside other things needed to run osx on standard intel pc hardware).

I admit I have been a bit ignorant of the exact process that was necessary to get everything to work. In this particular case, I am using the Chameleon kernel, as described on Gabriel Somlo's old page (http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/index_old.html). (I hadn't been able to get his OVMF fork to work at all so far.)

I hadn't thought about this before, but the issue due to which only IDE controllers are acceptable for booting the installation media might be a limitation of Chameleon, not OS X itself.

> > Gabriel Somlo's page lists the
> > necessary steps, and Dhiru Kholia's repository (both linked above) includes
> > a script that will perform these steps automatically.
> 
> Why go the extra mile and create an iso?  Just let createinstallmedia do the
> job of creating a bootable disk image.  Then add clover to it if needed, but
> when using a OVMF build with HFS+ support you can probably skip that.

I had not heard of neither clover nor createinstallmedia. If these truly make the process easier, I wonder why Gabriel Somlo's page doesn't mention them, despite him going as far as adding HFS+ support to EDK2, and updating QEMU's applesmc emulation.

(In reply to Gerd Hoffmann from comment #13)
> Tried USB?  I had trouble with SATA cdrom too, even though that was with
> iBoot (https://www.tonymacx86.com/resources/iboot-3-3-0.38/), i.e. before
> the macos sata driver even comes to play.  Throws read errors after changing
> the (virtual) cdrom.  Didn't investigate whenever that was a iBoot or
> seabios bug though.

I tested with the 10.6 installation media. Here is what I discovered:

- I made the wrong assumption that the installation media must be presented to the virtual machine as a CD-ROM drive. This is wrong! It must be presented as a disk device, otherwise Chameleon's boot loader will not see the drive (and it will not appear in its boot menu).
- Attaching the installation media using a SATA disk drive does not work, and results in either "readonly sata disks are not supported" or "Block node is read-only" errors depending on whether the "Read-only" checkbox is checked in virt-manager.
- Using USB and SCSI disk devices causes the installer to get to the loading screen with the spinning progress indicator, and seemingly get stuck there.
- Using an IDE disk device works, and is the only setting I found to get all the way to the language selection screen.
- As an aside, virt-manager does not seem to allow to configure an USB CD-ROM device (only a disk device), though I don't think it would matter because IDE, SATA and SCSI CD-ROM devices all fail to be detected by Chameleon.

Same information, presented as a table:

|------+------+----|
| Bus  | Disk | CD |
|------+------+----|
| IDE  | OK   | NB |
| SATA | RO   | NB |
| USB  | KS   | NC |
| SCSI | KS   | NB |
|------+------+----|

Legend:
NC = Not configurable
NB = Not in Chameleon boot menu
KS = Keeps spinning (even with USB 3)
OK = Works (gets to the language selection menu)
RO = "readonly sata disks are not supported" / "Block node is read-only"

Thank you for reporting this issue to the libvirt project. Unfortunately we have been unable to resolve this issue due to insufficient maintainer capacity and it will now be closed. This is not a reflection on the possible validity of the issue, merely the lack of resources to investigate and address it, for which we apologise. If you none the less feel the issue is still important, you may choose to report it again at the new project issue tracker https://gitlab.com/libvirt/libvirt/-/issues The project also welcomes contribution from anyone who believes they can provide a solution.