This is not a cockpit-machines bug, it's either libvirt or qemu-kvm one, writing a cockpit-independent reproducer and assigning to the libvirt since it's closer layer to cockpit-machines.

I have a VM with the following two disk devices:

    <disk type='volume' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source pool='default' volume='fedora-29.qcow2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <boot order='2'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>

    <disk type='volume' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source pool='default' volume='test.img'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>

The first one, which is marked with 'boot order' 2 is bootable, whist the second one, is just a dummy qcow2 image created with 'qemu-img create test.img -f qcow2 50M'

When starting the VM, it never tries to boot from the second disk device, although the XML file has the 'boot order'=2 set for that.


The qemu command I see generated when running the domain is the following:

$ ps aux | grep qemu 
qemu     21594  2.7  0.5 1656624 66044 ?       Sl   18:20   0:18 /usr/bin/qemu-system-x86_64 -name guest=rhel8.0,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-13-rhel8.0/master-key.aes -machine pc-q35-4.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -cpu Skylake-Client-IBRS,ss=on,vmx=on,hypervisor=on,tsc_adjust=on,clflushopt=on,umip=on,stibp=on,ssbd=on,xsaves=on,pdpe1gb=on,mpx=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 91114a64-25ea-4771-9650-e35283bb05a6 -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=26,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -drive file=/var/lib/libvirt/images/fedora-29.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.4,addr=0x0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/var/lib/libvirt/images/test.img,format=qcow2,if=none,id=drive-virtio-disk1 -device virtio-blk-pci,scsi=off,bus=pci.7,addr=0x0,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=1 -netdev tap,fd=28,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:28:c1:97,bus=pci.1,addr=0x0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=30,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.6,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on


I see the disks are properly created with bootindex=2 and bootindex=1 accordingly in the qemu command, but the second disk is ignored and I don't get any interactive menu to choose the second disk to boot from.
Am I missing something here?

I think it's expected in the current stage, but I'm not sure why it is designed like this.

1) If the first disk boots failed, BIOS will not try to boot other disk.
boot order='1'  ->disk boot failed
boot order='2'  ->disk can not be bootable

2) If the first disk boots failed, BIOS will try to boot other order devices(cdrom,iPXE,etc.)
boot order='1'  ->disk boot failed
boot order='2'  ->disk can not be bootable 
boot order='3'  ->cdrom can be bootable

*** This bug has been marked as a duplicate of bug 1210651 ***