Bug 1677833

Summary: [Q35]Drive number is incorrect during install windows guest os when with two virtio-blk disks
Product: Red Hat Enterprise Linux 8 Reporter: CongLi <coli>
Component: virtio-winAssignee: Vadim Rozenfeld <vrozenfe>
virtio-win sub component: virtio-win-prewhql QA Contact: menli <menli>
Status: CLOSED CANTFIX Docs Contact:
Severity: high    
Priority: high CC: ailan, chayang, juzhang, kanderso, lijin, mdean, phou, qinwang, rbalakri, virt-maint, vrozenfe, wyu, xuwei, ymankad
Version: ---Keywords: Reopened
Target Milestone: rc   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1754750 (view as bug list) Environment:
Last Closed: 2021-01-05 10:55:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1754750, 1771318, 1897024    
Attachments:
Description Flags
screen dumps video of installation
none
blk-install-1803
none
blk-install-1809
none
finishing installation
none
boot menu with 2 virtio-blk-pci devices
none
System Reserved partition in data disk, not in system disk.
none
big_image_install_fail
none
drive_num is not match expected
none
screenshot
none
boot fail
none
data disk offline
none
show details
none
continue show details none

Description CongLi 2019-02-16 07:13:35 UTC 
Created attachment 1535375 [details]
screen dumps video of installation

Description of problem:
Installing procedure will repeat with:
1. two virtio-blk disks
2. machine type: pc
3. win 10 and win2019 guests

Version-Release number of selected component (if applicable):
qemu-kvm-2.12.0-61.module+el8+2786+5afd5ae3
kernel-4.18.0-68.el8.x86_64
virtio-win-1.9.7-3.el8.iso

How reproducible:
100%

Steps to Reproduce:
1. Create 2 disks.
 /usr/local/bin/qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage.qcow2 21G
/usr/local/bin/qemu-img create -f qcow2 /home/kvm_autotest_root/images/win10-64-virtio.qcow2 20G

2. Install a Win10 guest with two virtio-blk disks.
      -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win10-64-virtio.qcow2     \
      -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x4 \
      -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/storage.qcow2 \
      -device virtio-blk-pci,id=stg,bootindex=1,drive=drive_stg,bus=pci.0,addr=0x5 \


Actual results:
Installing procedure will repeat after restarting guest.

Expected results:
Installing procedure should be finished, and guest could be installed successfully.

Additional info:
1. it's only existed on virtio-blk, virtio-scsi works well.
2. win2016 guest works well.
3. machine type q35 works well.
4. Download virtio-win to virtio-win-1.9.6-1.el8.iso, could still met this issue, download qemu-kvm to qemu-kvm-2.12.0-48.module+el8+2529+a9686a4d.x86_64, could still met this issue.
Report as qemu-kvm bug at first, please correct me if it's a virtio-win bug.
5. QEMU CML:
 MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
     -S  \
     -name 'avocado-vt-vm1' \
     -machine pc  \
     -nodefaults \
     -device VGA,bus=pci.0,addr=0x2  \
     -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win10-64-virtio.qcow2 \
     -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x4 \
     -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/storage.qcow2 \
     -device virtio-blk-pci,id=stg,bootindex=1,drive=drive_stg,bus=pci.0,addr=0x5 \
     -device virtio-net-pci,mac=9a:0c:0d:0e:0f:10,id=idR5t2rg,vectors=4,netdev=idfPb3nh,bus=pci.0,addr=0x6  \
     -netdev tap,id=idfPb3nh,vhost=on \
     -m 30720  \ 
     -smp 12,maxcpus=12,cores=6,threads=1,sockets=2  \
     -cpu 'Skylake-Server',+kvm_pv_unhalt,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time \
     -drive id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/ISO/Win10/en_windows_10_business_edition_version_1809_updated_sept_2018_x64_dvd_f0b7dc68.iso \
     -device ide-cd,id=cd1,drive=drive_cd1,bootindex=2,bus=ide.0,unit=0 \
     -drive id=drive_winutils,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
     -device ide-cd,id=winutils,drive=drive_winutils,bootindex=3,bus=ide.0,unit=1 \
     -drive id=drive_unattended,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/virtio-win-1.9.6-1.el8.iso \
     -device ide-cd,id=unattended,drive=drive_unattended,bootindex=4,bus=ide.1,unit=0 \
     -vnc :0  \
     -rtc base=localtime,clock=host,driftfix=slew  \
     -boot order=cdn,once=d,menu=off,strict=off \
     -enable-kvm \
     -monitor stdio \
Comment 2 Yu Wang 2019-02-20 10:17:08 UTC 
Hi,

I can reproduce this issue on win10(1809) and cannot reproduce on win10(1803)

When I install this guest with 2 virtio-blk-pci, it will not installed successfully

For win10-1809
It shows "Drive0 unallocated space" for system disk
error as "Windows cannot be installed on the drive", refer to attachment "blk-install-1809"

For win10-1803
It shows "Drive1 unallocated space" for system disk
No error, and can install/boot guest successfully, refer to attachment "blk-install-1803"

It seems that the problem is not assign Drive0 to system disk. 

Vadim, could you help to check if it is a driver issue or qemu issue?

Thanks
Yu Wang
Comment 3 Yu Wang 2019-02-20 10:17:54 UTC 
Created attachment 1536626 [details]
blk-install-1803
Comment 4 Yu Wang 2019-02-20 10:18:47 UTC 
Created attachment 1536627 [details]
blk-install-1809
Comment 6 Vadim Rozenfeld 2019-02-26 23:43:45 UTC 
Not sure which drivers are inside of virtio-win-1.9.7-3.el8.iso
but could you please try the most recent drivers (b163 probably will be the best option)
as well as specifying serial names for both virtio-blk devices (WS2019 is quite picky 
in everything related to VPD pages)

Thanks,
Vadim.
Comment 7 lijin 2019-02-27 03:35:02 UTC 
phou, 
Could you handle comment#16?
Thanks
Comment 8 Peixiu Hou 2019-02-27 07:32:49 UTC 
Created attachment 1539053 [details]
finishing installation
Comment 9 Peixiu Hou 2019-02-27 07:33:46 UTC 
Created attachment 1539054 [details]
boot menu with 2 virtio-blk-pci devices
Comment 10 Peixiu Hou 2019-02-27 07:34:31 UTC 
Created attachment 1539055 [details]
System Reserved partition in data disk, not in system disk.
Comment 11 Peixiu Hou 2019-02-27 07:34:56 UTC 
(In reply to Vadim Rozenfeld from comment #6)
> Not sure which drivers are inside of virtio-win-1.9.7-3.el8.iso
> but could you please try the most recent drivers (b163 probably will be the
> best option)
> as well as specifying serial names for both virtio-blk devices (WS2019 is
> quite picky 
> in everything related to VPD pages)
> 
> Thanks,
> Vadim.

Hi Vadim,

Tried test with virtio-win-prewhql-163 on win2019 guest(specified serial names), also reproduced this issue.

commands as:
-drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=raw,file=/home/kvm_autotest_root/images/win2019.raw \
-device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=0x4,serial=px_system \
-drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=raw,file=/home/kvm_autotest_root/images/storage.raw \
-device virtio-blk-pci,id=stg,bootindex=1,drive=drive_stg,bus=pci.0,addr=0x5,serial=px_data \

qemu-img create -f raw /home/kvm_autotest_root/images/storage.raw 10G
qemu-img create -f raw /home/kvm_autotest_root/images/win2019.raw 30G

Additional info:
I upload 3 attachments: finishing.png, boot_menu.png, System_Reserved_to_data_disk.png.

After guest finishing installation(as finishing.png), guest restart, tried press "ESC" to boot menu, boot menu as attachment boot_menu.png;
Press first boot device "Virtio disk PCI:00:04.0", it cannot be booted in, guest booted from cdrom, restart the guest vm, press "ESC" to boot menu again;
Press second boot device "Virtio disk PCI:00:05.0", it can be booted in, the guest can finish the os installation;
Checked in guest os disk management, found System Reserved partition is shown in data disk, not in system disk, as attachment System_Reserved_to_data_disk.png

Best Regards~
Peixiu
Comment 12 Vadim Rozenfeld 2019-02-28 01:48:25 UTC 
A quick update on this bug.
I was able to reproduce this problem on Win10 1809 32bit as well.
The problem seems to be related to HW/BIOS rathere than to virtio-blk driver.
Windows refused to be installed on win2019.raw (30G) saying that this volume is not bootable,
but can be installed (partially) on storage.raw 10G. The problem with this volume is that 
it is too small for the entire installation, and windows takes only small part of it for
hosting the most critical boot time components, and making it Active. The rest goes to 
win2019.raw. But QEMU thinks that this partition (disk) is bootable, but it is not and
Windows fails to boot, which is why we enter endless installation cycle. 

I think we need to figure out why Windows is thinings that win2019.raw  partition is not
bootable first.
Comment 13 Yu Wang 2019-05-08 09:03:19 UTC 
Hit similar issue on ws2012-64(q35 chipset)
Occurred error when automatically install guests, shows "Setup ws unable to create a new system partition or locate an existing system partition"  

After restart this guest, and try to install guests again, it shows as attachment(not using the 16T disk)

Additional: 
1 only occurred when boot with Big image (16T/2T), cannot reproduce when install with 50G
2 we run on win2008R2(pc),win2012(q35),win10-32(q35),win2019(ovmf), only ws2012(q35) occurred error.
3 no error when install on ws2012(ovmf), ws 2012(pc), or with virtio-scsi driver.


version:
    virtio-win-prewhql-171
    qemu-kvm-3.1.0-22.module+el8.0.1+3032+a09688b9.x86_64
    kernel-4.18.0-80.el8.x86_64
    seabios-1.12.0-1.module+el8+2706+3c6581b6.x86_64


Boot cmd:
MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1' \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1  \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win2012-64-virtio-2.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pcie.0-root-port-3,addr=0x0 \
    -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/storage-2.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,bus=pcie.0-root-port-4,addr=0x0 \
    -m 4096  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -drive id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/ISO/Win2012/en_windows_server_2012_x64_dvd_915478.iso \
    -device ide-cd,id=cd1,drive=drive_cd1,bootindex=2,bus=ide.0,unit=0 \
    -drive id=drive_winutils,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
    -device ide-cd,id=winutils,drive=drive_winutils,bootindex=3,bus=ide.1,unit=0 \
    -drive id=drive_unattended,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-171.iso \
    -device ide-cd,id=unattended,drive=drive_unattended,bootindex=4,bus=ide.2,unit=0 \
    -vnc :2  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=d,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio
Comment 14 Yu Wang 2019-05-08 09:03:54 UTC 
Created attachment 1565540 [details]
big_image_install_fail
Comment 15 Vadim Rozenfeld 2019-08-19 03:53:44 UTC 
Window 10 1809 needs at least 16 GB for 32-bit OS and 32 GB for 64-bit OS
to be installed properly. 
Please see System requirements for installing Windows 10 at 
https://www.microsoft.com/en-au/windows/windows-10-specifications

Can we increase the the disk size and see if the problem is reproducible with 
the steps, mentioned in the bug description?

Thanks,
Vadim.
Comment 17 Yu Wang 2019-08-29 10:04:41 UTC 
(In reply to Vadim Rozenfeld from comment #15)
> Window 10 1809 needs at least 16 GB for 32-bit OS and 32 GB for 64-bit OS
> to be installed properly. 
> Please see System requirements for installing Windows 10 at 
> https://www.microsoft.com/en-au/windows/windows-10-specifications
> 
> Can we increase the the disk size and see if the problem is reproducible
> with 
> the steps, mentioned in the bug description?
> 
> Thanks,
> Vadim.

Boot cml:
 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win2012-64-virtio-2.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pcie.0-root-port-3,addr=0x0 \

    -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/data.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,bus=pcie.0-root-port-4,addr=0x0 \


I increased the system disk(win2012-64-virtio-2.qcow2) to 100G and data disk(data.qcow2) to 60G, install os to each disk. 
It shows install guest successfully, but boot to win2012-64-virtio-2.qcow2 Failed.
It shows install guest successfully, boot to win2012-64-virtio-2.qcow2 successfully.

If boot with only one disk , install os to it , it can boot to guest successfully.

If install with 1803, it can boot to guest successfully with system disk.

Note: 
it can boot successfully with installing on "Drive0 Unallocated Space" (1903) , boot failed on "Drive1 Unallocated Space" installed guest
But for 1803, it can boot "Drive1 Unallocated Space" installed guest sucessfully.

Thanks
Yu Wang

Thanks
Yu Wang
Comment 18 Yu Wang 2019-08-29 10:05:43 UTC 
It shows "Drive1 Unallocated Space" for win2012-64-virtio-2.qcow2  and "Drive0 Unallocated Space" for data.qcow2.
Comment 21 Yu Wang 2019-09-03 09:31:34 UTC 
Now the problem is Drive0 and Drive1 (tried on ws2012, win10-32(1803), win10-32(1903))

When boot with q35 chipset, the Drive0 is assigned to data disk, and Drive1 is assigned to systemdisk
When boot with pc chipset, the Drive0 is assigned to system disk, and Drive1 is assigned to datadisk
So when the data disk is not big enough , it cannot install guest to Drive0(default) with q35.

version:
virtio-win-prewhql-172
qemu-img-2.12.0-85.module+el8.1.0+4066+0f1aadab.x86_64
kernel-4.18.0-137.el8.x86_64
seabios-1.11.1-4.module+el8.1.0+4066+0f1aadab.x86_64

It is very strange that I cannot reproduce the result for comment#13 and commet#17 now.
For comment#13, it cannot install with two image booted, not related the image size now.
For comment#17, I can install and boot to guest successfully if I specified to install guest on Drive1(system disk)


Boot cmd:
MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1' \
    -machine q35  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x1  \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/test.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pcie.0-root-port-3,addr=0x0 \
    -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=storage-1.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,bus=pcie.0-root-port-4,addr=0x0 \
    -m 4096  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,+kvm_pv_unhalt \
    -drive id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/ISO/Win10/en_windows_10_business_editions_version_1903_x86_dvd_ca4f0f49.iso \
    -device ide-cd,id=cd1,drive=drive_cd1,bootindex=2,bus=ide.0,unit=0 \
    -drive id=drive_winutils,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
    -device ide-cd,id=winutils,drive=drive_winutils,bootindex=3,bus=ide.1,unit=0 \
    -drive id=drive_unattended,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-172.iso \
    -device ide-cd,id=unattended,drive=drive_unattended,bootindex=4,bus=ide.2,unit=0 \
    -vnc :2  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=d,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio
Comment 24 Peixiu Hou 2019-11-11 03:15:02 UTC 
Hit the similar issue with comment#21, Tested with q35+ovmf on win2019 guest, during install os, disk drive number is not match as expected, the Drive0 is assigned to data disk, and Drive1 is assigned to system_disk, expected drive0 should to system_disk and drive1 should to data_disk.

detail as attachment~

used versions:
kernel-4.18.0-147.el8.x86_64
qemu-kvm-4.1.0-14.module+el8.1.0+4548+ed1300f4.x86_64
edk2-ovmf-20190308git89910a39dcfd-6.el8.noarch
virtio-win-prewhql-172

Best Regards~
Peixiu
Comment 25 Peixiu Hou 2019-11-11 03:15:51 UTC 
Created attachment 1634748 [details]
drive_num is not match expected
Comment 28 Vadim Rozenfeld 2019-12-24 09:26:48 UTC 
Spent some time investigating this problem again. I don't believe that it is a bug.
This more like a kind ov inconsistency between qemu command line options, disk size and windows
requirement. A user who defines the system parameters and performs Windows installation needs
to understand and follow the Windows requirements. Otherwise  we will hit the situation
described in this bug report when the system/bootable disk is not big enough to install Windows,
while the data disk is big enough but not bootable. Which leads to the situation when windows lets
to install itself on non-bootable disk but fails to boot from this disk later on, and the only
bootable disk is the installation media, which VM tries to boot from over and over again.
Comment 29 Yu Wang 2019-12-25 07:43:52 UTC 
(In reply to Vadim Rozenfeld from comment #28)
> Spent some time investigating this problem again. I don't believe that it is
> a bug.
> This more like a kind ov inconsistency between qemu command line options,
> disk size and windows
> requirement. A user who defines the system parameters and performs Windows
> installation needs
> to understand and follow the Windows requirements. Otherwise  we will hit
> the situation
> described in this bug report when the system/bootable disk is not big enough
> to install Windows,
> while the data disk is big enough but not bootable. Which leads to the
> situation when windows lets
> to install itself on non-bootable disk but fails to boot from this disk
> later on, and the only
> bootable disk is the installation media, which VM tries to boot from over
> and over again.

Hi Vadim,

Now we hit the issue as comment#21, not "data disk is big enough but not bootable",
We create a big enough system disk , but it still shows Drive0 for smaller data disk.
And it only occurs on blk+q35 installation, not hit on blk+pc or scsi+q35.

For example, create a 16T system disk and 10G data disk

    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win2012-64r2-virtio.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,serial=SYSTEM_DISK0,bus=pcie.0-root-port-3,addr=0x0 \
    -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/storage.qcow2 \
    -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,bus=pcie.0-root-port-4,addr=0x0 \
    -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \

It shows Drive0 10G and Drive1 16T with blk+q35.
But it shows normal, Drive0 16T and Drive1 10G with blk+pc and scsi+q35.

Screenshot for installation drive, you can refer to the attachment.

So I think, we should not close this bug, it still a problem for qemu/q35/driver or something else.

Thanks
Yu Wang
Comment 30 Yu Wang 2019-12-25 07:44:49 UTC 
Created attachment 1647586 [details]
screenshot
Comment 31 Vadim Rozenfeld 2020-01-09 00:25:32 UTC 
(In reply to Yu Wang from comment #29)
> (In reply to Vadim Rozenfeld from comment #28)
> > Spent some time investigating this problem again. I don't believe that it is
> > a bug.
> > This more like a kind ov inconsistency between qemu command line options,
> > disk size and windows
> > requirement. A user who defines the system parameters and performs Windows
> > installation needs
> > to understand and follow the Windows requirements. Otherwise  we will hit
> > the situation
> > described in this bug report when the system/bootable disk is not big enough
> > to install Windows,
> > while the data disk is big enough but not bootable. Which leads to the
> > situation when windows lets
> > to install itself on non-bootable disk but fails to boot from this disk
> > later on, and the only
> > bootable disk is the installation media, which VM tries to boot from over
> > and over again.
> 
> Hi Vadim,
> 
> Now we hit the issue as comment#21, not "data disk is big enough but not
> bootable",
> We create a big enough system disk , but it still shows Drive0 for smaller
> data disk.
> And it only occurs on blk+q35 installation, not hit on blk+pc or scsi+q35.
> 
> For example, create a 16T system disk and 10G data disk
> 
>     -drive
> id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,
> file=/home/kvm_autotest_root/images/win2012-64r2-virtio.qcow2 \
>     -device
> pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
>     -device
> virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,serial=SYSTEM_DISK0,
> bus=pcie.0-root-port-3,addr=0x0 \
>     -drive
> id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/
> home/kvm_autotest_root/images/storage.qcow2 \
>     -device
> pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
>     -device
> virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,bus=pcie.0-root-port-4,
> addr=0x0 \
>     -device
> pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
> 
> It shows Drive0 10G and Drive1 16T with blk+q35.
> But it shows normal, Drive0 16T and Drive1 10G with blk+pc and scsi+q35.
> 
> Screenshot for installation drive, you can refer to the attachment.
> 
> So I think, we should not close this bug, it still a problem for
> qemu/q35/driver or something else.
> 
> Thanks
> Yu Wang

I see,
Technically, storage miniport driver doesn't deal with all this boot related stuff.
It just seats too low to make any decision on boot order. This decision is mostly between
BIOS and OS. I will try to dig a bit into this problem to see why Windows preferences 
are different in case of pc/q35, blk/scsi.

BEst,
Vadim.
Comment 32 Yu Wang 2020-01-09 07:24:05 UTC 
According to comment#31, for further tracking this issue, we will re-open it.

Thanks
Yu Wang
Comment 35 Vadim Rozenfeld 2020-05-12 07:12:54 UTC 
(In reply to Yu Wang from comment #32)
> According to comment#31, for further tracking this issue, we will re-open it.
> 
> Thanks
> Yu Wang

moving this issue to the next version (8.3.0)
Comment 37 qing.wang 2020-07-28 03:47:41 UTC 
hit same issue on
4.18.0-226.el8.x86_64
qemu-kvm-core-4.2.0-30.module+el8.3.0+7298+c26a06b8.x86_64
seabios-1.13.0-1.module+el8.3.0+6423+e4cb6418.x86_64
virtio-win-prewhql-0.1-187.iso

Test steps:
1.create images

qemu-img create -f qcow2 /home/kvm_autotest_root/images/virtio_scsi_cdrom.qcow2 30G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage.qcow2 30G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage2.qcow2 30G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage3.qcow2 30G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage4.qcow2 30G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage5.qcow2 30G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage6.qcow2 30G

2.boot vm 


 /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox on  \
    -machine q35 \
    -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
    -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x2 \
    -m 8096  \
    -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
    -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
    -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -object iothread,id=iothread0 \
    -object iothread,id=iothread1 \
    -blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/virtio_scsi_cdrom.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,write-cache=on,serial=SYSTEM_DISK0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
    \
    -blockdev node-name=file_stg,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/storage.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_stg,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg \
    -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,write-cache=on,bus=pcie-root-port-3,addr=0x0,iothread=iothread1 \
    -blockdev node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/storage2.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg2 \
    -device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,chassis=5 \
    -device virtio-blk-pci,id=stg2,drive=drive_stg2,bootindex=2,write-cache=on,bus=pcie-root-port-4,addr=0x0,iothread=iothread0 \
    -blockdev node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/storage3.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg3 \
    -device pcie-root-port,id=pcie-root-port-5,port=0x5,addr=0x1.0x5,bus=pcie.0,chassis=6 \
    -device virtio-blk-pci,id=stg3,drive=drive_stg3,bootindex=3,write-cache=on,bus=pcie-root-port-5,addr=0x0,iothread=iothread1 \
    -blockdev node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/storage4.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg4 \
    -device pcie-root-port,id=pcie-root-port-6,port=0x6,addr=0x1.0x6,bus=pcie.0,chassis=7 \
    -device virtio-blk-pci,id=stg4,drive=drive_stg4,bootindex=4,write-cache=on,bus=pcie-root-port-6,addr=0x0,iothread=iothread0 \
    -blockdev node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/images/storage5.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_stg5 \
    -device pcie-root-port,id=pcie-root-port-7,port=0x7,addr=0x1.0x7,bus=pcie.0,chassis=8 \
    -device virtio-blk-pci,id=stg5,drive=drive_stg5,bootindex=5,write-cache=on,bus=pcie-root-port-7,addr=0x0,iothread=iothread1 \
    \
    -device pcie-root-port,id=pcie-root-port-8,port=0x8,multifunction=on,bus=pcie.0,addr=0x3,chassis=9 \
    -device virtio-net-pci,mac=9a:1e:1a:be:c1:13,id=idoMHadp,netdev=idBNhvid,bus=pcie-root-port-8,addr=0x0  \
    -netdev tap,id=idBNhvid,vhost=on \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-pci-bridge-0,addr=0x1 \
    -blockdev node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/ISO/Win2019/en_windows_server_2019_updated_may_2020_x64_dvd_5651846f.iso,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
    -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=6,write-cache=on \
    -blockdev node-name=file_winutils,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_winutils,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_winutils \
    -device scsi-cd,id=winutils,drive=drive_winutils,bootindex=7,write-cache=on \
    -blockdev node-name=file_unattended,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/images/win2019-64/autounattend.iso,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_unattended,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_unattended \
    -device ide-cd,id=unattended,drive=drive_unattended,bootindex=8,write-cache=on,bus=ide.0,unit=0  \
    -vnc :5  \
    -qmp tcp:0:5955,server,nowait \
    -monitor stdio \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot menu=off,order=cdn,once=d,strict=off \
    -enable-kvm \
    -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x4,chassis=10 \


The installation failed create partition. 
It can succeed if remove other disks stg-stg5
Comment 41 Vadim Rozenfeld 2020-12-30 07:01:15 UTC 
(In reply to qing.wang from comment #37)
> hit same issue on
> 4.18.0-226.el8.x86_64
> qemu-kvm-core-4.2.0-30.module+el8.3.0+7298+c26a06b8.x86_64
> seabios-1.13.0-1.module+el8.3.0+6423+e4cb6418.x86_64
> virtio-win-prewhql-0.1-187.iso
> 
> Test steps:
> 1.create images
> 
> qemu-img create -f qcow2
> /home/kvm_autotest_root/images/virtio_scsi_cdrom.qcow2 30G
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage.qcow2 30G
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage2.qcow2 30G
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage3.qcow2 30G
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage4.qcow2 30G
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage5.qcow2 30G
> qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage6.qcow2 30G
> 
> 2.boot vm 
> 
> 
>  /usr/libexec/qemu-kvm \
>     -name 'avocado-vt-vm1'  \
>     -sandbox on  \
>     -machine q35 \
>     -device
> pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,
> chassis=1 \
>     -device
> pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
>     -nodefaults \
>     -device VGA,bus=pcie.0,addr=0x2 \
>     -m 8096  \
>     -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
>     -device
> pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,
> chassis=2 \
>     -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
>     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
>     -object iothread,id=iothread0 \
>     -object iothread,id=iothread1 \
>     -blockdev
> node-name=file_image1,driver=file,aio=threads,filename=/home/
> kvm_autotest_root/images/virtio_scsi_cdrom.qcow2,cache.direct=on,cache.no-
> flush=off \
>     -blockdev
> node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_image1 \
>     -device
> pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,
> chassis=3 \
>     -device
> virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,write-cache=on,
> serial=SYSTEM_DISK0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
>     \
>     -blockdev
> node-name=file_stg,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> images/storage.qcow2,cache.direct=on,cache.no-flush=off \
>     -blockdev
> node-name=drive_stg,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_stg \
>     -device
> pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,
> chassis=4 \
>     -device
> virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,write-cache=on,bus=pcie-
> root-port-3,addr=0x0,iothread=iothread1 \
>     -blockdev
> node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> images/storage2.qcow2,cache.direct=on,cache.no-flush=off \
>     -blockdev
> node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_stg2 \
>     -device
> pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,
> chassis=5 \
>     -device
> virtio-blk-pci,id=stg2,drive=drive_stg2,bootindex=2,write-cache=on,bus=pcie-
> root-port-4,addr=0x0,iothread=iothread0 \
>     -blockdev
> node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> images/storage3.qcow2,cache.direct=on,cache.no-flush=off \
>     -blockdev
> node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_stg3 \
>     -device
> pcie-root-port,id=pcie-root-port-5,port=0x5,addr=0x1.0x5,bus=pcie.0,
> chassis=6 \
>     -device
> virtio-blk-pci,id=stg3,drive=drive_stg3,bootindex=3,write-cache=on,bus=pcie-
> root-port-5,addr=0x0,iothread=iothread1 \
>     -blockdev
> node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> images/storage4.qcow2,cache.direct=on,cache.no-flush=off \
>     -blockdev
> node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_stg4 \
>     -device
> pcie-root-port,id=pcie-root-port-6,port=0x6,addr=0x1.0x6,bus=pcie.0,
> chassis=7 \
>     -device
> virtio-blk-pci,id=stg4,drive=drive_stg4,bootindex=4,write-cache=on,bus=pcie-
> root-port-6,addr=0x0,iothread=iothread0 \
>     -blockdev
> node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> images/storage5.qcow2,cache.direct=on,cache.no-flush=off \
>     -blockdev
> node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,
> file=file_stg5 \
>     -device
> pcie-root-port,id=pcie-root-port-7,port=0x7,addr=0x1.0x7,bus=pcie.0,
> chassis=8 \
>     -device
> virtio-blk-pci,id=stg5,drive=drive_stg5,bootindex=5,write-cache=on,bus=pcie-
> root-port-7,addr=0x0,iothread=iothread1 \
>     \
>     -device
> pcie-root-port,id=pcie-root-port-8,port=0x8,multifunction=on,bus=pcie.0,
> addr=0x3,chassis=9 \
>     -device
> virtio-net-pci,mac=9a:1e:1a:be:c1:13,id=idoMHadp,netdev=idBNhvid,bus=pcie-
> root-port-8,addr=0x0  \
>     -netdev tap,id=idBNhvid,vhost=on \
>     -device
> virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-pci-bridge-0,addr=0x1 \
>     -blockdev
> node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/
> kvm_autotest_root/iso/ISO/Win2019/
> en_windows_server_2019_updated_may_2020_x64_dvd_5651846f.iso,cache.direct=on,
> cache.no-flush=off \
>     -blockdev
> node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-
> flush=off,file=file_cd1 \
>     -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=6,write-cache=on \
>     -blockdev
> node-name=file_winutils,driver=file,read-only=on,aio=threads,filename=/home/
> kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-
> flush=off \
>     -blockdev
> node-name=drive_winutils,driver=raw,read-only=on,cache.direct=on,cache.no-
> flush=off,file=file_winutils \
>     -device
> scsi-cd,id=winutils,drive=drive_winutils,bootindex=7,write-cache=on \
>     -blockdev
> node-name=file_unattended,driver=file,read-only=on,aio=threads,filename=/
> home/kvm_autotest_root/images/win2019-64/autounattend.iso,cache.direct=on,
> cache.no-flush=off \
>     -blockdev
> node-name=drive_unattended,driver=raw,read-only=on,cache.direct=on,cache.no-
> flush=off,file=file_unattended \
>     -device
> ide-cd,id=unattended,drive=drive_unattended,bootindex=8,write-cache=on,
> bus=ide.0,unit=0  \
>     -vnc :5  \
>     -qmp tcp:0:5955,server,nowait \
>     -monitor stdio \
>     -rtc base=localtime,clock=host,driftfix=slew  \
>     -boot menu=off,order=cdn,once=d,strict=off \
>     -enable-kvm \
>     -device
> pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,
> addr=0x4,chassis=10 \
> 
> 
> The installation failed create partition. 
> It can succeed if remove other disks stg-stg5

I can reproduce this issue when Windows is giving different Disk numbers during installation 
phase and later on during "normal" boot. This is a known behaviour for Windows. Please
see the folowing link for more information 
https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-correspond-to-the-sata-channel-numbers-when
In any case, just as I've said before this is not a virtio-win drivers issue, and mostly related to the way
how the BIOS enumerates storage devices.


Best,
Vadim.
Comment 42 Vadim Rozenfeld 2020-12-30 09:53:22 UTC 
> I can reproduce this issue when Windows is giving different Disk numbers
> during installation 
> phase and later on during "normal" boot. This is a known behaviour for
> Windows. Please
> see the folowing link for more information 
> https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-
> correspond-to-the-sata-channel-numbers-when
> In any case, just as I've said before this is not a virtio-win drivers
> issue, and mostly related to the way
> how the BIOS enumerates storage devices.
> 
> 
> Best,
> Vadim.

btw, changing boot option "strict" to on "-boot menu=off,order=cdn,once=d,strict=on" together with
changing device property "bootindex" to -1 for non-system disks can limit the number of disks that Windows
treats as bootable.
Comment 44 menli@redhat.com 2021-01-04 08:25:25 UTC 
(In reply to Vadim Rozenfeld from comment #41)
> (In reply to qing.wang from comment #37)
> > hit same issue on
> > 4.18.0-226.el8.x86_64
> > qemu-kvm-core-4.2.0-30.module+el8.3.0+7298+c26a06b8.x86_64
> > seabios-1.13.0-1.module+el8.3.0+6423+e4cb6418.x86_64
> > virtio-win-prewhql-0.1-187.iso
> > 
> > Test steps:
> > 1.create images
> > 
> > qemu-img create -f qcow2
> > /home/kvm_autotest_root/images/virtio_scsi_cdrom.qcow2 30G
> > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage.qcow2 30G
> > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage2.qcow2 30G
> > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage3.qcow2 30G
> > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage4.qcow2 30G
> > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage5.qcow2 30G
> > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage6.qcow2 30G
> > 
> > 2.boot vm 
> > 
> > 
> >  /usr/libexec/qemu-kvm \
> >     -name 'avocado-vt-vm1'  \
> >     -sandbox on  \
> >     -machine q35 \
> >     -device
> > pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,
> > chassis=1 \
> >     -device
> > pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
> >     -nodefaults \
> >     -device VGA,bus=pcie.0,addr=0x2 \
> >     -m 8096  \
> >     -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
> >     -device
> > pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,
> > chassis=2 \
> >     -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
> >     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
> >     -object iothread,id=iothread0 \
> >     -object iothread,id=iothread1 \
> >     -blockdev
> > node-name=file_image1,driver=file,aio=threads,filename=/home/
> > kvm_autotest_root/images/virtio_scsi_cdrom.qcow2,cache.direct=on,cache.no-
> > flush=off \
> >     -blockdev
> > node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > file=file_image1 \
> >     -device
> > pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,
> > chassis=3 \
> >     -device
> > virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,write-cache=on,
> > serial=SYSTEM_DISK0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
> >     \
> >     -blockdev
> > node-name=file_stg,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > images/storage.qcow2,cache.direct=on,cache.no-flush=off \
> >     -blockdev
> > node-name=drive_stg,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > file=file_stg \
> >     -device
> > pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,
> > chassis=4 \
> >     -device
> > virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,write-cache=on,bus=pcie-
> > root-port-3,addr=0x0,iothread=iothread1 \
> >     -blockdev
> > node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > images/storage2.qcow2,cache.direct=on,cache.no-flush=off \
> >     -blockdev
> > node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > file=file_stg2 \
> >     -device
> > pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,
> > chassis=5 \
> >     -device
> > virtio-blk-pci,id=stg2,drive=drive_stg2,bootindex=2,write-cache=on,bus=pcie-
> > root-port-4,addr=0x0,iothread=iothread0 \
> >     -blockdev
> > node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > images/storage3.qcow2,cache.direct=on,cache.no-flush=off \
> >     -blockdev
> > node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > file=file_stg3 \
> >     -device
> > pcie-root-port,id=pcie-root-port-5,port=0x5,addr=0x1.0x5,bus=pcie.0,
> > chassis=6 \
> >     -device
> > virtio-blk-pci,id=stg3,drive=drive_stg3,bootindex=3,write-cache=on,bus=pcie-
> > root-port-5,addr=0x0,iothread=iothread1 \
> >     -blockdev
> > node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > images/storage4.qcow2,cache.direct=on,cache.no-flush=off \
> >     -blockdev
> > node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > file=file_stg4 \
> >     -device
> > pcie-root-port,id=pcie-root-port-6,port=0x6,addr=0x1.0x6,bus=pcie.0,
> > chassis=7 \
> >     -device
> > virtio-blk-pci,id=stg4,drive=drive_stg4,bootindex=4,write-cache=on,bus=pcie-
> > root-port-6,addr=0x0,iothread=iothread0 \
> >     -blockdev
> > node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > images/storage5.qcow2,cache.direct=on,cache.no-flush=off \
> >     -blockdev
> > node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > file=file_stg5 \
> >     -device
> > pcie-root-port,id=pcie-root-port-7,port=0x7,addr=0x1.0x7,bus=pcie.0,
> > chassis=8 \
> >     -device
> > virtio-blk-pci,id=stg5,drive=drive_stg5,bootindex=5,write-cache=on,bus=pcie-
> > root-port-7,addr=0x0,iothread=iothread1 \
> >     \
> >     -device
> > pcie-root-port,id=pcie-root-port-8,port=0x8,multifunction=on,bus=pcie.0,
> > addr=0x3,chassis=9 \
> >     -device
> > virtio-net-pci,mac=9a:1e:1a:be:c1:13,id=idoMHadp,netdev=idBNhvid,bus=pcie-
> > root-port-8,addr=0x0  \
> >     -netdev tap,id=idBNhvid,vhost=on \
> >     -device
> > virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-pci-bridge-0,addr=0x1 \
> >     -blockdev
> > node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/
> > kvm_autotest_root/iso/ISO/Win2019/
> > en_windows_server_2019_updated_may_2020_x64_dvd_5651846f.iso,cache.direct=on,
> > cache.no-flush=off \
> >     -blockdev
> > node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-
> > flush=off,file=file_cd1 \
> >     -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=6,write-cache=on \
> >     -blockdev
> > node-name=file_winutils,driver=file,read-only=on,aio=threads,filename=/home/
> > kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-
> > flush=off \
> >     -blockdev
> > node-name=drive_winutils,driver=raw,read-only=on,cache.direct=on,cache.no-
> > flush=off,file=file_winutils \
> >     -device
> > scsi-cd,id=winutils,drive=drive_winutils,bootindex=7,write-cache=on \
> >     -blockdev
> > node-name=file_unattended,driver=file,read-only=on,aio=threads,filename=/
> > home/kvm_autotest_root/images/win2019-64/autounattend.iso,cache.direct=on,
> > cache.no-flush=off \
> >     -blockdev
> > node-name=drive_unattended,driver=raw,read-only=on,cache.direct=on,cache.no-
> > flush=off,file=file_unattended \
> >     -device
> > ide-cd,id=unattended,drive=drive_unattended,bootindex=8,write-cache=on,
> > bus=ide.0,unit=0  \
> >     -vnc :5  \
> >     -qmp tcp:0:5955,server,nowait \
> >     -monitor stdio \
> >     -rtc base=localtime,clock=host,driftfix=slew  \
> >     -boot menu=off,order=cdn,once=d,strict=off \
> >     -enable-kvm \
> >     -device
> > pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,
> > addr=0x4,chassis=10 \
> > 
> > 
> > The installation failed create partition. 
> > It can succeed if remove other disks stg-stg5
> 
> I can reproduce this issue when Windows is giving different Disk numbers
> during installation 
> phase and later on during "normal" boot. This is a known behaviour for
> Windows. Please
> see the folowing link for more information 
> https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-
> correspond-to-the-sata-channel-numbers-when
> In any case, just as I've said before this is not a virtio-win drivers
> issue, and mostly related to the way
> how the BIOS enumerates storage devices.
> 
> 
> Best,
> Vadim.


Hi Vadim,

Thanks for your clarify again， reproduce it like comment 21, tested with q35 on win2019 guest, during install os, disk drive number is not match as expected, the Drive0 is assigned to data disk, and Drive1 is assigned to system_disk

reproduce it on build:
qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f.x86_64
virtio-win-prewhql-0.1-191.iso 

not reproduce on build:
qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64
virtio-win-prewhql-0.1-191.iso

so it not related to virtio-win drivers, may releated to qemu , I think we can move it to qemu component to have a track for moment
Comment 45 menli@redhat.com 2021-01-04 08:28:12 UTC 
(In reply to Vadim Rozenfeld from comment #42)
> > I can reproduce this issue when Windows is giving different Disk numbers
> > during installation 
> > phase and later on during "normal" boot. This is a known behaviour for
> > Windows. Please
> > see the folowing link for more information 
> > https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-
> > correspond-to-the-sata-channel-numbers-when
> > In any case, just as I've said before this is not a virtio-win drivers
> > issue, and mostly related to the way
> > how the BIOS enumerates storage devices.
> > 
> > 
> > Best,
> > Vadim.
> 
> btw, changing boot option "strict" to on "-boot
> menu=off,order=cdn,once=d,strict=on" together with
> changing device property "bootindex" to -1 for non-system disks can limit
> the number of disks that Windows
> treats as bootable.

Thanks for your suggestion, we can use it as a workaround  for moment if necessary
Comment 46 menli@redhat.com 2021-01-05 03:09:07 UTC 
(In reply to menli from comment #44)
> (In reply to Vadim Rozenfeld from comment #41)
> > (In reply to qing.wang from comment #37)
> > > hit same issue on
> > > 4.18.0-226.el8.x86_64
> > > qemu-kvm-core-4.2.0-30.module+el8.3.0+7298+c26a06b8.x86_64
> > > seabios-1.13.0-1.module+el8.3.0+6423+e4cb6418.x86_64
> > > virtio-win-prewhql-0.1-187.iso
> > > 
> > > Test steps:
> > > 1.create images
> > > 
> > > qemu-img create -f qcow2
> > > /home/kvm_autotest_root/images/virtio_scsi_cdrom.qcow2 30G
> > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage.qcow2 30G
> > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage2.qcow2 30G
> > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage3.qcow2 30G
> > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage4.qcow2 30G
> > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage5.qcow2 30G
> > > qemu-img create -f qcow2 /home/kvm_autotest_root/images/storage6.qcow2 30G
> > > 
> > > 2.boot vm 
> > > 
> > > 
> > >  /usr/libexec/qemu-kvm \
> > >     -name 'avocado-vt-vm1'  \
> > >     -sandbox on  \
> > >     -machine q35 \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,
> > > chassis=1 \
> > >     -device
> > > pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
> > >     -nodefaults \
> > >     -device VGA,bus=pcie.0,addr=0x2 \
> > >     -m 8096  \
> > >     -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,
> > > chassis=2 \
> > >     -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
> > >     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
> > >     -object iothread,id=iothread0 \
> > >     -object iothread,id=iothread1 \
> > >     -blockdev
> > > node-name=file_image1,driver=file,aio=threads,filename=/home/
> > > kvm_autotest_root/images/virtio_scsi_cdrom.qcow2,cache.direct=on,cache.no-
> > > flush=off \
> > >     -blockdev
> > > node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > > file=file_image1 \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,
> > > chassis=3 \
> > >     -device
> > > virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,write-cache=on,
> > > serial=SYSTEM_DISK0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
> > >     \
> > >     -blockdev
> > > node-name=file_stg,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > > images/storage.qcow2,cache.direct=on,cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_stg,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > > file=file_stg \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,
> > > chassis=4 \
> > >     -device
> > > virtio-blk-pci,id=stg,drive=drive_stg,bootindex=1,write-cache=on,bus=pcie-
> > > root-port-3,addr=0x0,iothread=iothread1 \
> > >     -blockdev
> > > node-name=file_stg2,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > > images/storage2.qcow2,cache.direct=on,cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_stg2,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > > file=file_stg2 \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,
> > > chassis=5 \
> > >     -device
> > > virtio-blk-pci,id=stg2,drive=drive_stg2,bootindex=2,write-cache=on,bus=pcie-
> > > root-port-4,addr=0x0,iothread=iothread0 \
> > >     -blockdev
> > > node-name=file_stg3,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > > images/storage3.qcow2,cache.direct=on,cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_stg3,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > > file=file_stg3 \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-5,port=0x5,addr=0x1.0x5,bus=pcie.0,
> > > chassis=6 \
> > >     -device
> > > virtio-blk-pci,id=stg3,drive=drive_stg3,bootindex=3,write-cache=on,bus=pcie-
> > > root-port-5,addr=0x0,iothread=iothread1 \
> > >     -blockdev
> > > node-name=file_stg4,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > > images/storage4.qcow2,cache.direct=on,cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_stg4,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > > file=file_stg4 \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-6,port=0x6,addr=0x1.0x6,bus=pcie.0,
> > > chassis=7 \
> > >     -device
> > > virtio-blk-pci,id=stg4,drive=drive_stg4,bootindex=4,write-cache=on,bus=pcie-
> > > root-port-6,addr=0x0,iothread=iothread0 \
> > >     -blockdev
> > > node-name=file_stg5,driver=file,aio=threads,filename=/home/kvm_autotest_root/
> > > images/storage5.qcow2,cache.direct=on,cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_stg5,driver=qcow2,cache.direct=on,cache.no-flush=off,
> > > file=file_stg5 \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-7,port=0x7,addr=0x1.0x7,bus=pcie.0,
> > > chassis=8 \
> > >     -device
> > > virtio-blk-pci,id=stg5,drive=drive_stg5,bootindex=5,write-cache=on,bus=pcie-
> > > root-port-7,addr=0x0,iothread=iothread1 \
> > >     \
> > >     -device
> > > pcie-root-port,id=pcie-root-port-8,port=0x8,multifunction=on,bus=pcie.0,
> > > addr=0x3,chassis=9 \
> > >     -device
> > > virtio-net-pci,mac=9a:1e:1a:be:c1:13,id=idoMHadp,netdev=idBNhvid,bus=pcie-
> > > root-port-8,addr=0x0  \
> > >     -netdev tap,id=idBNhvid,vhost=on \
> > >     -device
> > > virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-pci-bridge-0,addr=0x1 \
> > >     -blockdev
> > > node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/
> > > kvm_autotest_root/iso/ISO/Win2019/
> > > en_windows_server_2019_updated_may_2020_x64_dvd_5651846f.iso,cache.direct=on,
> > > cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-
> > > flush=off,file=file_cd1 \
> > >     -device scsi-cd,id=cd1,drive=drive_cd1,bootindex=6,write-cache=on \
> > >     -blockdev
> > > node-name=file_winutils,driver=file,read-only=on,aio=threads,filename=/home/
> > > kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-
> > > flush=off \
> > >     -blockdev
> > > node-name=drive_winutils,driver=raw,read-only=on,cache.direct=on,cache.no-
> > > flush=off,file=file_winutils \
> > >     -device
> > > scsi-cd,id=winutils,drive=drive_winutils,bootindex=7,write-cache=on \
> > >     -blockdev
> > > node-name=file_unattended,driver=file,read-only=on,aio=threads,filename=/
> > > home/kvm_autotest_root/images/win2019-64/autounattend.iso,cache.direct=on,
> > > cache.no-flush=off \
> > >     -blockdev
> > > node-name=drive_unattended,driver=raw,read-only=on,cache.direct=on,cache.no-
> > > flush=off,file=file_unattended \
> > >     -device
> > > ide-cd,id=unattended,drive=drive_unattended,bootindex=8,write-cache=on,
> > > bus=ide.0,unit=0  \
> > >     -vnc :5  \
> > >     -qmp tcp:0:5955,server,nowait \
> > >     -monitor stdio \
> > >     -rtc base=localtime,clock=host,driftfix=slew  \
> > >     -boot menu=off,order=cdn,once=d,strict=off \
> > >     -enable-kvm \
> > >     -device
> > > pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,
> > > addr=0x4,chassis=10 \
> > > 
> > > 
> > > The installation failed create partition. 
> > > It can succeed if remove other disks stg-stg5
> > 
> > I can reproduce this issue when Windows is giving different Disk numbers
> > during installation 
> > phase and later on during "normal" boot. This is a known behaviour for
> > Windows. Please
> > see the folowing link for more information 
> > https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-
> > correspond-to-the-sata-channel-numbers-when
> > In any case, just as I've said before this is not a virtio-win drivers
> > issue, and mostly related to the way
> > how the BIOS enumerates storage devices.
> > 
> > 
> > Best,
> > Vadim.
> 
> 
> Hi Vadim,
> 
> Thanks for your clarify again， reproduce it like comment 21, tested with q35
> on win2019 guest, during install os, disk drive number is not match as
> expected, the Drive0 is assigned to data disk, and Drive1 is assigned to
> system_disk
> 
> reproduce it on build:
> qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f.x86_64
> virtio-win-prewhql-0.1-191.iso 
seabios-1.14.0-1.module+el8.3.0+7638+07cf13d2.x86_64

> not reproduce on build:
> qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64
> virtio-win-prewhql-0.1-191.iso
seabios-1.14.0-1.module+el8.4.0+8855+a9e237a9.x86_64

> so it not related to virtio-win drivers, may releated to qemu , I think we
> can move it to qemu component to have a track for moment
Comment 47 Vadim Rozenfeld 2021-01-05 04:27:17 UTC 
> > Hi Vadim,
> > 
> > Thanks for your clarify again， reproduce it like comment 21, tested with q35
> > on win2019 guest, during install os, disk drive number is not match as
> > expected, the Drive0 is assigned to data disk, and Drive1 is assigned to
> > system_disk
> > 
> > reproduce it on build:
> > qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f.x86_64
> > virtio-win-prewhql-0.1-191.iso 
> seabios-1.14.0-1.module+el8.3.0+7638+07cf13d2.x86_64
> 
> > not reproduce on build:
> > qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64
> > virtio-win-prewhql-0.1-191.iso
> seabios-1.14.0-1.module+el8.4.0+8855+a9e237a9.x86_64
> 
> > so it not related to virtio-win drivers, may releated to qemu , I think we
> > can move it to qemu component to have a track for moment

Probably not related to seabios, but to qemu, because both seabios builds
seabios-1.14.0-1.module+el8.3.0+7638+07cf13d2
( https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1284694 )
and
seabios-1.14.0-1.module+el8.4.0+8855+a9e237a9
( https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1392976 )
look absolute the same in terms of code base.

Best,
Vadim.
Comment 48 menli@redhat.com 2021-01-05 09:09:51 UTC 
(In reply to Vadim Rozenfeld from comment #47)
> > > Hi Vadim,
> > > 
> > > Thanks for your clarify again， reproduce it like comment 21, tested with q35
> > > on win2019 guest, during install os, disk drive number is not match as
> > > expected, the Drive0 is assigned to data disk, and Drive1 is assigned to
> > > system_disk
> > > 
> > > reproduce it on build:
> > > qemu-kvm-5.1.0-4.module+el8.3.0+7846+ae9b566f.x86_64
> > > virtio-win-prewhql-0.1-191.iso 
> > seabios-1.14.0-1.module+el8.3.0+7638+07cf13d2.x86_64
> > 
> > > not reproduce on build:
> > > qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64
> > > virtio-win-prewhql-0.1-191.iso
> > seabios-1.14.0-1.module+el8.4.0+8855+a9e237a9.x86_64
> > 
> > > so it not related to virtio-win drivers, may releated to qemu , I think we
> > > can move it to qemu component to have a track for moment
> 
> Probably not related to seabios, but to qemu, because both seabios builds
> seabios-1.14.0-1.module+el8.3.0+7638+07cf13d2
> ( https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1284694 )
> and
> seabios-1.14.0-1.module+el8.4.0+8855+a9e237a9
> ( https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1392976 )
> look absolute the same in terms of code base.
> 
> Best,
> Vadim.

keep driver&qemu same, try with different seabios :
qemu-kvm-5.2.0-2.module+el8.4.0+9186+ec44380f.x86_64
virtio-win-prewhql-0.1-191.iso

seabios-1.14.0-1.module+el8.4.0+8855+a9e237a9.x86_64  and (seabios-1.14.0-1.module+el8.3.0+7638+07cf13d2)

both not reproduce this issue, so it may not related to seabios
Comment 49 lijin 2021-01-05 10:55:58 UTC 
After confirm with Vadim in weekly meeting, we prefer to close this bugs according to comment#41.

Pls use workaroudn in comment#42 to avoid this issue in the future.
Comment 50 menli@redhat.com 2021-01-07 08:23:36 UTC 
(In reply to Vadim Rozenfeld from comment #42)
> > I can reproduce this issue when Windows is giving different Disk numbers
> > during installation 
> > phase and later on during "normal" boot. This is a known behaviour for
> > Windows. Please
> > see the folowing link for more information 
> > https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-
> > correspond-to-the-sata-channel-numbers-when
> > In any case, just as I've said before this is not a virtio-win drivers
> > issue, and mostly related to the way
> > how the BIOS enumerates storage devices.
> > 
> > 
> > Best,
> > Vadim.
> 
> btw, changing boot option "strict" to on "-boot
> menu=off,order=cdn,once=d,strict=on" together with
> changing device property "bootindex" to -1 for non-system disks can limit
> the number of disks that Windows
> treats as bootable.

Hi Vadim，
I want to confirm something about this workaround, after I add  property "bootindex" to -1 for data disk together with strict=on
the guest boot failed like in the attachment, in my understanding it seems not work, please correct me if my misunderstanding.

Additional info：
I try to set bootindex=1 for system disk win2019.qcow2 and bootindex=2 for data disk storage.qcow2 when strict=on, the guest boot fail.


 /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm3' \
    -machine q35 \
    -nodefaults \
    -vga std  \
    -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \
    -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
    -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
    -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
    -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
    -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
    -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \
    -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/test/win2019.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.2,bootindex=1  \
    -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/test/storage.qcow2 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=-1,bus=pci.4 \
    -device virtio-net-pci,mac=9a:36:83:b6:3d:05,id=idJVpmsF,netdev=id23ZUK6,bus=pci.3  \
    -netdev tap,id=id23ZUK6,vhost=on \
    -m 14336  \
    -smp 2,maxcpus=4 \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/ISO/Win2019/en_windows_server_2019_updated_march_2019_x64_dvd_2ae967ab.iso \
    -device ide-cd,id=cd2,drive=drive_cd1,bus=ide.0,unit=0 \
    -cdrom /home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-191.iso \
    -device piix3-usb-uhci,id=usb -device usb-tablet,id=input0 \
    -vnc :1  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot menu=off,order=cdn,once=d,strict=on \
    -enable-kvm \
    -qmp tcp:0:1232,server,nowait \
    -monitor stdio \

Thanks
Menghuan
Comment 51 menli@redhat.com 2021-01-07 08:25:00 UTC 
Created attachment 1745215 [details]
boot fail
Comment 52 menli@redhat.com 2021-01-07 09:10:38 UTC 
(In reply to menli from comment #50)
> (In reply to Vadim Rozenfeld from comment #42)
> > > I can reproduce this issue when Windows is giving different Disk numbers
> > > during installation 
> > > phase and later on during "normal" boot. This is a known behaviour for
> > > Windows. Please
> > > see the folowing link for more information 
> > > https://support.microsoft.com/en-us/help/937251/disk-drive-numbers-may-not-
> > > correspond-to-the-sata-channel-numbers-when
> > > In any case, just as I've said before this is not a virtio-win drivers
> > > issue, and mostly related to the way
> > > how the BIOS enumerates storage devices.
> > > 
> > > 
> > > Best,
> > > Vadim.
> > 
> > btw, changing boot option "strict" to on "-boot
> > menu=off,order=cdn,once=d,strict=on" together with
> > changing device property "bootindex" to -1 for non-system disks can limit
> > the number of disks that Windows
> > treats as bootable.
> 
> Hi Vadim，
> I want to confirm something about this workaround, after I add  property
> "bootindex" to -1 for data disk together with strict=on
> the guest boot failed like in the attachment, in my understanding it seems
> not work, please correct me if my misunderstanding.
> 
> Additional info：
> I try to set bootindex=1 for system disk win2019.qcow2 and bootindex=2 for
> data disk storage.qcow2 when strict=on, the guest boot fail.
> 
> 
>  /usr/libexec/qemu-kvm \
>     -name 'avocado-vt-vm3' \
>     -machine q35 \
>     -nodefaults \
>     -vga std  \
>     -device
> pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,
> addr=0x2 \
>     -device
> pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
>     -device
> pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
>     -device
> pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
>     -device
> pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
>     -device
> pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
>     -device
> pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \
>     -device
> pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 \
>     -drive
> id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,
> file=/home/test/win2019.qcow2 \
>     -device
> virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.2,bootindex=1  \
>     -drive
> id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/
> home/test/storage.qcow2 \
>     -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=-1,bus=pci.4 \
>     -device
> virtio-net-pci,mac=9a:36:83:b6:3d:05,id=idJVpmsF,netdev=id23ZUK6,bus=pci.3  \
>     -netdev tap,id=id23ZUK6,vhost=on \
>     -m 14336  \
>     -smp 2,maxcpus=4 \
>     -cpu
> 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,
> hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
>     -drive
> id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/
> home/kvm_autotest_root/iso/ISO/Win2019/
> en_windows_server_2019_updated_march_2019_x64_dvd_2ae967ab.iso \
>     -device ide-cd,id=cd2,drive=drive_cd1,bus=ide.0,unit=0 \
>     -cdrom
> /home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-191.iso \
>     -device piix3-usb-uhci,id=usb -device usb-tablet,id=input0 \
>     -vnc :1  \
>     -rtc base=localtime,clock=host,driftfix=slew  \
>     -boot menu=off,order=cdn,once=d,strict=on \
>     -enable-kvm \
>     -qmp tcp:0:1232,server,nowait \
>     -monitor stdio \
> 
> Thanks
> Menghuan


I try to add the booindex to cdrom(above result without booindex in cdrom) , guest can boot, and data disk be Offline status like in the attachment（bootindex=-1.png）, whether this result is expected?

 /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm3' \
    -machine q35 \
    -nodefaults \
    -vga std  \
    -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \
    -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
    -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
    -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
    -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
    -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
    -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \
    -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/test/win2019.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.2,bootindex=1  \
    -drive id=drive_stg,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/home/test/storage.qcow2 \
    -device virtio-blk-pci,id=stg,drive=drive_stg,bootindex=-1,bus=pci.4 \
    -device virtio-net-pci,mac=9a:36:83:b6:3d:05,id=idJVpmsF,netdev=id23ZUK6,bus=pci.3  \
    -netdev tap,id=id23ZUK6,vhost=on \
    -m 14336  \
    -smp 2,maxcpus=4 \
    -cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt \
    -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/ISO/Win2019/en_windows_server_2019_updated_march_2019_x64_dvd_2ae967ab.iso \
    -device ide-cd,id=cd2,drive=drive_cd1,bus=ide.0,unit=0,bootindex=2 \
    -cdrom /home/kvm_autotest_root/iso/windows/virtio-win-prewhql-0.1-191.iso \
    -device piix3-usb-uhci,id=usb -device usb-tablet,id=input0 \
    -vnc :1  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot menu=off,order=cdn,once=d,strict=on \
    -enable-kvm \
    -qmp tcp:0:1232,server,nowait \
    -monitor stdio \

Thanks

Menghuan
Comment 53 menli@redhat.com 2021-01-07 09:11:40 UTC 
Created attachment 1745229 [details]
data disk offline
Comment 54 Vadim Rozenfeld 2021-01-08 00:22:43 UTC 
(In reply to menli from comment #53)
> Created attachment 1745229 [details]
> data disk offline

Yes, I think this is fine. What doe's it say when you you click on 
:Show details"? Something like "disabled by Administrator" or any different
reasom?

Thanks,
Vadim.
Comment 55 menli@redhat.com 2021-01-08 02:07:40 UTC 
Created attachment 1745473 [details]
show details
Comment 56 menli@redhat.com 2021-01-08 02:08:59 UTC 
Created attachment 1745474 [details]
continue show details
Comment 57 menli@redhat.com 2021-01-08 02:10:47 UTC 
(In reply to Vadim Rozenfeld from comment #54)
> (In reply to menli from comment #53)
> > Created attachment 1745229 [details]
> > data disk offline
> 
> Yes, I think this is fine. What doe's it say when you you click on 
> :Show details"? Something like "disabled by Administrator" or any different
> reasom?
> 
> Thanks,
> Vadim.

when click 'Show details'  it shows like attachment  'show details.png', continue click 'ok' shows like attachment 'show details2.png'


Thanks
Menghuan