Created attachment 1629704 [details] logs - hlk Description of problem: Version-Release number of selected component (if applicable): qemu-kvm-4.1.0-13.module+el8.1.0+4313+ef76ec61.x86_64 kernel-4.18.0-147.el8.x86_64 How reproducible: 100% Steps to Reproduce: 1. boot sut with two e1000 nics: /usr/libexec/qemu-kvm \ -name SVVP_RHEL8_AMD_SUT \ -M pc \ -cpu EPYC-IBPB,hv_stimer,hv_synic,hv_time,hv_relaxed,hv_vpindex,hv_spinlocks=0xfff,hv_vapic,hv_reset,hv_tlbflush \ -enable-kvm -nodefaults \ -m 64G \ -smp 32 \ -k en-us \ -monitor stdio \ -boot menu=on \ -device piix3-usb-uhci,id=usb \ -device usb-tablet,id=tablet0 \ -uuid 6a533967-50be-4277-a90e-e9a676815ae7 \ -smbios type=1,manufacturer="Red Hat",product="RHEL OpenStack Platform Version 8 Fast Train & RHEV Version 8",version=8Server,serial="14F00000-BF99-11E1-BAD6-80 C85 BB28 B80",uuid=35d8ddd5-4e70-49df-938f-87dfa6b12e41 \ -rtc base=localtime,clock=host,driftfix=slew \ -object iothread,id=thread0 \ -drive file=win2019.qcow2,if=none,format=qcow2,cache=none,werror=stop,rerror=stop,id=drive-virtio-disk0 \ -device virtio-blk-pci,scsi=off,iothread=thread0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \ -drive file=/home/kvm_autotest_root/iso/ISO/Win2019/en_windows_server_2019_x64_dvd_4cb967d8.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw \ -device ide-drive,drive=drive-ide0-1-0,id=ide0-1-0 \ -cdrom /usr/share/virtio-win/virtio-win.iso \ -netdev tap,script=/etc/qemu-ifup1,id=hostnet1 -device e1000,netdev=hostnet1,id=net1,mac=00:52:33:52:36:01 \ -netdev tap,script=/etc/qemu-ifup1,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:e2:52:36:54:05 \ -vnc 0.0.0.0:3 \ -vga std \ -device usb-ehci,id=ehci0 \ -drive file=usb-disk-2.raw,if=none,id=drive-usb-2-0,media=disk,format=raw,cache=none,werror=stop,rerror=stop,aio=threads \ -device usb-storage,bus=ehci0.0,drive=drive-usb-2-0,id=usb-2-0,removable=on \ 2. boot debug vm with two e1000 nics: /usr/libexec/qemu-kvm \ -name SVVP_RHEL8_AMD_DEBUG \ -M pc \ -cpu EPYC-IBPB,hv_stimer,hv_synic,hv_time,hv_relaxed,hv_vpindex,hv_spinlocks=0xfff,hv_vapic,hv_reset \ -enable-kvm -nodefaults \ -m 2G \ -smp 2 \ -k en-us \ -monitor stdio \ -qmp tcp:0:4234,server,nowait \ -boot menu=on \ -device piix3-usb-uhci,id=usb \ -device usb-tablet,id=tablet0 \ -uuid df3bf2c6-9a26-45de-88d3-f5dceb3cc127 \ -rtc base=localtime,clock=host,driftfix=slew \ -object iothread,id=thread0 \ -drive file=debug.qcow2,if=none,format=qcow2,cache=none,werror=stop,rerror=stop,id=drive-virtio-disk0,aio=native \ -device virtio-blk-pci,scsi=off,iothread=thread0,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \ -drive file=/home/kvm_autotest_root/iso/ISO/Win2019/en_windows_server_2019_x64_dvd_4cb967d8.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw \ -device ide-drive,drive=drive-ide0-1-0,id=ide0-1-0 \ -cdrom /usr/share/virtio-win/virtio-win.iso \ -vnc 0.0.0.0:4 \ -vga std \ -netdev tap,script=/etc/qemu-ifup1,id=hostnet1 -device e1000,netdev=hostnet1,id=net1,mac=00:e2:52:20:13:06 \ -netdev tap,script=/etc/qemu-ifup1,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=00:e2:52:20:13:05 \ 3. submit job in HLK, submit prameters are: BusParams 0.5.0 NetHostIP 192.168.0.18 ---> debug machine ip NetKey this.key.isnt.secure NetPort 50000 Actual results: Job always fails at subtask "KernelDebugLogoTestSetup.DebugSetupVerifier.DebugSetupVerify": System.Exception: Kernel debugging over the network fails due to NIC hardware initialization failed. error code -1073741438.Please address the problem or run the test from another debug connection. at KernelDebugLogoTestSetup.DebugSetupVerifier.DebugSetupVerify() Expected results: job can pass Additional info: Try with q35 and e1000e, still hit this issue.
Created attachment 1629706 [details] logs - html
Hi Amnon, Yan, This bug is basically same bug with bz1090237. As bz1090237 was closed as CANTFIX, QE use serial transport during running "Debug Capability test (Logo)" from rhel7.0 release. But we now are running svpp on CNV, and connect two vm with serial transport is NOT supported on CNV. So we must use net transport. If this bug is still CANTFIX, could you help to request manual errata from msft? Thanks.
The difference between this bz and bz1090237 is that vm nics can get ip normally after first reboot.
Hi Lijin, Can you try the test with e1000e? Is this accepted configuration CNV?
(In reply to Yan Vugenfirer from comment #4) > Hi Lijin, > > Can you try the test with e1000e? Is this accepted configuration CNV? As I mentioned in comment#0, q35+e1000e also hit this issue. And CNV accept q35+e1000e
> But we now are running svpp on CNV, and connect two vm with serial transport is NOT supported on CNV. So we must use net transport. Li, you can connect to two VMs with serial console at the same time. But we do not support ATM to have two serial consoles on one VM. Li, what would be needed to make it work for you?
(In reply to Fabian Deutsch from comment #7) > > But we now are running svpp on CNV, and connect two vm with serial transport is NOT supported on CNV. So we must use net transport. > > Li, you can connect to two VMs with serial console at the same time. > But we do not support ATM to have two serial consoles on one VM. > > Li, what would be needed to make it work for you? Li Jin is pto until Nov 11th. I will let her have a try and add comment once she comes back. I will keep Lijin's needinfo. Best regards, Junyi
(In reply to Fabian Deutsch from comment #7) > > But we now are running svpp on CNV, and connect two vm with serial transport is NOT supported on CNV. So we must use net transport. > > Li, you can connect to two VMs with serial console at the same time. > But we do not support ATM to have two serial consoles on one VM. > > Li, what would be needed to make it work for you? I was told cnv does not support connect two vm via serial port. For rhel svvp, we boot vm with qemu cli like this, one works as a server, the other is client: for target vm: "-serial tcp:<debug_host_ip>:4444" for debug vm: "-serial tcp:0:4444,server" On cnv vm, I see one isa-serial and one virtserialport, both work as servers: -chardev socket,id=charserial0,fd=34,server,nowait \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev socket,id=charchannel0,fd=35,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \ I'm not sure "connect to two VMs with serial console" will work, I will try run this case with serial console in cnv2.2 svvp testing to see if it can pass. And what the case does is basically same in https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/setting-up-a-null-modem-cable-connection Thanks
IIUIC you want to: Create a connection between two VMs via a serial console. Is this correct? If yes, then: This is correct, we do not support anything like this. Can you please explain the problem why network transprot can not be used?
(In reply to Fabian Deutsch from comment #12) > IIUIC you want to: Create a connection between two VMs via a serial console. > Is this correct? > > If yes, then: This is correct, we do not support anything like this. Yes. > Can you please explain the problem why network transprot can not be used? network transport should work, we reported similar bug(bz1090237) long time ago and it's closed as can't fix. That's why we use serial transport to pass the case from then. Now in order to pass cnv svvp certification, we need to either fix this issue in qemu or request manual errata from msft to waive this case. the detailed failure logs are attached, it always complains "NIC hardware initialization failed", Yan is investigating the root cause.
Hi Martin, Thanks for the quick response, Adding Fabian to answer it. Fabian can you answer Martin question which RHEL-AV to use?
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks