Bug 1716352

Summary: unhotplug virtio-net-pci NIC cause qemu process Segmentation fault
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: FuXiangChun <xfu>
Component: qemu-kvmAssignee: ybendito
qemu-kvm sub component: PCI QA Contact: Lei Yang <leiyang>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: ailan, chayang, ddepaula, jinzhao, jsuvorov, jusual, juzhang, leiyang, lijin, michen, mjahoda, mrezanin, pezhang, rbalakri, virt-maint, wyu, ybendito, yfu, yvugenfi
Version: 8.1   
Target Milestone: rc   
Target Release: 8.0   
Hardware: x86_64   
OS: Windows   
Whiteboard:
Fixed In Version: qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-17 17:44:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1744438, 1771318    

Description FuXiangChun 2019-06-03 09:59:04 UTC 
Description of problem:
Hotplug multiple virtio Nics to win2019 guest. Then unhotplug them. sometimes cause qemu process core dump( Segmentation fault)

Version-Release number of selected component (if applicable):

qemu-kvm: qemu-kvm-4.0.0-3.module+el8.1.0+3265+26c4ed71.x86_64
kernel: kernel-4.18.0-94.el8.x86_64
spice: spice-server-0.14.2-1.el8.x86_64
seabios: seabios-bin-1.12.0-1.module+el8.1.0+3258+4c45705b.noarch
seavgabios: seavgabios-bin-1.12.0-1.module+el8.1.0+3258+4c45705b.noarch
edk2: edk2-ovmf-20190308git89910a39dcfd-1.el8.noarch
sgabios: sgabios-bin-0.20170427git-2.module+el8.1.0+3258+4c45705b.noarch
ipxe: ipxe-roms-qemu-20181214-1.git133f4c47.el8.noarch
virtio-win: virtio-win-1.9.8-1.el8.iso

How reproducible:
sometimes

Steps to Reproduce:
1. Boot win2019 guest.

/usr/libexec/qemu-kvm -name 'avocado-vt-vm1' -machine q35 -nodefaults -device qxl-vga,bus=pcie.0,addr=0x1 -device intel-hda,bus=pcie.0,addr=0x2 -device hda-duplex -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_euqtdlr5/monitor-qmpmonitor1-20190530-152000-V6wO6iZB,server,nowait -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_euqtdlr5/monitor-catch_monitor-20190530-152000-V6wO6iZB,server,nowait -mon chardev=qmp_id_catch_monitor,mode=control -device pvpanic,ioport=0x505,id=id7laEDt -chardev socket,id=serial_id_serial0,path=/var/tmp/avocado_euqtdlr5/serial-serial0-20190530-152000-V6wO6iZB,server,nowait -device isa-serial,chardev=serial_id_serial0 -device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 -device virtio-serial-pci,id=virtio_serial_pci0,bus=pcie.0-root-port-3,addr=0x0 -chardev socket,path=/var/tmp/avocado_euqtdlr5/virtio_port-vs-20190530-152000-V6wO6iZB,nowait,server,id=idi2Y17T -device virtserialport,id=idGappu0,name=vs,bus=virtio_serial_pci0.0,chardev=idi2Y17T -object rng-random,filename=/dev/random,id=passthrough-Vfe6gEjm -device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 -device virtio-rng-pci,id=virtio-rng-pci-y5Nw8lYe,rng=passthrough-Vfe6gEjm,bus=pcie.0-root-port-4,addr=0x0 -chardev socket,id=seabioslog_id_20190530-152000-V6wO6iZB,path=/var/tmp/avocado_euqtdlr5/seabios-20190530-152000-V6wO6iZB,server,nowait -device isa-debugcon,chardev=seabioslog_id_20190530-152000-V6wO6iZB,iobase=0x402 -device ich9-usb-ehci1,id=usb1,addr=0x1d.7,multifunction=on,bus=pcie.0 -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=0x1d.0,firstport=0,bus=pcie.0 -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=0x1d.2,firstport=2,bus=pcie.0 -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=0x1d.4,firstport=4,bus=pcie.0 -device pcie-root-port,id=pcie.0-root-port-5,slot=5,chassis=5,addr=0x5,bus=pcie.0 -device qemu-xhci,id=usb2,bus=pcie.0-root-port-5,addr=0x0 -device pcie-root-port,id=pcie.0-root-port-6,slot=6,chassis=6,addr=0x6,bus=pcie.0 -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-6,addr=0x0 -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/win2019-64-virtio-scsi.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -m 30720 -smp 40,maxcpus=40,cores=20,threads=1,sockets=2 -cpu 'IvyBridge',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt -drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso -device scsi-cd,id=cd1,drive=drive_cd1 -device usb-tablet,id=usb-tablet1,bus=usb2.0,port=1 -spice port=3005,password=123456,addr=0,tls-port=3381,x509-dir=/tmp/spice_x509d,tls-channel=main,tls-channel=inputs,image-compression=auto_glz,zlib-glz-wan-compression=auto,streaming-video=all,agent-mouse=on,playback-compression=on,ipv4 -rtc base=localtime,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -net none -no-hpet -enable-kvm -watchdog i6300esb -watchdog-action reset -device pcie-root-port,id=pcie.0-root-port-7,slot=7,chassis=7,addr=0x7,bus=pcie.0 -device virtio-balloon-pci,id=balloon0,bus=pcie.0-root-port-7,addr=0x0 -device pcie-root-port,id=pcie_extra_root_port_0,slot=8,chassis=8,addr=0x8,bus=pcie.0 -device pcie-root-port,id=pcie_extra_root_port_1,slot=9,chassis=9,addr=0x9,bus=pcie.0 -device pcie-root-port,id=pcie_extra_root_port_2,slot=10,chassis=10,addr=0xa,bus=pcie.0 -device pcie-root-port,id=pcie_extra_root_port_3,slot=11,chassis=11,addr=0xb,bus=pcie.0 -vnc :2 -monitor stdio -monitor unix:/tmp/monitor2,server,nowait

2.hotplug and un hotplug virtio-net-pci with this script.

i=1
while [ $i -lt 1000000 ]
do
echo "**************$i**************"
sleep 3
echo "netdev_add type=tap,id=idqBoFx0"|nc -U /tmp/monitor2
sleep 3
echo "device_add driver=virtio-net-pci,netdev=idqBoFx0,mac=9a:d5:d6:d7:d8:d9,id=hotplug_nic1,vectors=4,bus=pcie_extra_root_port_0"|nc -U /tmp/monitor2
sleep 5
echo "netdev_add type=tap,id=idqg6rD9"|nc -U /tmp/monitor2
sleep 3
echo "device_add driver=virtio-net-pci,netdev=idqg6rD9,mac=9a:da:db:dc:dd:de,id=hotplug_nic2,vectors=4,bus=pcie_extra_root_port_1"|nc -U /tmp/monitor2
sleep 5
echo "netdev_add type=tap,id=idSC5OSV"|nc -U /tmp/monitor2
sleep 3
echo "device_add driver=virtio-net-pci,netdev=idSC5OSV,mac=9a:df:e0:e1:e2:e3,id=hotplug_nic3,vectors=4,bus=pcie_extra_root_port_2"|nc -U /tmp/monitor2
sleep 5
echo "netdev_add type=tap,id=idXYXqMl"|nc -U /tmp/monitor2
sleep 3
echo "device_add driver=virtio-net-pci,netdev=idXYXqMl,mac=9a:e4:e5:e6:e7:e8,id=hotplug_nic4,vectors=4,bus=pcie_extra_root_port_3"|nc -U /tmp/monitor2

sleep 5
echo "device_del hotplug_nic1"|nc -U /tmp/monitor2
sleep 3
echo "netdev_del idqBoFx0"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2

echo "device_del hotplug_nic2"|nc -U /tmp/monitor2
sleep 3
echo "netdev_del idqg6rD9"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2

echo "device_del hotplug_nic3"|nc -U /tmp/monitor2
sleep 3
echo "netdev_del idSC5OSV"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2

echo "device_del hotplug_nic4"|nc -U /tmp/monitor2
sleep 3
echo "netdev_del idXYXqMl"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2
echo "system_reset"|nc -U /tmp/monitor2
sleep 60
i=$(($i+1))
done

3.

Actual results:
qemu process Segmentation fault.

dmesg:
qemu-kvm[27033]: segfault at 28 ip 0000563ee7058b7d sp 00007f69cd3fe490 error 4 in qemu-kvm[563ee6b41000+a0a000]

Expected results:
works

Additional info:
will test Linux guest and e100e Nic. and update test result to bz asap.
Comment 2 Yanan Fu 2019-06-03 10:34:20 UTC 
Here is the virtio netkvm driver version:  DriverVer = 04/12/2019,100.77.104.17100
Comment 3 FuXiangChun 2019-06-04 06:34:05 UTC 
Re-tested this 2 scenarios.

e1000e + win2019 guest-->works.
virtio-net-pci+RHEL.8.1.0 guest-->works
Comment 6 Lei Yang 2019-12-02 08:36:14 UTC 
Hi,Julia

The core dump file from comment 1 can be accessed now,please tell me know if you need others test.

Best regards,
LeiYang.
Comment 7 ybendito 2019-12-22 15:26:59 UTC 
How many iterations (approx) needed to reproduce the problem?
Please provide a console log from script side (using tee, for example)
Comment 8 Lei Yang 2019-12-24 01:33:15 UTC 
(In reply to ybendito from comment #7)
> How many iterations (approx) needed to reproduce the problem?
> Please provide a console log from script side (using tee, for example)

Hi

I can not reproduce this issue because of an new issue.

Test Version:
kernel-4.18.0-147.4.1.el8_1.x86_64
qemu-kvm-4.1.0-19.module+el8.1.1+5172+e3ff58a1.x86_64
virtio-win-prewhql-0.1-172.iso

==>Test Steps

1.Boot a guest.
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-machine q35  \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x1 \
-m 14336  \
-smp 16,maxcpus=16,cores=8,threads=1,sockets=2  \
-cpu 'EPYC',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt  \
-device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
-device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
-device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0 \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2 \
-device scsi-hd,id=image1,drive=drive_image1 \
-device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
-drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
-device scsi-cd,id=cd1,drive=drive_cd1 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
-vnc :0  \
-rtc base=localtime,clock=host,driftfix=slew  \
-boot order=cdn,once=c,menu=off,strict=off \
-enable-kvm \
-qmp tcp:0:5555,server,nowait \
-device pcie-root-port,id=pcie_extra_root_port_0,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_1,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_4,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_5,slot=8,chassis=8,addr=0x8,bus=pcie.0 \
-monitor stdio \

2.hotplug a virtio-net nic.
# telnet 10.73.196.43 5555
{"execute":"qmp_capabilities"}
{"return": {}}
{'execute': 'netdev_add', 'arguments': {'type': 'tap', 'id': 'idJxmmIZ','vhost':'on'}}
{"return": {}}
{'execute': 'device_add', 'arguments': {'driver': 'virtio-net-pci', 'netdev': 'idJxmmIZ', 'mac': '9a:90:e8:73:1c:72', 'id': 'idguH3SC', 'bus': 'pcie_extra_root_port_0'}}
{"return": {}}
{"timestamp": {"seconds": 1575271737, "microseconds": 753018}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "idguH3SC", "path": "/machine/peripheral/idguH3SC/virtio-backend"}}

3.Hot unplug the nic (only one "DEVICE_DELETE" event return).
{'execute': 'device_del', 'arguments': {'id':'idguH3SC'}}
{"return": {}}
{"timestamp": {"seconds": 1575271884, "microseconds": 833278}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/idguH3SC/virtio-backend"}}
{'execute': 'netdev_del', 'arguments': {'id':'idJxmmIZ'}}
{"return": {}}

4.Hotplug the nic again failed.

{'execute': 'netdev_add', 'arguments': {'type': 'tap', 'id': 'idJxmmIZ','vhost':'on'}}
{"return": {}}
{'execute': 'device_add', 'arguments': {'driver': 'virtio-net-pci', 'netdev': 'idJxmmIZ', 'mac': '9a:90:e8:73:1c:72', 'id': 'idguH3SC', 'bus': 'pcie_extra_root_port_0'}}
{"error": {"class": "GenericError", "desc": "Duplicate ID 'idguH3SC' for device"}}

Best regards
LeiYang
Comment 14 Lei Yang 2019-12-31 00:46:41 UTC 
Hit similar problem

Test Version:
kernel-4.18.0-147.4.1.el8_1.x86_64
qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64
virtio-win-prewhql-0.1-172.iso

How reproducible:
always

Steps to Reproduce:
1,Boot guest
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-machine q35  \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x1 \
-m 7168  \
-smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
-cpu 'Haswell-noTSX',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt  \
-device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
-device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
-device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0 \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2 \
-device scsi-hd,id=image1,drive=drive_image1 \
-device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
-drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
-device scsi-cd,id=cd1,drive=drive_cd1 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
-vnc :2  \
-rtc base=localtime,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device pcie-root-port,id=pcie_extra_root_port_0,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_1,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_4,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_5,slot=8,chassis=8,addr=0x8,bus=pcie.0 \
-monitor stdio \
-monitor unix:/tmp/monitor2,server,nowait \

2.hotplug and un hotplug virtio-net-pci with this script.
i=1
while [ $i -lt 1000000 ] 
do
echo "**************$i**************"
sleep 2
echo "netdev_add type=tap,id=net$i,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown"|nc -U /tmp/monitor2
sleep 1
echo "device_add driver=virtio-net-pci,netdev=net$i,mac=9a:d5:d6:d7:d8:d9,id=dev$i,bus=pcie_extra_root_port_0"|nc -U /tmp/monitor2
sleep 5
echo "device_del dev$i"|nc -U /tmp/monitor2
sleep 10
echo "netdev_del net$i"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2
sleep 3
echo "system_reset"|nc -U /tmp/monitor2
sleep 60
i=$(($i+1))

Actual results:
qemu process Segmentation fault.

dmesg:
qemu-kvm[7760]: segfault at 28 ip 000056471f64438d sp 00007f68b91fe4a0 error 4 in qemu-kvm[56471f120000+a31000]

core dump file:
# gdb /var/cordump.7748
(gdb) bt full
#0  0x000056471f64438d in virtio_pci_notify_write (opaque=0x564722322630, addr=8, val=<optimized out>, size=<optimized out>)
    at hw/virtio/virtio-pci.c:1306
        vdev = 0x564722322630
        proxy = <optimized out>
        __func__ = "virtio_pci_notify_write"
        queue = <optimized out>
#1  0x000056471f473b13 in memory_region_write_accessor
    (mr=<optimized out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>, shift=<optimized out>, mask=<optimized out>, attrs=...) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/memory.c:507
        tmp = <optimized out>
#2  0x000056471f471d26 in access_with_adjusted_size
    (addr=addr@entry=8, value=value@entry=0x7f68b91fe558, size=size@entry=2, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=access_fn@entry=0x56471f473ac0 <memory_region_write_accessor>, mr=0x56472231b190, attrs=...)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/memory.c:573
        access_mask = 65535
        access_size = 2
        i = <optimized out>
        r = 0
#3  0x000056471f475d00 in memory_region_dispatch_write (mr=0x56472231b190, addr=8, data=<optimized out>, size=2, attrs=...)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/memory.c:1509
#4  0x000056471f422ca3 in flatview_write_continue
    (fv=0x7f68b00a1960, addr=4234162184, attrs=..., buf=0x7f68cdc11028 <error: Cannot access memory at address 0x7f68cdc11028>, len=2, addr1=<optimized out>, l=<optimized out>, mr=0x56472231b190)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3369
        ptr = <optimized out>
        val = <optimized out>
        result = 0
        release_lock = true
#5  0x000056471f422ec6 in flatview_write
--Type <RET> for more, q to quit, c to continue without paging--
    (fv=0x7f68b00a1960, addr=4234162184, attrs=..., buf=0x7f68cdc11028 <error: Cannot access memory at address 0x7f68cdc11028>, len=2) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3408
        l = 2
        addr1 = 8
        mr = <optimized out>
        result = 0
#6  0x000056471f42725a in address_space_write
    (as=<optimized out>, addr=<optimized out>, attrs=..., buf=<optimized out>, len=<optimized out>)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3498
        result = 0
        fv = <optimized out>
#7  0x000056471f4849ba in kvm_cpu_exec (cpu=<optimized out>)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/accel/kvm/kvm-all.c:2316
        attrs = 
          {unspecified = 0, secure = 0, user = 0, requester_id = 0, target_tlb_bit0 = 0, target_tlb_bit1 = 0, target_tlb_bit2 = 0}
        run = <optimized out>
        ret = <optimized out>
        run_ret = 0
#8  0x000056471f4699fe in qemu_kvm_cpu_thread_fn (arg=0x564721f38240)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/cpus.c:1285
        cpu = 0x564721f38240
        r = <optimized out>
#9  0x000056471f78b954 in qemu_thread_start (args=0x564721f5b190) at util/qemu-thread-posix.c:502
        __clframe = {__cancel_routine = <optimized out>, __cancel_arg = 0x0, __do_it = 1, __cancel_type = <optimized out>}
        qemu_thread_args = 0x564721f5b190
        start_routine = 0x56471f469940 <qemu_kvm_cpu_thread_fn>
        arg = 0x564721f38240
        r = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
#10 0x00007f68c85732de in start_thread () at /lib64/libpthread.so.0
#11 0x00007f68c82a4133 in clone () at /lib64/libc.so.6

Expected results:
works well

Additional info:
e1000e + win2019 guest -->works well
virtio-net-pci + rhel8.1.1 guest -->works well
Comment 15 FuXiangChun 2020-01-09 01:52:33 UTC 
e1000e + win2019 hit the same issue. 

10:03:58 DEBUG| Send command: {'execute': 'device_del', 'arguments': {'id': 'iddzysxb'}, 'id': 'PVDq1PO6'}
10:03:58 INFO | waiting for the guest to finish the unplug
10:04:04 INFO | [qemu output] qemu-kvm: hw/net/net_tx_pkt.c:382: net_tx_pkt_add_raw_fragment: Assertion `pkt->max_raw_frags > pkt->raw_frags' failed.
10:04:38 WARNI| registers is not alive. Can't query the avocado-vt-vm1 status
10:04:42 INFO | [qemu output] /tmp/aexpect_I0kEz5tm/aexpect-r2dp88m1.sh: line 1: 14881 Aborted                 (core dumped) MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -sandbox on

qemu-kvm and kernel version as below.

4.18.0-147.3.1.el8_1.x86_64
qemu-kvm-core-4.1.0-21.module+el8.1.1+5388+fd51bfbc.x86_64
Comment 20 Ademar Reis 2020-02-05 22:58:32 UTC 
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks
Comment 23 Yvugenfi@redhat.com 2020-03-01 11:18:42 UTC 
Upstream commit: b952544fe8a061f0c0cccfd50a58220bc6ac94da
Comment 24 Yvugenfi@redhat.com 2020-03-03 10:09:22 UTC 
Sorry, the right commit is d945d9f1731244ef341f74ede93120fc9de35913
Comment 26 Lei Yang 2020-03-12 00:53:07 UTC 
Hi,Julia

I tried test on the qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64,reproduce this bug.

Test Version:
qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64
kernel-4.18.0-187.el8.x86_64
virtio-win-prewhql-0.1-180.iso

Steps to Reproduce:
1.Boot win2019 guest
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-machine q35  \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x1 \
-m 7168  \
-smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
-cpu 'Haswell-noTSX',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt  \
-device pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
-device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
-device pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0 \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2 \
-device scsi-hd,id=image1,drive=drive_image1 \
-device pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
-drive id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/home/kvm_autotest_root/iso/windows/winutils.iso \
-device scsi-cd,id=cd1,drive=drive_cd1 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
-vnc :2  \
-rtc base=localtime,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device pcie-root-port,id=pcie_extra_root_port_0,slot=5,chassis=5,addr=0x5,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_1,slot=6,chassis=6,addr=0x6,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_4,slot=7,chassis=7,addr=0x7,bus=pcie.0 \
-device pcie-root-port,id=pcie_extra_root_port_5,slot=8,chassis=8,addr=0x8,bus=pcie.0 \
-monitor stdio \
-monitor unix:/tmp/monitor2,server,nowait \

2.Hotplug and un hotplug virtio-net-pci with this script.
i=1
while [ $i -lt 1000000 ] 
do
echo "**************$i**************"
sleep 2
echo "netdev_add type=tap,id=net$i,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown"|nc -U /tmp/monitor2
sleep 1
echo "device_add driver=virtio-net-pci,netdev=net$i,mac=9a:d5:d6:d7:d8:d9,id=dev$i,bus=pcie_extra_root_port_0"|nc -U /tmp/monitor2
sleep 5
echo "device_del dev$i"|nc -U /tmp/monitor2
sleep 10
echo "netdev_del net$i"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2
sleep 3
echo "system_reset"|nc -U /tmp/monitor2
sleep 60
i=$(($i+1))
done

3.core dump file.
# gdb /var/virtualization/core-dumps/core.qemu-kvm.131529.dell-per730-29.lab.eng.pek2.redhat.com.1583930572
(gdb) bt full
#0  0x000055b5e3c120dd in virtio_pci_notify_write
    (opaque=0x55b5e66a71b0, addr=8, val=<optimized out>, size=<optimized out>)
    at hw/virtio/virtio-pci.c:1315
        vdev = 0x55b5e66a71b0
        proxy = <optimized out>
        __func__ = "virtio_pci_notify_write"
        queue = <optimized out>
#1  0x000055b5e3a3f177 in memory_region_write_accessor
    (mr=<optimized out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>, shift=<optimized out>, mask=<optimized out>, attrs=...)
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/memory.c:483
        tmp = <optimized out>
#2  0x000055b5e3a3d3ae in access_with_adjusted_size
    (addr=addr@entry=8, value=value@entry=0x7f3ea51fe508, size=size@entry=2, access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=
    0x55b5e3a3f100 <memory_region_write_accessor>, mr=0x55b5e669fd10, attrs=...)
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/memory.c:544
        access_mask = 65535
        access_size = 2
        i = <optimized out>
        r = 0
#3  0x000055b5e3a4128c in memory_region_dispatch_write
    (mr=0x55b5e669fd10, addr=8, data=<optimized out>, op=<optimized out>, attrs=...)
--Type <RET> for more, q to quit, c to continue without paging--
   mu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/memory.c:1475
        size = 2
#4  0x000055b5e39ee317 in flatview_write_continue
    (fv=0x7f3e94f24890, addr=4234162184, attrs=..., buf=0x7f3ec2fa8028 <error: Cannot access memory at address 0x7f3ec2fa8028>, len=2, addr1=<optimized out>, l=<optimized out>, mr=0x55b5e669fd10)
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/include/qemu/host-utils.h:164
        ptr = <optimized out>
        val = <optimized out>
        result = 0
        release_lock = true
#5  0x000055b5e39ee536 in flatview_write
    (fv=0x7f3e94f24890, addr=4234162184, attrs=..., buf=0x7f3ec2fa8028 <error: Cannot access memory at address 0x7f3ec2fa8028>, len=2)
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/exec.c:3169
        l = 2
        addr1 = 8
        mr = <optimized out>
        result = 0
#6  0x000055b5e39f2a4f in address_space_write ()
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/exec.c:3259
#7  0x000055b5e3a5016a in kvm_cpu_exec (cpu=<optimized out>)
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/accel/kvm/kvm-all.c:2386
--Type <RET> for more, q to quit, c to continue without paging--
        attrs = 
          {unspecified = 0, secure = 0, user = 0, requester_id = 0, byte_swap = 0, target_tlb_bit0 = 0, target_tlb_bit1 = 0, target_tlb_bit2 = 0}
        run = <optimized out>
        ret = <optimized out>
        run_ret = 0
#8  0x000055b5e3a34fde in qemu_kvm_cpu_thread_fn (arg=0x55b5e62498a0)
    at /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/cpus.c:1318
        cpu = 0x55b5e62498a0
        r = <optimized out>
#9  0x000055b5e3d5e5a4 in qemu_thread_start (args=0x55b5e6270c10) at util/qemu-thread-posix.c:519
        __clframe = 
          {__cancel_routine = <optimized out>, __cancel_arg = 0x0, __do_it = 1, __cancel_type = <optimized out>}
        qemu_thread_args = 0x55b5e6270c10
        start_routine = 0x55b5e3a34f20 <qemu_kvm_cpu_thread_fn>
        arg = 0x55b5e62498a0
        r = <optimized out>
#10 0x00007f3ebdf002de in start_thread () at /lib64/libpthread.so.0
#11 0x00007f3ebdc31e83 in clone () at /lib64/libc.so.6

So i think this bz have no fixed included in qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64.Could you provid a new qemu package?

Best regards
LeiYang
Comment 27 Chao Yang 2020-03-16 03:36:56 UTC 
Per Comment 26
Comment 28 Julia Suvorova 2020-03-16 18:48:25 UTC 
(In reply to Lei Yang from comment #26)
> Hi,Julia
> 
> I tried test on the
> qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64,reproduce this bug.
> 
> Test Version:
> qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64
> kernel-4.18.0-187.el8.x86_64
> virtio-win-prewhql-0.1-180.iso
> 
> Steps to Reproduce:
> 1.Boot win2019 guest
> /usr/libexec/qemu-kvm \
> -name 'avocado-vt-vm1' \
> -machine q35  \
> -nodefaults \
> -device VGA,bus=pcie.0,addr=0x1 \
> -m 7168  \
> -smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
> -cpu
> 'Haswell-noTSX',hv_stimer,hv_synic,hv_vpindex,hv_reset,hv_relaxed,
> hv_spinlocks=0x1fff,hv_vapic,hv_time,hv-tlbflush,+kvm_pv_unhalt  \
> -device
> pcie-root-port,id=pcie.0-root-port-2,slot=2,chassis=2,addr=0x2,bus=pcie.0 \
> -device qemu-xhci,id=usb1,bus=pcie.0-root-port-2,addr=0x0 \
> -device
> pcie-root-port,id=pcie.0-root-port-3,slot=3,chassis=3,addr=0x3,bus=pcie.0 \
> -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0-root-port-3,addr=0x0 \
> -drive
> id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,
> file=/home/kvm_autotest_root/images/win2019-64-virtio-scsi.qcow2 \
> -device scsi-hd,id=image1,drive=drive_image1 \
> -device
> pcie-root-port,id=pcie.0-root-port-4,slot=4,chassis=4,addr=0x4,bus=pcie.0 \
> -drive
> id=drive_cd1,if=none,snapshot=off,aio=threads,cache=none,media=cdrom,file=/
> home/kvm_autotest_root/iso/windows/winutils.iso \
> -device scsi-cd,id=cd1,drive=drive_cd1 \
> -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
> -vnc :2  \
> -rtc base=localtime,clock=host,driftfix=slew  \
> -boot menu=off,order=cdn,once=c,strict=off \
> -enable-kvm \
> -device
> pcie-root-port,id=pcie_extra_root_port_0,slot=5,chassis=5,addr=0x5,bus=pcie.
> 0 \
> -device
> pcie-root-port,id=pcie_extra_root_port_1,slot=6,chassis=6,addr=0x6,bus=pcie.
> 0 \
> -device
> pcie-root-port,id=pcie_extra_root_port_4,slot=7,chassis=7,addr=0x7,bus=pcie.
> 0 \
> -device
> pcie-root-port,id=pcie_extra_root_port_5,slot=8,chassis=8,addr=0x8,bus=pcie.
> 0 \
> -monitor stdio \
> -monitor unix:/tmp/monitor2,server,nowait \
> 
> 2.Hotplug and un hotplug virtio-net-pci with this script.
> i=1
> while [ $i -lt 1000000 ] 
> do
> echo "**************$i**************"
> sleep 2
> echo "netdev_add
> type=tap,id=net$i,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown"|nc -U
> /tmp/monitor2
> sleep 1
> echo "device_add
> driver=virtio-net-pci,netdev=net$i,mac=9a:d5:d6:d7:d8:d9,id=dev$i,
> bus=pcie_extra_root_port_0"|nc -U /tmp/monitor2
> sleep 5
> echo "device_del dev$i"|nc -U /tmp/monitor2
> sleep 10
> echo "netdev_del net$i"|nc -U /tmp/monitor2
> sleep 10
> echo "info network"|nc -U /tmp/monitor2
> sleep 3
> echo "system_reset"|nc -U /tmp/monitor2
> sleep 60
> i=$(($i+1))
> done
> 
> 3.core dump file.
> # gdb
> /var/virtualization/core-dumps/core.qemu-kvm.131529.dell-per730-29.lab.eng.
> pek2.redhat.com.1583930572
> (gdb) bt full
> #0  0x000055b5e3c120dd in virtio_pci_notify_write
>     (opaque=0x55b5e66a71b0, addr=8, val=<optimized out>, size=<optimized
> out>)
>     at hw/virtio/virtio-pci.c:1315
>         vdev = 0x55b5e66a71b0
>         proxy = <optimized out>
>         __func__ = "virtio_pci_notify_write"
>         queue = <optimized out>
> #1  0x000055b5e3a3f177 in memory_region_write_accessor
>     (mr=<optimized out>, addr=<optimized out>, value=<optimized out>,
> size=<optimized out>, shift=<optimized out>, mask=<optimized out>, attrs=...)
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/memory.
> c:483
>         tmp = <optimized out>
> #2  0x000055b5e3a3d3ae in access_with_adjusted_size
>     (addr=addr@entry=8, value=value@entry=0x7f3ea51fe508, size=size@entry=2,
> access_size_min=<optimized out>, access_size_max=<optimized out>, access_fn=
>     0x55b5e3a3f100 <memory_region_write_accessor>, mr=0x55b5e669fd10,
> attrs=...)
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/memory.
> c:544
>         access_mask = 65535
>         access_size = 2
>         i = <optimized out>
>         r = 0
> #3  0x000055b5e3a4128c in memory_region_dispatch_write
>     (mr=0x55b5e669fd10, addr=8, data=<optimized out>, op=<optimized out>,
> attrs=...)
> --Type <RET> for more, q to quit, c to continue without paging--
>    mu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/memory.c:1475
>         size = 2
> #4  0x000055b5e39ee317 in flatview_write_continue
>     (fv=0x7f3e94f24890, addr=4234162184, attrs=..., buf=0x7f3ec2fa8028
> <error: Cannot access memory at address 0x7f3ec2fa8028>, len=2,
> addr1=<optimized out>, l=<optimized out>, mr=0x55b5e669fd10)
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/include/
> qemu/host-utils.h:164
>         ptr = <optimized out>
>         val = <optimized out>
>         result = 0
>         release_lock = true
> #5  0x000055b5e39ee536 in flatview_write
>     (fv=0x7f3e94f24890, addr=4234162184, attrs=..., buf=0x7f3ec2fa8028
> <error: Cannot access memory at address 0x7f3ec2fa8028>, len=2)
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/exec.c:
> 3169
>         l = 2
>         addr1 = 8
>         mr = <optimized out>
>         result = 0
> #6  0x000055b5e39f2a4f in address_space_write ()
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/exec.c:
> 3259
> #7  0x000055b5e3a5016a in kvm_cpu_exec (cpu=<optimized out>)
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/accel/
> kvm/kvm-all.c:2386
> --Type <RET> for more, q to quit, c to continue without paging--
>         attrs = 
>           {unspecified = 0, secure = 0, user = 0, requester_id = 0,
> byte_swap = 0, target_tlb_bit0 = 0, target_tlb_bit1 = 0, target_tlb_bit2 = 0}
>         run = <optimized out>
>         ret = <optimized out>
>         run_ret = 0
> #8  0x000055b5e3a34fde in qemu_kvm_cpu_thread_fn (arg=0x55b5e62498a0)
>     at
> /usr/src/debug/qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64/cpus.c:
> 1318
>         cpu = 0x55b5e62498a0
>         r = <optimized out>
> #9  0x000055b5e3d5e5a4 in qemu_thread_start (args=0x55b5e6270c10) at
> util/qemu-thread-posix.c:519
>         __clframe = 
>           {__cancel_routine = <optimized out>, __cancel_arg = 0x0, __do_it =
> 1, __cancel_type = <optimized out>}
>         qemu_thread_args = 0x55b5e6270c10
>         start_routine = 0x55b5e3a34f20 <qemu_kvm_cpu_thread_fn>
>         arg = 0x55b5e62498a0
>         r = <optimized out>
> #10 0x00007f3ebdf002de in start_thread () at /lib64/libpthread.so.0
> #11 0x00007f3ebdc31e83 in clone () at /lib64/libc.so.6
> 
> So i think this bz have no fixed included in
> qemu-kvm-4.2.0-11.module+el8.2.0+5837+4c1442ec.x86_64.Could you provid a new
> qemu package?


No, the package is right. I guess we need to investigate this bug further as the upstream
commit d945d9f173 is not enough.

Best regards, Julia Suvorova.
Comment 29 FuXiangChun 2020-03-18 05:33:20 UTC 
I update priority to urgent due to the following reasons, I repeated  many times(>30 times) when I filed this bz but now I could easily reproduce(once) this bz w/o repetition by using latest qemu-kvm build(qemu-kvm-4.2.0-15.module+el8.2.0+6029+618ef2ec.x86_64). The following are test steps.


1) Boot win2019 guest

2) hotplug tap via qmp command

{"execute": "netdev_add", "arguments": {"type": "tap", "id": "id2GW53D", "fd": 50}, "id": "TGpYp8l2"}

result:

QEMU 4.2.0 monitor - type 'help' for more information
(qemu) qemu-kvm: util/oslib-posix.c:247: qemu_set_nonblock: Assertion `f != -1' failed.
Comment 30 Julia Suvorova 2020-03-19 12:38:20 UTC 
(In reply to FuXiangChun from comment #29)
> I update priority to urgent due to the following reasons, I repeated  many
> times(>30 times) when I filed this bz but now I could easily reproduce(once)
> this bz w/o repetition by using latest qemu-kvm
> build(qemu-kvm-4.2.0-15.module+el8.2.0+6029+618ef2ec.x86_64). The following
> are test steps.
> 
> 
> 1) Boot win2019 guest
> 
> 2) hotplug tap via qmp command
> 
> {"execute": "netdev_add", "arguments": {"type": "tap", "id": "id2GW53D",
> "fd": 50}, "id": "TGpYp8l2"}
> 
> result:
> 
> QEMU 4.2.0 monitor - type 'help' for more information
> (qemu) qemu-kvm: util/oslib-posix.c:247: qemu_set_nonblock: Assertion `f !=
> -1' failed.

This is not related to this bug. Seems like something happened with file descriptor that you specified.
(In case it's not just a typo - "fd" takes strings ("50", not 50). Luckily, this raises qmp error since upstream's db2a380c84).

In case we need to track new problem, create another bugzilla, please.

Best regards, Julia Suvorova.
Comment 31 FuXiangChun 2020-03-20 07:32:25 UTC 
(In reply to Julia Suvorova from comment #30)
> (In reply to FuXiangChun from comment #29)
> > I update priority to urgent due to the following reasons, I repeated  many
> > times(>30 times) when I filed this bz but now I could easily reproduce(once)
> > this bz w/o repetition by using latest qemu-kvm
> > build(qemu-kvm-4.2.0-15.module+el8.2.0+6029+618ef2ec.x86_64). The following
> > are test steps.
> > 
> > 
> > 1) Boot win2019 guest
> > 
> > 2) hotplug tap via qmp command
> > 
> > {"execute": "netdev_add", "arguments": {"type": "tap", "id": "id2GW53D",
> > "fd": 50}, "id": "TGpYp8l2"}
> > 
> > result:
> > 
> > QEMU 4.2.0 monitor - type 'help' for more information
> > (qemu) qemu-kvm: util/oslib-posix.c:247: qemu_set_nonblock: Assertion `f !=
> > -1' failed.
> 
> This is not related to this bug. Seems like something happened with file
> descriptor that you specified.
> (In case it's not just a typo - "fd" takes strings ("50", not 50). Luckily,
> this raises qmp error since upstream's db2a380c84).
> 
> In case we need to track new problem, create another bugzilla, please.
> 
> Best regards, Julia Suvorova.

Thanks Julia,  I filed a new bz1815399 to track this issue.
Comment 35 Lei Yang 2020-07-25 09:28:01 UTC 
Hit same issue.
Test Version:
qemu-kvm-5.1.0-0.scrmod+el8.3.0+7384+2e5aeafb.wrb200716.x86_64
kernel-4.18.0-227.el8.x86_64
virtio-win-prewhql-0.1-185.iso
Comment 36 Yvugenfi@redhat.com 2020-07-27 07:55:37 UTC 
https://lists.gnu.org/archive/html/qemu-stable/2020-07/msg00037.html
Comment 43 Lei Yang 2020-08-27 13:17:33 UTC 
==Steps
Test Version:
qemu-kvm-4.0.0-3.module+el8.1.0+3265+26c4ed71.x86_64
kernel-4.18.0-100.el8.x86_64
virtio-win-prewhql-0.1-189.iso

1,Boot win2019 guest
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1'  \
-sandbox on  \
-machine q35 \
-device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
-device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x2 \
-m 6G  \
-smp 12,maxcpus=12,cores=6,threads=1,sockets=2  \
-cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0xfff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_ipi,+kvm_pv_unhalt \
-device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
-device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0 \
-blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/win2019-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
-device scsi-hd,id=image1,drive=drive_image1,write-cache=on \
-device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
-blockdev node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
-device scsi-cd,id=cd1,drive=drive_cd1,write-cache=on  \
-vnc :0  \
-rtc base=localtime,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 \
-monitor stdio \
-monitor unix:/tmp/monitor2,server,nowait \

2.Hotplug and un hotplug virtio-net-pci with this script.
i=1
while [ $i -lt 2000 ] 
do
echo "**************$i**************"
sleep 2
echo "netdev_add type=tap,id=net$i,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown"|nc -U /tmp/monitor2
sleep 1
echo "device_add driver=virtio-net-pci,netdev=net$i,mac=9a:d5:d6:d7:d8:d9,id=dev$i,bus=pcie_extra_root_port_0"|nc -U /tmp/monitor2
sleep 5
echo "device_del dev$i"|nc -U /tmp/monitor2
sleep 10
echo "netdev_del net$i"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2
sleep 3
echo "system_reset"|nc -U /tmp/monitor2
sleep 60
i=$(($i+1))
done

3.guest core dump
qemu-kvm[9425]: segfault at 28 ip 0000563348169b7d sp 00007f5577dfe490 error 4 in qemu-kvm[563347c52000+a0a000]

==Reproduced with qemu-kvm-4.0.0-3.module+el8.1.0+3265+26c4ed71.x86_64

==Verified with qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901.x86_64
Test Version:
qemu-kvm-5.1.0-2.module+el8.3.0+7652+b30e6901.x86_64
kernel-4.18.0-234.el8.x86_64
virtio-win-prewhql-0.1-189.iso

1,Boot win2019 guest
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1'  \
-sandbox on  \
-machine q35 \
-device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
-device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
-nodefaults \
-device VGA,bus=pcie.0,addr=0x2 \
-m 6G  \
-smp 12,maxcpus=12,cores=6,threads=1,sockets=2  \
-cpu 'Skylake-Server',hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0xfff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_ipi,+kvm_pv_unhalt \
-device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
-device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0 \
-blockdev node-name=file_image1,driver=file,aio=threads,filename=/home/win2019-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
-device scsi-hd,id=image1,drive=drive_image1,write-cache=on \
-device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
-blockdev node-name=file_cd1,driver=file,read-only=on,aio=threads,filename=/home/kvm_autotest_root/iso/windows/winutils.iso,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_cd1,driver=raw,read-only=on,cache.direct=on,cache.no-flush=off,file=file_cd1 \
-device scsi-cd,id=cd1,drive=drive_cd1,write-cache=on  \
-vnc :0  \
-rtc base=localtime,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 \
-monitor stdio \
-monitor unix:/tmp/monitor2,server,nowait \

2.Hotplug and un hotplug virtio-net-pci with this script.
i=1
while [ $i -lt 2000 ] 
do
echo "**************$i**************"
sleep 2
echo "netdev_add type=tap,id=net$i,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown"|nc -U /tmp/monitor2
sleep 1
echo "device_add driver=virtio-net-pci,netdev=net$i,mac=9a:d5:d6:d7:d8:d9,id=dev$i,bus=pcie_extra_root_port_0"|nc -U /tmp/monitor2
sleep 5
echo "device_del dev$i"|nc -U /tmp/monitor2
sleep 10
echo "netdev_del net$i"|nc -U /tmp/monitor2
sleep 10
echo "info network"|nc -U /tmp/monitor2
sleep 3
echo "system_reset"|nc -U /tmp/monitor2
sleep 60
i=$(($i+1))
done

3. Guest works well.So this bug has been fixed very well. Move to 'VERIFIED'.
Comment 46 errata-xmlrpc 2020-11-17 17:44:46 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5137