Bug 949972

Summary: Fail to initialize virtio_serialport/virtio_console with multiple virtio-serial-pci
Product: [Fedora] Fedora Reporter: Lukáš Doktor <ldoktor>
Component: virtio-winAssignee: Gal Hammer <ghammer>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: amit.shah, bcao, berrange, cfergeau, crobinso, dwmw2, ghammer, itamar, juzhang, ldoktor, pbonzini, qzhang, rhod, rjones, scottt.tw, virt-maint, vrozenfe, yvugenfi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: virtio-win-prewhql-0.1-67 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-03 00:28:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
qemu coredump when using 8 virtio ports spread across 4 virtio-serial-pcis
none
qemu-1.5.0 coredump when using 8 virtio ports spread across 4 virtio-serial-pcis on RHEL6 none

Description Lukáš Doktor 2013-04-09 11:54:31 UTC 
Description of problem:
When I start VM with Windows, 4 virtserialports and 4 virtconsoles spread across multiple virtio-serial-pcis, not all are present in Windows \\.\.

Version-Release number of selected component (if applicable):
Host: Fedora 17, upstream qemu-1.4.0, smp4
Guest: WindowsXP.i386.sp3, vio-0.1.58(pre-whql), smp4

How reproducible:
Always

Steps to Reproduce:
1. Execute:
/usr/local/bin/qemu-system-x86_64 \
    -S \
    -name 'virt-tests-vm1' \
    -nodefaults \
    -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20130409-082021-mIYrdOnT,server,nowait \
    -mon chardev=hmp_id_hmp1,mode=readline \
    -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130409-082021-mIYrdOnT,server,nowait \
    -device isa-serial,chardev=serial_id_serial1 \
    -device virtio-serial-pci,id=virtio_serial_pci0 \
    -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs1,name=vs1,id=vs1,bus=virtio_serial_pci0.0 \
    -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs2,name=vs2,id=vs2,bus=virtio_serial_pci0.0 \
    -device virtio-serial-pci,id=virtio_serial_pci1 \
    -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs3,name=vs3,id=vs3,bus=virtio_serial_pci1.0 \
    -chardev socket,id=devvs4,path=/tmp/virtio_port-vs4-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs4,name=vs4,id=vs4,bus=virtio_serial_pci1.0 \
    -device virtio-serial-pci,id=virtio_serial_pci2 \
    -chardev socket,id=devvs5,path=/tmp/virtio_port-vs5-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs5,name=vs5,id=vs5,bus=virtio_serial_pci2.0 \
    -chardev socket,id=devvs6,path=/tmp/virtio_port-vs6-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs6,name=vs6,id=vs6,bus=virtio_serial_pci2.0 \
    -device virtio-serial-pci,id=virtio_serial_pci3 \
    -chardev socket,id=devvs7,path=/tmp/virtio_port-vs7-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs7,name=vs7,id=vs7,bus=virtio_serial_pci3.0 \
    -chardev socket,id=devvs8,path=/tmp/virtio_port-vs8-20130409-082021-mIYrdOnT,server,nowait \
    -device virtserialport,chardev=devvs8,name=vs8,id=vs8,bus=virtio_serial_pci3.0 \
    -chardev socket,id=seabioslog_id_20130409-082021-mIYrdOnT,path=/tmp/seabios-20130409-082021-mIYrdOnT,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20130409-082021-mIYrdOnT,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1 \
    -drive file='/home/medic/Work/Projekty/autotest/autotest/client/tests/virt/shared/data/images/winXP-32.qcow2',index=0,if=ide \
    -device virtio-net-pci,netdev=idtzEvOA,mac='9a:7a:7b:7c:7d:7e',id='ideMrIjp' \
    -netdev tap,id=idtzEvOA,fd=16 \
    -m 1024 \
    -smp 4,maxcpus=4,cores=1,threads=1,sockets=4 \
    -cpu 'SandyBridge' \
    -M pc \
    -drive file='/home/medic/Work/Projekty/autotest/autotest/client/tests/virt/shared/data/isos/windows/winutils.iso',media=cdrom,index=1 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -vnc :0 \
    -vga std \
    -rtc base=localtime,clock=host,driftfix=none  \
    -boot order=cdn,once=c,menu=off  \
    -enable-kvm

2. Use VNC to log-in
3. Check all available consoles (eg. using winobj).
  
Actual results:
ports vc3, vc4 are missing (\Device\vport1p0 and \Device\vport1p1 are missing)

Expected results:
All vio ports are present and ready for use (\Device\vport0p0, \Device\vport0p1, \Device\vport1p0, \Device\vport1p1, \Device\vport2p1, \Device\vport2p2, \Device\vport3p1 and \Device\vport3p2)


Additional info:
There is an automated autotest test for this. Install autotest, download virt-test and use:
run -t qemu --tests='virtio_console..loopback' -g WinXP.i386.sp3 --smp=4 --disk-bus=ide --qemu-bin=/usr/local/bin/qemu-system-x86_64 --nettype=bridge --no=spread_linear

to run the automated test.
Comment 1 Lukáš Doktor 2013-04-09 12:03:04 UTC 
This but was discovered in https://bugzilla.redhat.com/show_bug.cgi?id=858674 where some information might be found.
Comment 2 Lukáš Doktor 2013-04-09 12:58:51 UTC 
I tested various settings with those results:

SETUP           | RESULTS
4vs 0vc spread0 | PASS
8vs 0vc spread0 | PASS
4vs 1vc spread0 | PASS
4vs 2vc spread0 | PASS
4vs 3vc spread0 | PASS
4vs 0vc spread1 | missing vs2
8vs 0vc spread1 | missing vs2 vs6
4vs 1vc spread1 | missing vs3
4vs 2vc spread1 | PASS
4vs 3vc spread1 | missing vs4 vc1
4vs 0vc spread2 | PASS
8vs 0vc spread2 | missing vs3 vs4
4vs 1vc spread2 | PASS
4vs 2vc spread2 | PASS
4vs 3vc spread2 | missing vs3 vs4
4vs 0vc spread3 | PASS
8vs 0vc spread3 | PASS
4vs 1vc spread3 | PASS
4vs 2vc spread3 | PASS
4vs 3vc spread3 | missing vs3 vs4

The test tried to opened all of the virtserialports and virtconsoles. If successful it copied data from vs1 to vs2, vs3 and vs4. Other ports were unused.

Host: Fedora 17, upstream qemu-1.4.0, smp4
Guest: WindowsXP.i386.sp3, vio-0.1.58(pre-whql), smp4

$(int)vs = how many virtserialports were created
$(int)vc = how many virtconsoles were created
spread$(int) = how many ports per virtio-serial-pci starting with virtconsoles
    // 4vs 3vc spread3 => pci1(vc1, vc2, vc3) pci2(vs1, vs2, vs3) pci3(vs4)

Every test was executed at least twice with the same results. Let me know if you need more detailed output or different setup.
Comment 3 Lukáš Doktor 2013-04-09 15:14:32 UTC 
Different results were produced on RHEL6-NIGHTLY with kernel-2.6.32-358.2.1 with the exact same upstream qemu-1.4.0, smp2.

SETUP           | RESULTS
4vs 4vc spread0 | PASS
4vs 4vc spread1 | qemu coredump after ~30s (err 134)
4vs 4vc spread2 | qemu coredump after ~60s (err 134)
4vs 4vc spread3 | PASS
4vs 4vc spread4 | PASS
Comment 4 Ronen Hod 2013-05-25 19:25:50 UTC 
Amit,

See comment 3. Could it be that the new chardev flow control solves it?
Comment 5 Amit Shah 2013-05-27 07:30:51 UTC 
No, this doesn't do any data transfer, just checks for port availability.

Question: why is virtconsole important for Windows guests?
Comment 6 Ronen Hod 2013-05-28 07:58:56 UTC 
Amit is right. virtconsole is irrelevant for Windows. Since the new chardev does not solve it, we still need to do something about the QEMU coredumps.
Still, not high priority, since it will not affect any properly working setup.
Comment 7 Amit Shah 2013-05-28 12:43:27 UTC 
comment 3 mentions core dumps, but they're not attached anywhere.  Please provide them for triage.
Comment 8 Lukáš Doktor 2013-05-30 09:50:32 UTC 
Hi,

I executed it today with default spread_2 setting:
* RHEL6-NIGHTLY (not updated regularly)
* kernel-2.6.32-358.2.1.el6
* upstream qemu-kvm-1.4.0
* 4 virtio_serial ports + 4 virtio_console ports, 2 ports per 1 virtio-serial-pci

Crash after ~60s, screen froze when the windows logo with black background occurred, 20-30s was frozen (no lines were circulating below the logo) and than it core-dumped (viz. attachment).
Comment 9 Lukáš Doktor 2013-05-30 10:06:23 UTC 
Created attachment 754754 [details]
qemu coredump when using 8 virtio ports spread across 4 virtio-serial-pcis

[root@dhcp-27-50 qemu]# MALLOC_PERTURB_=1 /usr/local/bin/qemu-system-x86_64 -name 'virt-tests-vm1' -nodefaults -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20130530-103049-FuCJSlT7,server,nowait -mon chardev=hmp_id_hmp1,mode=readline -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130530-103049-FuCJSlT7,server,nowait -device isa-serial,chardev=serial_id_serial1 -device virtio-serial-pci,id=virtio_serial_pci0,addr=0x4 -chardev socket,id=devvc1,path=/tmp/virtio_port-vc1-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc1,name=vc1,id=vc1,bus=virtio_serial_pci0.0 -chardev socket,id=devvc2,path=/tmp/virtio_port-vc2-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc2,name=vc2,id=vc2,bus=virtio_serial_pci0.0 -device virtio-serial-pci,id=virtio_serial_pci1,addr=0x5 -chardev socket,id=devvc3,path=/tmp/virtio_port-vc3-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc3,name=vc3,id=vc3,bus=virtio_serial_pci1.0 -chardev socket,id=devvc4,path=/tmp/virtio_port-vc4-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc4,name=vc4,id=vc4,bus=virtio_serial_pci1.0 -device virtio-serial-pci,id=virtio_serial_pci2,addr=0x6 -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs1,name=vs1,id=vs1,bus=virtio_serial_pci2.0 -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs2,name=vs2,id=vs2,bus=virtio_serial_pci2.0 -device virtio-serial-pci,id=virtio_serial_pci3,addr=0x7 -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs3,name=vs3,id=vs3,bus=virtio_serial_pci3.0 -chardev socket,id=devvs4,path=/tmp/virtio_port-vs4-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs4,name=vs4,id=vs4,bus=virtio_serial_pci3.0 -chardev socket,id=seabioslog_id_20130530-103049-FuCJSlT7,path=/tmp/seabios-20130530-103049-FuCJSlT7,server,nowait -device isa-debugcon,chardev=seabioslog_id_20130530-103049-FuCJSlT7,iobase=0x402 -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=0x8 -drive file='/mnt/store/KVM-autotest/autotest/client/tests/virt/shared/data/images/winXP-32.qcow2',if=none,id=drive-ide0-0-0,media=disk,format=qcow2,aio=native -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0 -device virtio-net-pci,netdev=idAQfC8A,mac='9a:1f:20:21:22:23',bus=pci.0,addr=0x9,id='id2gAvzM' -netdev user,id=idAQfC8A,hostfwd=tcp::5000-:10022,hostfwd=tcp::5001-:10023 -m 1024 -smp 2,maxcpus=2,cores=1,threads=1,sockets=2 -cpu 'Conroe' -M pc -drive file='/mnt/store/KVM-autotest/autotest/client/tests/virt/shared/data/isos/windows/winutils.iso',if=none,id=drive-ide0-0-1,media=cdrom,format=raw -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=localtime,clock=host,driftfix=none -boot order=cdn,once=c,menu=off -enable-kvm
Invalid write to memory region kvm-pic
Invalid write to memory region kvm-pic
qemu-system-x86_64: virtio-serial-bus: Unexpected port id 2 for device virtio_serial_pci0.0
Aborted (core dumped)
Comment 10 Amit Shah 2013-05-30 10:52:00 UTC 
Please post the gdb backtrace of the core here.

Also, pls try on qemu 1.5 as well.
Comment 11 Lukáš Doktor 2013-05-30 11:17:51 UTC 
Backtrace of the core.5360 - qemu-1.4.0

#0  0x00007fe1804c18a5 in raise () from /lib64/libc.so.6
#1  0x00007fe1804c3085 in abort () from /lib64/libc.so.6
#2  0x00007fe18267ffb4 in kvm_io_ioeventfd_add (listener=<value optimized out>, section=0x7fe17e8658c0, match_data=true, data=6, e=<value optimized out>) at /mnt/store/qemu/kvm-all.c:788
#3  0x00007fe18268618c in address_space_add_del_ioeventfds () at /mnt/store/qemu/memory.c:616
#4  address_space_update_ioeventfds () at /mnt/store/qemu/memory.c:649
#5  address_space_update_topology () at /mnt/store/qemu/memory.c:730
#6  memory_region_transaction_commit () at /mnt/store/qemu/memory.c:750
#7  0x00007fe182687185 in memory_region_add_eventfd (mr=0x7fe184b1a8e8, addr=<value optimized out>, size=<value optimized out>, match_data=<value optimized out>, data=<value optimized out>,
    e=<value optimized out>) at /mnt/store/qemu/memory.c:1273
#8  0x00007fe1825bbc04 in virtio_pci_set_host_notifier_internal (proxy=0x7fe184b1a250, n=6, assign=<value optimized out>, set_handler=true) at hw/virtio-pci.c:191
#9  0x00007fe1825bbe9a in virtio_pci_start_ioeventfd (proxy=0x7fe184b1a250) at hw/virtio-pci.c:217
#10 0x00007fe1825bc4b0 in virtio_ioport_write (opaque=0x7fe184b1a250, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio-pci.c:308
#11 virtio_pci_config_write (opaque=0x7fe184b1a250, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio-pci.c:427
#12 0x00007fe1826823e0 in access_with_adjusted_size (addr=18, value=0x7fe17e865c08, size=1, access_size_min=<value optimized out>, access_size_max=<value optimized out>, access=
    0x7fe182683b50 <memory_region_write_accessor>, opaque=0x7fe184b1a8e8) at /mnt/store/qemu/memory.c:364
#13 0x00007fe182682a0b in memory_region_iorange_write (iorange=<value optimized out>, offset=<value optimized out>, width=<value optimized out>, data=5) at /mnt/store/qemu/memory.c:439
#14 0x00007fe182681812 in kvm_handle_io (env=0x7fe184a46720) at /mnt/store/qemu/kvm-all.c:1426
#15 kvm_cpu_exec (env=0x7fe184a46720) at /mnt/store/qemu/kvm-all.c:1581
#16 0x00007fe182632e41 in qemu_kvm_cpu_thread_fn (arg=0x7fe184a46720) at /mnt/store/qemu/cpus.c:759
#17 0x00007fe181288851 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fe18057790d in clone () from /lib64/libc.so.6
Comment 12 Amit Shah 2013-05-30 11:21:39 UTC 
Can you use 't a a bt' to get a bt of all the threads?
Comment 13 Lukáš Doktor 2013-05-30 11:26:29 UTC 
Created attachment 754785 [details]
qemu-1.5.0 coredump when using 8 virtio ports spread across 4 virtio-serial-pcis on RHEL6

Executing this command:

[root@dhcp-27-50 qemu]# MALLOC_PERTURB_=1 /usr/local/bin/qemu-system-x86_64 -name 'virt-tests-vm1' -nodefaults -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20130530-103049-FuCJSlT7,server,no
wait -mon chardev=hmp_id_hmp1,mode=readline -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130530-103049-FuCJSlT7,server,nowait -device isa-serial,chardev=serial_id_serial1 -device v
irtio-serial-pci,id=virtio_serial_pci0,addr=0x4 -chardev socket,id=devvc1,path=/tmp/virtio_port-vc1-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc1,name=vc1,id=vc1,bus=virti
o_serial_pci0.0 -chardev socket,id=devvc2,path=/tmp/virtio_port-vc2-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc2,name=vc2,id=vc2,bus=virtio_serial_pci0.0 -device virtio-s
erial-pci,id=virtio_serial_pci1,addr=0x5 -chardev socket,id=devvc3,path=/tmp/virtio_port-vc3-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc3,name=vc3,id=vc3,bus=virtio_seria
l_pci1.0 -chardev socket,id=devvc4,path=/tmp/virtio_port-vc4-20130530-103049-FuCJSlT7,server,nowait -device virtconsole,chardev=devvc4,name=vc4,id=vc4,bus=virtio_serial_pci1.0 -device virtio-serial-p
ci,id=virtio_serial_pci2,addr=0x6 -chardev socket,id=devvs1,path=/tmp/virtio_port-vs1-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs1,name=vs1,id=vs1,bus=virtio_serial_pc
i2.0 -chardev socket,id=devvs2,path=/tmp/virtio_port-vs2-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs2,name=vs2,id=vs2,bus=virtio_serial_pci2.0 -device virtio-serial-pc
i,id=virtio_serial_pci3,addr=0x7 -chardev socket,id=devvs3,path=/tmp/virtio_port-vs3-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs3,name=vs3,id=vs3,bus=virtio_serial_pci
3.0 -chardev socket,id=devvs4,path=/tmp/virtio_port-vs4-20130530-103049-FuCJSlT7,server,nowait -device virtserialport,chardev=devvs4,name=vs4,id=vs4,bus=virtio_serial_pci3.0 -chardev socket,id=seabio
slog_id_20130530-103049-FuCJSlT7,path=/tmp/seabios-20130530-103049-FuCJSlT7,server,nowait -device isa-debugcon,chardev=seabioslog_id_20130530-103049-FuCJSlT7,iobase=0x402 -device ich9-usb-uhci1,id=us
b1,bus=pci.0,addr=0x8 -drive file='/mnt/store/KVM-autotest/autotest/client/tests/virt/shared/data/images/winXP-32.qcow2',if=none,id=drive-ide0-0-0,media=disk,format=qcow2,aio=native -device ide-drive
,bus=ide.0,unit=0,drive=drive-ide0-0-0 -device virtio-net-pci,netdev=idAQfC8A,mac='9a:1f:20:21:22:23',bus=pci.0,addr=0x9,id='id2gAvzM' -netdev user,id=idAQfC8A,hostfwd=tcp::5000-:10022,hostfwd=tcp::5
001-:10023 -m 1024 -smp 2,maxcpus=2,cores=1,threads=1,sockets=2 -cpu 'Conroe' -M pc -drive file='/mnt/store/KVM-autotest/autotest/client/tests/virt/shared/data/isos/windows/winutils.iso',if=none,id=d
rive-ide0-0-1,media=cdrom,format=raw -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1 -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -vga std -rtc base=localtime,clock=host,driftf
ix=none -boot order=cdn,once=c,menu=off -enable-kvm
Invalid write to memory region kvm-pic
Invalid write to memory region kvm-pic
qemu-system-x86_64: virtio-serial-bus: Unexpected port id 2 for device virtio_serial_pci0.0
Aborted (core dumped)


the same result as previously, only qemu-1.5.0 crashes faster (~5s after the windows logo occurs).

Backtrace:
#0  0x00007fd3ff0f98a5 in raise () from /lib64/libc.so.6
#1  0x00007fd3ff0fb085 in abort () from /lib64/libc.so.6
#2  0x00007fd400cb947b in kvm_io_ioeventfd_add (listener=<value optimized out>, section=0x7fd3fdac2930, match_data=true, data=6, e=<value optimized out>) at /mnt/store/qemu/kvm-all.c:846
#3  0x00007fd400cbeabc in address_space_add_del_ioeventfds () at /mnt/store/qemu/memory.c:616
#4  address_space_update_ioeventfds () at /mnt/store/qemu/memory.c:649
#5  address_space_update_topology () at /mnt/store/qemu/memory.c:730
#6  memory_region_transaction_commit () at /mnt/store/qemu/memory.c:750
#7  0x00007fd400cbf805 in memory_region_add_eventfd (mr=0x7fd402f85940, addr=<value optimized out>, size=<value optimized out>, match_data=<value optimized out>, data=<value optimized out>,
    e=<value optimized out>) at /mnt/store/qemu/memory.c:1273
#8  0x00007fd400c0ff44 in virtio_pci_set_host_notifier_internal (proxy=0x7fd402f85290, n=6, assign=<value optimized out>, set_handler=true) at hw/virtio/virtio-pci.c:195
#9  0x00007fd400c1013a in virtio_pci_start_ioeventfd (proxy=0x7fd402f85290) at hw/virtio/virtio-pci.c:221
#10 0x00007fd400c10638 in virtio_ioport_write (opaque=0x7fd402f85290, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio/virtio-pci.c:303
#11 virtio_pci_config_write (opaque=0x7fd402f85290, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio/virtio-pci.c:422
#12 0x00007fd400cbb3d1 in access_with_adjusted_size (addr=18, value=0x7fd3fdac2c28, size=1, access_size_min=<value optimized out>, access_size_max=<value optimized out>, access=
    0x7fd400cbc7a0 <memory_region_write_accessor>, opaque=0x7fd402f85940) at /mnt/store/qemu/memory.c:364
#13 0x00007fd400cbb78f in memory_region_iorange_write (iorange=<value optimized out>, offset=<value optimized out>, width=<value optimized out>, data=5) at /mnt/store/qemu/memory.c:439
#14 0x00007fd400cbaa01 in kvm_handle_io (env=0x7fd402efee60) at /mnt/store/qemu/kvm-all.c:1482
#15 kvm_cpu_exec (env=0x7fd402efee60) at /mnt/store/qemu/kvm-all.c:1634
#16 0x00007fd400c7645e in qemu_kvm_cpu_thread_fn (arg=0x7fd402efee60) at /mnt/store/qemu/cpus.c:759
#17 0x00007fd3ff461851 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fd3ff1af90d in clone () from /lib64/libc.so.6


When I execute the same command on Fedora 17 with the same qemu-1.4.0 (1.5.0) it boots successfully but few ports are missing:
Fail to open port vs3
Fail to open port vs4
Fail to open port vc3
Fail to open port vc4
Comment 14 Lukáš Doktor 2013-05-30 12:26:56 UTC 
I retested this with the latest (pre0.1.63) windows virtio drivers with the same results.
Comment 15 Gal Hammer 2013-05-30 12:30:39 UTC 
(In reply to Lukas Doktor from comment #14)
> I retested this with the latest (pre0.1.63) windows virtio drivers with the
> same results.

Is it possible that you meant to bz#858674?
Comment 16 Lukáš Doktor 2013-05-30 12:34:32 UTC 
I forgot to append the host libraries versions:
glib2-2.22.5-7.el6.x86_64
glibc-2.12-1.107.el6.x86_64
libuuid-2.17.2-12.9.el6.x86_64
pixman-0.26.2-5.el6_4.x86_64
zlib-1.2.3-29.el6.x86_64
kernel-2.6.358.2.1.el6.x86_64
Comment 17 Lukáš Doktor 2013-05-30 12:36:00 UTC 
(In reply to Gal Hammer from comment #15)
> (In reply to Lukas Doktor from comment #14)
> > I retested this with the latest (pre0.1.63) windows virtio drivers with the
> > same results.
> 
> Is it possible that you meant to bz#858674?

No, I retested this as Amit asked me to test it with newer drivers too. Anyway I'll retest the bz#858674 too.
Comment 18 Gal Hammer 2013-05-30 12:38:40 UTC 
(In reply to Lukas Doktor from comment #17)
> (In reply to Gal Hammer from comment #15)
> > (In reply to Lukas Doktor from comment #14)
> > > I retested this with the latest (pre0.1.63) windows virtio drivers with the
> > > same results.
> > 
> > Is it possible that you meant to bz#858674?
> 
> No, I retested this as Amit asked me to test it with newer drivers too.

I thought Amit asked for more back traces in comment#12.

> Anyway I'll retest the bz#858674 too.

Thanks :-).
Comment 19 Lukáš Doktor 2013-05-31 07:36:05 UTC 
qemu-1.5.0 backtrace when using 8 virtio ports spread across 4 virtio-serial-pcis on RHEL6 (using t a a tb)

Thread 5 (Thread 0x7fd3ff0c6700 (LWP 5670)):
#0  0x00007fd3ff467811 in sem_timedwait () from /lib64/libpthread.so.0
#1  0x00007fd400d62c43 in qemu_sem_timedwait (sem=0x7fd402d64098, ms=<value optimized out>) at util/qemu-thread-posix.c:237
#2  0x00007fd400c50b8e in worker_thread (opaque=0x7fd402d64000) at thread-pool.c:96
#3  0x00007fd3ff461851 in start_thread () from /lib64/libpthread.so.0
#4  0x00007fd3ff1af90d in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7fd3fe4c4700 (LWP 5671)):
#0  0x00007fd3ff468054 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fd3ff463388 in _L_lock_854 () from /lib64/libpthread.so.0
#2  0x00007fd3ff463257 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007fd400d630e9 in qemu_mutex_lock (mutex=<value optimized out>) at util/qemu-thread-posix.c:57
#4  0x00007fd400cba85d in kvm_cpu_exec (env=0x7fd402ecfcf0) at /mnt/store/qemu/kvm-all.c:1616
#5  0x00007fd400c7645e in qemu_kvm_cpu_thread_fn (arg=0x7fd402ecfcf0) at /mnt/store/qemu/cpus.c:759
#6  0x00007fd3ff461851 in start_thread () from /lib64/libpthread.so.0
#7  0x00007fd3ff1af90d in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7fd400a998c0 (LWP 5669)):
#0  0x00007fd3ff468054 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007fd3ff463388 in _L_lock_854 () from /lib64/libpthread.so.0
#2  0x00007fd3ff463257 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x00007fd400d630e9 in qemu_mutex_lock (mutex=<value optimized out>) at util/qemu-thread-posix.c:57
#4  0x00007fd400c13565 in os_host_main_loop_wait (nonblocking=<value optimized out>) at main-loop.c:229
#5  main_loop_wait (nonblocking=<value optimized out>) at main-loop.c:464
#6  0x00007fd400c6ce95 in main_loop (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at vl.c:2029
#7  main (argc=<value optimized out>, argv=<value optimized out>, envp=<value optimized out>) at vl.c:4419

Thread 2 (Thread 0x7fd3aefff700 (LWP 5674)):
#0  0x00007fd3ff46543c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fd400d62e69 in qemu_cond_wait (cond=<value optimized out>, mutex=<value optimized out>) at util/qemu-thread-posix.c:116
#2  0x00007fd400c636fb in vnc_worker_thread_loop (queue=0x7fd403000ac0) at ui/vnc-jobs.c:222
#3  0x00007fd400c63b58 in vnc_worker_thread (arg=0x7fd403000ac0) at ui/vnc-jobs.c:318
#4  0x00007fd3ff461851 in start_thread () from /lib64/libpthread.so.0
#5  0x00007fd3ff1af90d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7fd3fdac3700 (LWP 5672)):
#0  0x00007fd3ff0f98a5 in raise () from /lib64/libc.so.6
#1  0x00007fd3ff0fb085 in abort () from /lib64/libc.so.6
#2  0x00007fd400cb947b in kvm_io_ioeventfd_add (listener=<value optimized out>, section=0x7fd3fdac2930, match_data=true, data=6, e=<value optimized out>) at /mnt/store/qemu/kvm-all.c:846
#3  0x00007fd400cbeabc in address_space_add_del_ioeventfds () at /mnt/store/qemu/memory.c:616
#4  address_space_update_ioeventfds () at /mnt/store/qemu/memory.c:649
#5  address_space_update_topology () at /mnt/store/qemu/memory.c:730
#6  memory_region_transaction_commit () at /mnt/store/qemu/memory.c:750
#7  0x00007fd400cbf805 in memory_region_add_eventfd (mr=0x7fd402f85940, addr=<value optimized out>, size=<value optimized out>, match_data=<value optimized out>, data=<value optimized out>,
    e=<value optimized out>) at /mnt/store/qemu/memory.c:1273
#8  0x00007fd400c0ff44 in virtio_pci_set_host_notifier_internal (proxy=0x7fd402f85290, n=6, assign=<value optimized out>, set_handler=true) at hw/virtio/virtio-pci.c:195
#9  0x00007fd400c1013a in virtio_pci_start_ioeventfd (proxy=0x7fd402f85290) at hw/virtio/virtio-pci.c:221
#10 0x00007fd400c10638 in virtio_ioport_write (opaque=0x7fd402f85290, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio/virtio-pci.c:303
#11 virtio_pci_config_write (opaque=0x7fd402f85290, addr=<value optimized out>, val=<value optimized out>, size=<value optimized out>) at hw/virtio/virtio-pci.c:422
#12 0x00007fd400cbb3d1 in access_with_adjusted_size (addr=18, value=0x7fd3fdac2c28, size=1, access_size_min=<value optimized out>, access_size_max=<value optimized out>, access=
    0x7fd400cbc7a0 <memory_region_write_accessor>, opaque=0x7fd402f85940) at /mnt/store/qemu/memory.c:364
#13 0x00007fd400cbb78f in memory_region_iorange_write (iorange=<value optimized out>, offset=<value optimized out>, width=<value optimized out>, data=5) at /mnt/store/qemu/memory.c:439
#14 0x00007fd400cbaa01 in kvm_handle_io (env=0x7fd402efee60) at /mnt/store/qemu/kvm-all.c:1482
#15 kvm_cpu_exec (env=0x7fd402efee60) at /mnt/store/qemu/kvm-all.c:1634
#16 0x00007fd400c7645e in qemu_kvm_cpu_thread_fn (arg=0x7fd402efee60) at /mnt/store/qemu/cpus.c:759
#17 0x00007fd3ff461851 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fd3ff1af90d in clone () from /lib64/libc.so.6
Comment 20 Gal Hammer 2013-08-04 13:29:56 UTC 
When working with only virtserialports (e.g. no virtconsole), I was able to reproduce the problem with build-64.

The problem was not reproduced with build-66.

There is a submitted patch that fixed the virtconsole related problems. So I hope this bug will be closed on the next build.
Comment 21 Lukáš Doktor 2013-08-08 11:45:15 UTC 
As I promissed in https://bugzilla.redhat.com/show_bug.cgi?id=858674 I tested the initialization with build-66 with these results:

Fedora 19, qemu-kvm-1.4.2-4.fc19.x86_64, 4CPUs, 12G mem
windows XP, vio-drivers-66, 4CPU and 1CPU, 1G mem

Tests:
1) boot with 4 virtconsoles and 4 virtserialport and try to open all 8 ports
2) boot with 15 virtconsoles and 15 virtserialports and try to open all 30 ports

Variants:
1) smp1, smp4
2) spread0, spread1, spread2, ..., spread8 (=> the number means how many ports per single virtio-serial-pci)

[SMP1]
spread0:
  1) PASS
  2) PASS
spread1:
  1) windows hanged just when the first windows logo appeared
  2) SKIPPED (not enough PCI slots)
spread2:
  1) windows hanged just when the first windows logo appeared
  2) windows hanged just when the first windows logo appeared
spread3:
  1) windows hanged when the first logo disappeared, the screen blanked (desktop never showed up)
  2) windows hanged just when the first windows logo appeared
spread4:
  1) PASS
  2) windows hanged just when the first windows logo appeared
spread5:
  1) PASS
  2) windows hanged just when the first windows logo appeared
...

[SMP4]
spread0:
  1) PASS
  2) PASS
spread1:
  1) vs2 vc2 (pci 2, 4)
  2) SKIPPED (not enough PCI slots)
spread2:
  1) vc3 vc4 (pci 2)
  2) serialport 0 1 2 7 8 9 10; console 6 7 8 9 14 (pci 4 5 8 9 12 13)
spread3:
  1) windows hanged just when the first windows logo appeared
  2) serialport 6 7 8; console 9 10 11 (pci 4 8)
spread4:
  1) PASS
  2) serialport 5 6 7 8; console 4 5 6 7 (pci 2 6)
spread5:
  1) PASS
  2) serialport 0 1 2 3 4 (pci 4)
spread6:
  1) PASS
  2) serialport 0 1 2; console 12 13 14 (pci 3)
spread7:
  1) PASS
  2) serialport 0 1 2 3 4 5; console 14 (pci 3)
spread8:
  1) PASS
  2) serialport 0; console 8 9 10 11 12 13 14 (pci 2)



In cases where windows hanged, there was nothing unusual in the qemu output. In some cases those messages occured:
13:03:05 INFO | [qemu output] qemu-system-x86_64: virtio-serial-bus: Unexpected port id 2254519380 for device virtio_serial_pci1.0
13:03:05 INFO | [qemu output] qemu-system-x86_64: virtio-serial-bus: Unexpected port id 4154515272 for device virtio_serial_pci1.0
13:03:05 INFO | [qemu output] qemu-system-x86_64: virtio-serial-bus: Unexpected port id 4154515272 for device virtio_serial_pci2.0
13:03:05 INFO | [qemu output] qemu-system-x86_64: virtio-serial-bus: Unexpected port id 4154515272 for device virtio_serial_pci2.0

(but they are quite common with various ids even on tests which PASSED)


After the last test (30 ports with spread3 smp4) the qemu consumed 250% cpu (out of 400). I was able to use windows, system idle process took most of the cpu time (95) and in performance tab it showed 95% utilisation (out of 2 cpus).
Comment 22 Gal Hammer 2013-08-12 09:42:40 UTC 
Build virtio-win-prewhql-0.1-67 should fix the error messages that was mention in comment #21.
Comment 23 Lukáš Doktor 2013-08-12 12:24:24 UTC 
Hi, I haven't met the messages, but the problems persists. With smp=1 and multiple virtio-serial-pcis the system hangs during startup.

With smp=4 it boots, but is very slow with 95+% utilization (process System Idle Process).

Once I met those messages:
13:57:56 INFO | [qemu output] qemu-system-x86_64: virtio_pci_start_ioeventfd: failed. Fallback to a userspace (slower).
13:57:56 INFO | [qemu output] qemu-system-x86_64: virtio_pci_set_host_notifier_internal: unable to init event notifier: -24
13:57:56 INFO | [qemu output] qemu-system-x86_64: virtio_pci_start_ioeventfd: failed. Fallback to a userspace (slower).
13:57:56 INFO | [qemu output] qemu-system-x86_64: virtio_pci_set_host_notifier_internal: unable to init event notifier: -24
13:57:56 INFO | [qemu output] qemu-system-x86_64: virtio_pci_start_ioeventfd: failed. Fallback to a userspace (slower).

(15 serialports, 15 consoles spread across 6 virtio-serial-pcis)

When the system booted-up successfully and used more than 1 virtio-serial-pci it didn't create all of the serialports/consoles (missing ports error were similar to previous test run so I didn't executed the full loop).
Comment 24 Lukáš Doktor 2013-08-12 13:13:12 UTC 
smp=4, virtio win 0.1-67

(1/12) virtio_console.spread_0.specifiable.virtserialport.without_vm.max_ports: PASS (33.20 s)
(2/12) virtio_console.spread_0.specifiable.virtconsole.without_vm.max_ports: PASS (39.50 s)
(3/12) virtio_console.spread_1.specifiable.virtserialport.without_vm.max_ports: TESTNA (9.74 s)
(4/12) virtio_console.spread_1.specifiable.virtconsole.without_vm.max_ports: TESTNA (2.27 s)
(5/12) virtio_console.spread_2.specifiable.virtserialport.without_vm.max_ports: FAIL (97.10 s)
14:50:02 ERROR| Fail to open port serialport-23
14:50:02 ERROR| Fail to open port serialport-22
14:50:02 ERROR| Fail to open port serialport-25
14:50:02 ERROR| Fail to open port serialport-24
14:50:02 ERROR| Fail to open port serialport-8
14:50:02 ERROR| Fail to open port serialport-9
14:50:02 ERROR| Fail to open port serialport-6
14:50:02 ERROR| Fail to open port serialport-7
14:50:02 ERROR| Fail to open port serialport-14
14:50:02 ERROR| Fail to open port serialport-15
14:50:02 ERROR| Fail to open port serialport-16
14:50:02 ERROR| Fail to open port serialport-17
(6/12) virtio_console.spread_2.specifiable.virtconsole.without_vm.max_ports: FAIL (99.36 s)
14:51:41 ERROR| Fail to open port console-22
14:51:41 ERROR| Fail to open port console-23
14:51:41 ERROR| Fail to open port console-24
14:51:41 ERROR| Fail to open port console-25
14:51:41 ERROR| Fail to open port console-7
14:51:41 ERROR| Fail to open port console-6
14:51:41 ERROR| Fail to open port console-9
14:51:41 ERROR| Fail to open port console-8
14:51:41 ERROR| Fail to open port console-15
14:51:41 ERROR| Fail to open port console-14
14:51:41 ERROR| Fail to open port console-17
14:51:41 ERROR| Fail to open port console-16
(7/12) virtio_console.spread_3.specifiable.virtserialport.without_vm.max_ports: FAIL (80.13 s)
14:53:01 ERROR| Fail to open port serialport-21
14:53:01 ERROR| Fail to open port serialport-23
14:53:01 ERROR| Fail to open port serialport-22
14:53:01 ERROR| Fail to open port serialport-9
14:53:01 ERROR| Fail to open port serialport-10
14:53:01 ERROR| Fail to open port serialport-11
(8/12) virtio_console.spread_3.specifiable.virtconsole.without_vm.max_ports: FAIL (75.44 s)
14:54:17 ERROR| Fail to open port console-21
14:54:17 ERROR| Fail to open port console-22
14:54:17 ERROR| Fail to open port console-23
14:54:17 ERROR| Fail to open port console-9
14:54:17 ERROR| Fail to open port console-11
14:54:17 ERROR| Fail to open port console-10
(9/12) virtio_console.spread_4.specifiable.virtserialport.without_vm.max_ports: FAIL (84.48 s)
14:55:41 ERROR| Fail to open port serialport-21
14:55:41 ERROR| Fail to open port serialport-20
14:55:41 ERROR| Fail to open port serialport-23
14:55:41 ERROR| Fail to open port serialport-22
14:55:41 ERROR| Fail to open port serialport-6
14:55:41 ERROR| Fail to open port serialport-7
14:55:41 ERROR| Fail to open port serialport-4
14:55:41 ERROR| Fail to open port serialport-5
(10/12) virtio_console.spread_4.specifiable.virtconsole.without_vm.max_ports: FAIL (71.57 s)
14:56:53 ERROR| Fail to open port console-20
14:56:53 ERROR| Fail to open port console-21
14:56:53 ERROR| Fail to open port console-22
14:56:53 ERROR| Fail to open port console-23
14:56:53 ERROR| Fail to open port console-5
14:56:53 ERROR| Fail to open port console-4
14:56:53 ERROR| Fail to open port console-7
14:56:53 ERROR| Fail to open port console-6
(11/12) virtio_console.spread_5.specifiable.virtserialport.without_vm.max_ports: FAIL 14:58:07 ERROR| Fail to open port serialport-18
14:58:07 ERROR| Fail to open port serialport-19
14:58:07 ERROR| Fail to open port serialport-15
14:58:07 ERROR| Fail to open port serialport-16
14:58:07 ERROR| Fail to open port serialport-17
(73.88 s)
(12/12) virtio_console.spread_5.specifiable.virtconsole.without_vm.max_ports: FAIL (81.47 s)
14:59:28 ERROR| Fail to open port console-19
14:59:28 ERROR| Fail to open port console-18
14:59:28 ERROR| Fail to open port console-15
14:59:28 ERROR| Fail to open port console-17
14:59:28 ERROR| Fail to open port console-16
Comment 25 Lukáš Doktor 2013-08-12 13:27:19 UTC 
SMP=4, virtio win 0.1-67

Test - boot with 4 consoles and 4 serialports and try to open them.

(1/12) virtio_console.spread_0.specifiable.virtserialport.virtio_console_smoke.open: PASS (38.14 s)
(2/12) virtio_console.spread_0.specifiable.virtconsole.virtio_console_smoke.open: PASS (13.13 s)
(3/12) virtio_console.spread_1.specifiable.virtserialport.virtio_console_smoke.open: FAIL (76.67 s)
15:15:29 ERROR| Fail to open port vs2
15:15:29 ERROR| Fail to open port vc2
(4/12) virtio_console.spread_1.specifiable.virtconsole.virtio_console_smoke.open: FAIL (76.48 s)
15:16:46 ERROR| Fail to open port vs2
15:16:46 ERROR| Fail to open port vc2
(5/12) virtio_console.spread_2.specifiable.virtserialport.virtio_console_smoke.open: FAIL (74.45 s)
15:18:00 ERROR| Fail to open port vc3
15:18:00 ERROR| Fail to open port vc4
(6/12) virtio_console.spread_2.specifiable.virtconsole.virtio_console_smoke.open: FAIL (67.69 s)
15:19:08 ERROR| Fail to open port vc3
15:19:08 ERROR| Fail to open port vc4
(7/12) virtio_console.spread_3.specifiable.virtserialport.virtio_console_smoke.open: PASS (49.35 s)
(8/12) virtio_console.spread_3.specifiable.virtconsole.virtio_console_smoke.open: PASS (14.23 s)
(9/12) virtio_console.spread_4.specifiable.virtserialport.virtio_console_smoke.open: PASS (52.12 s)
(10/12) virtio_console.spread_4.specifiable.virtconsole.virtio_console_smoke.open: PASS (13.29 s)
(11/12) virtio_console.spread_5.specifiable.virtserialport.virtio_console_smoke.open: PASS (46.25 s)
(12/12) virtio_console.spread_5.specifiable.virtconsole.virtio_console_smoke.open: PASS (13.56 s)
Comment 26 Gal Hammer 2013-08-12 14:14:36 UTC 
I see two problems here:

One, the error message "virtio_pci_start_ioeventfd: failed. Fallback to a userspace (slower)" which is new to me and, in a first glance, look like a QEMU issue.

And two, when you trying to create more ports that the bus device can handle. AFAIK the default is 4 ports per device. So unless you increase this number with the "max_ports" command line parameter, I don't think this is a bug.

Is your tests are failing if you set max_ports=32?
Comment 27 Gal Hammer 2013-08-13 08:47:49 UTC 
I think that error "unable to init event notifier: -24" mean that you're out of file descriptors. What does your "ulimit -n" say and can you please run your tests again with a larger value?
Comment 28 Lukáš Doktor 2013-08-13 10:39:03 UTC 
Hi Gal,

I tested it a bit more and I was able to get all the ports working. When I boot the VM for the first time, at least one virtio-serial-pci is missing. Also in device-manager when I look at details in "Bus Relations", no devices are mentioned (in the broken one).

I tried to disable it and enable again, without success. Anyway I tried to disable one which was working, enable the broken one, reboot the VM ... BUM. Windows didn't start. I was able to boot into safe mode where the interesting part began as the previously broken port worked. Than I re-enabled the one which worked previously and it worked too. So I got all the ports working.

Unfortunately now I'm unable to boot normal mode, but in safe mode it works fine (after few reboots). When I boot in normal mode, the first logo appears, 3 blue rectangles are moving for infinity. When I look in monitor "info cpus" reports one cpu halted:
* CPU #0: pc=0x00000000f76f0162 (halted) thread_id=7437
  CPU #1: pc=0x00000000806eca5b thread_id=7438

Details:
* qemu-kvm-1.5.0 (upstream)
* virtio-win 0.1-67
* 6xvirtio-serial-pci, each with 5 virtio-ports, max_ports=31

MALLOC_PERTURB_=1  /usr/local/bin/qemu-system-x86_64 \
    -S  \
    -name 'virt-tests-vm1' \
    -M pc  \
    -nodefaults  \
    -vga std  \
    -chardev socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20130812-190723-yoNWQZ3M,server,nowait \
    -mon chardev=hmp_id_hmp1,mode=readline  \
    -chardev socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130812-190723-yoNWQZ3M,server,nowait \
    -device isa-serial,chardev=serial_id_serial1 \
    -device driver=virtio-serial-pci,id=virtio_serial_pci0,max_ports=31  \
    -chardev socket,id=devconsole-0,path=/tmp/virtio_port-console-0-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-0,name=console-0,id=console-0,bus=virtio_serial_pci0.0  \
    -chardev socket,id=devconsole-1,path=/tmp/virtio_port-console-1-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-1,name=console-1,id=console-1,bus=virtio_serial_pci0.0  \
    -chardev socket,id=devconsole-2,path=/tmp/virtio_port-console-2-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-2,name=console-2,id=console-2,bus=virtio_serial_pci0.0  \
    -chardev socket,id=devconsole-3,path=/tmp/virtio_port-console-3-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-3,name=console-3,id=console-3,bus=virtio_serial_pci0.0  \
    -chardev socket,id=devconsole-4,path=/tmp/virtio_port-console-4-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-4,name=console-4,id=console-4,bus=virtio_serial_pci0.0 \
    -device driver=virtio-serial-pci,id=virtio_serial_pci1,max_ports=31  \
    -chardev socket,id=devconsole-5,path=/tmp/virtio_port-console-5-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-5,name=console-5,id=console-5,bus=virtio_serial_pci1.0  \
    -chardev socket,id=devconsole-6,path=/tmp/virtio_port-console-6-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-6,name=console-6,id=console-6,bus=virtio_serial_pci1.0  \
    -chardev socket,id=devconsole-7,path=/tmp/virtio_port-console-7-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-7,name=console-7,id=console-7,bus=virtio_serial_pci1.0  \
    -chardev socket,id=devconsole-8,path=/tmp/virtio_port-console-8-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-8,name=console-8,id=console-8,bus=virtio_serial_pci1.0  \
    -chardev socket,id=devconsole-9,path=/tmp/virtio_port-console-9-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-9,name=console-9,id=console-9,bus=virtio_serial_pci1.0 \
    -device driver=virtio-serial-pci,id=virtio_serial_pci2,max_ports=31  \
    -chardev socket,id=devconsole-10,path=/tmp/virtio_port-console-10-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-10,name=console-10,id=console-10,bus=virtio_serial_pci2.0  \
    -chardev socket,id=devconsole-11,path=/tmp/virtio_port-console-11-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-11,name=console-11,id=console-11,bus=virtio_serial_pci2.0  \
    -chardev socket,id=devconsole-12,path=/tmp/virtio_port-console-12-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-12,name=console-12,id=console-12,bus=virtio_serial_pci2.0  \
    -chardev socket,id=devconsole-13,path=/tmp/virtio_port-console-13-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-13,name=console-13,id=console-13,bus=virtio_serial_pci2.0  \
    -chardev socket,id=devconsole-14,path=/tmp/virtio_port-console-14-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-14,name=console-14,id=console-14,bus=virtio_serial_pci2.0 \
    -device driver=virtio-serial-pci,id=virtio_serial_pci3,max_ports=31  \
    -chardev socket,id=devconsole-15,path=/tmp/virtio_port-console-15-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-15,name=console-15,id=console-15,bus=virtio_serial_pci3.0  \
    -chardev socket,id=devconsole-16,path=/tmp/virtio_port-console-16-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-16,name=console-16,id=console-16,bus=virtio_serial_pci3.0  \
    -chardev socket,id=devconsole-17,path=/tmp/virtio_port-console-17-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-17,name=console-17,id=console-17,bus=virtio_serial_pci3.0  \
    -chardev socket,id=devconsole-18,path=/tmp/virtio_port-console-18-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-18,name=console-18,id=console-18,bus=virtio_serial_pci3.0  \
    -chardev socket,id=devconsole-19,path=/tmp/virtio_port-console-19-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-19,name=console-19,id=console-19,bus=virtio_serial_pci3.0 \
    -device driver=virtio-serial-pci,id=virtio_serial_pci4,max_ports=31  \
    -chardev socket,id=devconsole-20,path=/tmp/virtio_port-console-20-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-20,name=console-20,id=console-20,bus=virtio_serial_pci4.0  \
    -chardev socket,id=devconsole-21,path=/tmp/virtio_port-console-21-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-21,name=console-21,id=console-21,bus=virtio_serial_pci4.0  \
    -chardev socket,id=devconsole-22,path=/tmp/virtio_port-console-22-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-22,name=console-22,id=console-22,bus=virtio_serial_pci4.0  \
    -chardev socket,id=devconsole-23,path=/tmp/virtio_port-console-23-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-23,name=console-23,id=console-23,bus=virtio_serial_pci4.0  \
    -chardev socket,id=devconsole-24,path=/tmp/virtio_port-console-24-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-24,name=console-24,id=console-24,bus=virtio_serial_pci4.0 \
    -device driver=virtio-serial-pci,id=virtio_serial_pci5,max_ports=31  \
    -chardev socket,id=devconsole-25,path=/tmp/virtio_port-console-25-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-25,name=console-25,id=console-25,bus=virtio_serial_pci5.0  \
    -chardev socket,id=devconsole-26,path=/tmp/virtio_port-console-26-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-26,name=console-26,id=console-26,bus=virtio_serial_pci5.0  \
    -chardev socket,id=devconsole-27,path=/tmp/virtio_port-console-27-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-27,name=console-27,id=console-27,bus=virtio_serial_pci5.0  \
    -chardev socket,id=devconsole-28,path=/tmp/virtio_port-console-28-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-28,name=console-28,id=console-28,bus=virtio_serial_pci5.0  \
    -chardev socket,id=devconsole-29,path=/tmp/virtio_port-console-29-20130812-190723-yoNWQZ3M,server,nowait \
    -device virtconsole,chardev=devconsole-29,name=console-29,id=console-29,bus=virtio_serial_pci5.0  \
    -chardev socket,id=seabioslog_id_20130812-190723-yoNWQZ3M,path=/tmp/seabios-20130812-190723-yoNWQZ3M,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20130812-190723-yoNWQZ3M,iobase=0x402 \
    -device driver=ich9-usb-uhci1,id=usb1 \
    -drive file=/home/medic/Work/Projekty/autotest/autotest/client/tests/virt/shared/data/images/winXP-32.qcow2,if=none,id=drive-ide0-0-0,media=disk,format=qcow2,aio=native \
    -device driver=ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
    -device driver=virtio-net-pci,mac=9a:cb:cc:cd:ce:cf,id=idJQ17RY,netdev=idHAFvrh  \
    -netdev user,id=idHAFvrh,hostfwd=tcp::5001-:10022,hostfwd=tcp::5002-:10023  \
    -m 1024  \
    -smp 2,maxcpus=2,cores=1,threads=1,sockets=2  \
    -cpu 'SandyBridge' \
    -drive file=/home/medic/Work/Projekty/autotest/autotest/client/tests/virt/shared/data/isos/windows/winutils.iso,if=none,id=drive-ide0-0-1,media=cdrom,format=raw \
    -device driver=ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1  \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :1  \
    -rtc base=localtime,clock=host,driftfix=none  \
    -boot order=cdn,once=c,menu=off \
    -enable-kvm
Comment 29 Lukáš Doktor 2013-08-13 10:45:28 UTC 
I also retested the loop with max_ports=31 appended to each virtio-serial-pci with no differences. When I tried to boot VM with max_ports=1 with multiple consoles on the virtio-serial-pci it raised exception:

Device initialization failed.\nqemu-system-x86_64: -device virtconsole,chardev=devconsole-1,name=console-1,id=console-1,bus=virtio_serial_pci0.0: Device 'virtconsole' could not be initialized


About the "unable to init event notifier: -24" it appeared only few times. My ulimit is 1024 and I might reached it...
Comment 30 Lukáš Doktor 2013-08-13 10:48:23 UTC 
(In reply to Lukas Doktor from comment #28)
> Hi Gal,
> 
> I tested it a bit more and I was able to get all the ports working. When I
> boot the VM for the first time, at least one virtio-serial-pci is missing.
> Also in device-manager when I look at details in "Bus Relations", no devices
> are mentioned (in the broken one).
> 
> I tried to disable it and enable again, without success. Anyway I tried to
> disable one which was working, enable the broken one, reboot the VM ... BUM.
> Windows didn't start. I was able to boot into safe mode where the
> interesting part began as the previously broken port worked. Than I
> re-enabled the one which worked previously and it worked too. So I got all
> the ports working.
> 
> Unfortunately now I'm unable to boot normal mode, but in safe mode it works
> fine (after few reboots). When I boot in normal mode, the first logo
> appears, 3 blue rectangles are moving for infinity. When I look in monitor
> "info cpus" reports one cpu halted:
> * CPU #0: pc=0x00000000f76f0162 (halted) thread_id=7437
>   CPU #1: pc=0x00000000806eca5b thread_id=7438
> 
> Details:
> * qemu-kvm-1.5.0 (upstream)
> * virtio-win 0.1-67
> * 6xvirtio-serial-pci, each with 5 virtio-ports, max_ports=31
> 
> MALLOC_PERTURB_=1  /usr/local/bin/qemu-system-x86_64 \
>     -S  \
>     -name 'virt-tests-vm1' \
>     -M pc  \
>     -nodefaults  \
>     -vga std  \
>     -chardev
> socket,id=hmp_id_hmp1,path=/tmp/monitor-hmp1-20130812-190723-yoNWQZ3M,server,
> nowait \
>     -mon chardev=hmp_id_hmp1,mode=readline  \
>     -chardev
> socket,id=serial_id_serial1,path=/tmp/serial-serial1-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device isa-serial,chardev=serial_id_serial1 \
>     -device driver=virtio-serial-pci,id=virtio_serial_pci0,max_ports=31  \
>     -chardev
> socket,id=devconsole-0,path=/tmp/virtio_port-console-0-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-0,name=console-0,id=console-0,
> bus=virtio_serial_pci0.0  \
>     -chardev
> socket,id=devconsole-1,path=/tmp/virtio_port-console-1-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-1,name=console-1,id=console-1,
> bus=virtio_serial_pci0.0  \
>     -chardev
> socket,id=devconsole-2,path=/tmp/virtio_port-console-2-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-2,name=console-2,id=console-2,
> bus=virtio_serial_pci0.0  \
>     -chardev
> socket,id=devconsole-3,path=/tmp/virtio_port-console-3-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-3,name=console-3,id=console-3,
> bus=virtio_serial_pci0.0  \
>     -chardev
> socket,id=devconsole-4,path=/tmp/virtio_port-console-4-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-4,name=console-4,id=console-4,
> bus=virtio_serial_pci0.0 \
>     -device driver=virtio-serial-pci,id=virtio_serial_pci1,max_ports=31  \
>     -chardev
> socket,id=devconsole-5,path=/tmp/virtio_port-console-5-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-5,name=console-5,id=console-5,
> bus=virtio_serial_pci1.0  \
>     -chardev
> socket,id=devconsole-6,path=/tmp/virtio_port-console-6-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-6,name=console-6,id=console-6,
> bus=virtio_serial_pci1.0  \
>     -chardev
> socket,id=devconsole-7,path=/tmp/virtio_port-console-7-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-7,name=console-7,id=console-7,
> bus=virtio_serial_pci1.0  \
>     -chardev
> socket,id=devconsole-8,path=/tmp/virtio_port-console-8-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-8,name=console-8,id=console-8,
> bus=virtio_serial_pci1.0  \
>     -chardev
> socket,id=devconsole-9,path=/tmp/virtio_port-console-9-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-9,name=console-9,id=console-9,
> bus=virtio_serial_pci1.0 \
>     -device driver=virtio-serial-pci,id=virtio_serial_pci2,max_ports=31  \
>     -chardev
> socket,id=devconsole-10,path=/tmp/virtio_port-console-10-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-10,name=console-10,id=console-10,
> bus=virtio_serial_pci2.0  \
>     -chardev
> socket,id=devconsole-11,path=/tmp/virtio_port-console-11-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-11,name=console-11,id=console-11,
> bus=virtio_serial_pci2.0  \
>     -chardev
> socket,id=devconsole-12,path=/tmp/virtio_port-console-12-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-12,name=console-12,id=console-12,
> bus=virtio_serial_pci2.0  \
>     -chardev
> socket,id=devconsole-13,path=/tmp/virtio_port-console-13-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-13,name=console-13,id=console-13,
> bus=virtio_serial_pci2.0  \
>     -chardev
> socket,id=devconsole-14,path=/tmp/virtio_port-console-14-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-14,name=console-14,id=console-14,
> bus=virtio_serial_pci2.0 \
>     -device driver=virtio-serial-pci,id=virtio_serial_pci3,max_ports=31  \
>     -chardev
> socket,id=devconsole-15,path=/tmp/virtio_port-console-15-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-15,name=console-15,id=console-15,
> bus=virtio_serial_pci3.0  \
>     -chardev
> socket,id=devconsole-16,path=/tmp/virtio_port-console-16-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-16,name=console-16,id=console-16,
> bus=virtio_serial_pci3.0  \
>     -chardev
> socket,id=devconsole-17,path=/tmp/virtio_port-console-17-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-17,name=console-17,id=console-17,
> bus=virtio_serial_pci3.0  \
>     -chardev
> socket,id=devconsole-18,path=/tmp/virtio_port-console-18-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-18,name=console-18,id=console-18,
> bus=virtio_serial_pci3.0  \
>     -chardev
> socket,id=devconsole-19,path=/tmp/virtio_port-console-19-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-19,name=console-19,id=console-19,
> bus=virtio_serial_pci3.0 \
>     -device driver=virtio-serial-pci,id=virtio_serial_pci4,max_ports=31  \
>     -chardev
> socket,id=devconsole-20,path=/tmp/virtio_port-console-20-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-20,name=console-20,id=console-20,
> bus=virtio_serial_pci4.0  \
>     -chardev
> socket,id=devconsole-21,path=/tmp/virtio_port-console-21-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-21,name=console-21,id=console-21,
> bus=virtio_serial_pci4.0  \
>     -chardev
> socket,id=devconsole-22,path=/tmp/virtio_port-console-22-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-22,name=console-22,id=console-22,
> bus=virtio_serial_pci4.0  \
>     -chardev
> socket,id=devconsole-23,path=/tmp/virtio_port-console-23-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-23,name=console-23,id=console-23,
> bus=virtio_serial_pci4.0  \
>     -chardev
> socket,id=devconsole-24,path=/tmp/virtio_port-console-24-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-24,name=console-24,id=console-24,
> bus=virtio_serial_pci4.0 \
>     -device driver=virtio-serial-pci,id=virtio_serial_pci5,max_ports=31  \
>     -chardev
> socket,id=devconsole-25,path=/tmp/virtio_port-console-25-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-25,name=console-25,id=console-25,
> bus=virtio_serial_pci5.0  \
>     -chardev
> socket,id=devconsole-26,path=/tmp/virtio_port-console-26-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-26,name=console-26,id=console-26,
> bus=virtio_serial_pci5.0  \
>     -chardev
> socket,id=devconsole-27,path=/tmp/virtio_port-console-27-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-27,name=console-27,id=console-27,
> bus=virtio_serial_pci5.0  \
>     -chardev
> socket,id=devconsole-28,path=/tmp/virtio_port-console-28-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-28,name=console-28,id=console-28,
> bus=virtio_serial_pci5.0  \
>     -chardev
> socket,id=devconsole-29,path=/tmp/virtio_port-console-29-20130812-190723-
> yoNWQZ3M,server,nowait \
>     -device
> virtconsole,chardev=devconsole-29,name=console-29,id=console-29,
> bus=virtio_serial_pci5.0  \
>     -chardev
> socket,id=seabioslog_id_20130812-190723-yoNWQZ3M,path=/tmp/seabios-20130812-
> 190723-yoNWQZ3M,server,nowait \
>     -device
> isa-debugcon,chardev=seabioslog_id_20130812-190723-yoNWQZ3M,iobase=0x402 \
>     -device driver=ich9-usb-uhci1,id=usb1 \
>     -drive
> file=/home/medic/Work/Projekty/autotest/autotest/client/tests/virt/shared/
> data/images/winXP-32.qcow2,if=none,id=drive-ide0-0-0,media=disk,format=qcow2,
> aio=native \
>     -device
> driver=ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
>     -device
> driver=virtio-net-pci,mac=9a:cb:cc:cd:ce:cf,id=idJQ17RY,netdev=idHAFvrh  \
>     -netdev
> user,id=idHAFvrh,hostfwd=tcp::5001-:10022,hostfwd=tcp::5002-:10023  \
>     -m 1024  \
>     -smp 2,maxcpus=2,cores=1,threads=1,sockets=2  \
>     -cpu 'SandyBridge' \
>     -drive
> file=/home/medic/Work/Projekty/autotest/autotest/client/tests/virt/shared/
> data/isos/windows/winutils.iso,if=none,id=drive-ide0-0-1,media=cdrom,
> format=raw \
>     -device
> driver=ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1  \
>     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
>     -vnc :1  \
>     -rtc base=localtime,clock=host,driftfix=none  \
>     -boot order=cdn,once=c,menu=off \
>     -enable-kvm

After destroying the VM and recreating it again, the VM booted up into normal mode, but not all ports were present. (the same ports are missing as before).
Comment 31 Cole Robinson 2013-12-03 00:28:55 UTC 
Okay this should be fixed with latest published virtio-win drivers. If there are any other lingering issues, please reproduce with the latest drivers and file a new bug report.
Comment 32 Lukáš Doktor 2014-01-09 12:08:47 UTC 
Hi Cole, thank you, it works fine now. I tested one and multiple pci cards with lots of ports and it worked correctly.