Bug 1370351

AFAIK there is no support for than 32 ports and the max value of max_ports is 31 (one port is reserved for the virtio-serial's internal console queue).

Does this bug reproduced with max_ports=31?

(In reply to Gal Hammer from comment #1)
> AFAIK there is no support for than 32 ports and the max value of max_ports
> is 31 (one port is reserved for the virtio-serial's internal console queue).

max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
Do you mean max_ports=31 is the windows limit?Is there any reason for this?
I tried on rhel guest,data can be transferred with nr=32.

> Does this bug reproduced with max_ports=31?
If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range port id specified, max. allowed: 30)

(In reply to lijin from comment #2)
> (In reply to Gal Hammer from comment #1)
> > AFAIK there is no support for than 32 ports and the max value of max_ports
> > is 31 (one port is reserved for the virtio-serial's internal console queue).
> 
> max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
> Do you mean max_ports=31 is the windows limit?Is there any reason for this?
> I tried on rhel guest,data can be transferred with nr=32.

No. 32 ports is the current QEMU limit.

https://fedoraproject.org/wiki/Features/VirtioSerial : "max_ports: This option can be used to set the maximum number of allowed ports for this device. The default is 31 (the current maximum)."

> > Does this bug reproduced with max_ports=31?
> If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range
> port id specified, max. allowed: 30)

Does it reproduced with port id 30? or any other value within the 32 ports limit?

Thanks.

(In reply to Gal Hammer from comment #3)
> (In reply to lijin from comment #2)
> > (In reply to Gal Hammer from comment #1)
> > > AFAIK there is no support for than 32 ports and the max value of max_ports
> > > is 31 (one port is reserved for the virtio-serial's internal console queue).
> > 
> > max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
> > Do you mean max_ports=31 is the windows limit?Is there any reason for this?
> > I tried on rhel guest,data can be transferred with nr=32.
> 
> No. 32 ports is the current QEMU limit.
> 
> https://fedoraproject.org/wiki/Features/VirtioSerial : "max_ports: This
> option can be used to set the maximum number of allowed ports for this
> device. The default is 31 (the current maximum)."

Apparently,this document is out of date.
rhel7.3 qemu limit is 511,not 31,I have tried on qemu-kvm-rhev-2.6.0-21.el7.x86_64

> > > Does this bug reproduced with max_ports=31?
> > If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range
> > port id specified, max. allowed: 30)
> 
> Does it reproduced with port id 30? or any other value within the 32 ports
> limit?

it works normally when nr<30

(In reply to lijin from comment #4)
> (In reply to Gal Hammer from comment #3)
> > (In reply to lijin from comment #2)
> > > (In reply to Gal Hammer from comment #1)
> > > > AFAIK there is no support for than 32 ports and the max value of max_ports
> > > > is 31 (one port is reserved for the virtio-serial's internal console queue).
> > > 
> > > max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
> > > Do you mean max_ports=31 is the windows limit?Is there any reason for this?
> > > I tried on rhel guest,data can be transferred with nr=32.
> > 
> > No. 32 ports is the current QEMU limit.
> > 
> > https://fedoraproject.org/wiki/Features/VirtioSerial : "max_ports: This
> > option can be used to set the maximum number of allowed ports for this
> > device. The default is 31 (the current maximum)."
> 
> Apparently,this document is out of date.
> rhel7.3 qemu limit is 511,not 31,I have tried on
> qemu-kvm-rhev-2.6.0-21.el7.x86_64

Can you back your claim of 511 ports support?

When you tested it, didn't you see error messages in the terminal window?

> > > > Does this bug reproduced with max_ports=31?
> > > If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range
> > > port id specified, max. allowed: 30)
> > 
> > Does it reproduced with port id 30? or any other value within the 32 ports
> > limit?
> 
> it works normally when nr<30

(In reply to Gal Hammer from comment #5)

> Can you back your claim of 511 ports support?
> 
> When you tested it, didn't you see error messages in the terminal window?

when I boot with 512 max_ports,qemu has following message:

/usr/libexec/qemu-kvm -device virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,max_ports=512 -chardev socket,path=/tmp/tt0,server,nowait,id=chardev0 -device virtserialport,nr=31,chardev=chardev0,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port0
qemu-kvm: -device virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,max_ports=512: maximum ports supported: 511


In guest,cmd just return:"The system cannot find the file specified.0 file(s) copied."No other errors.

(In reply to lijin from comment #6)
> (In reply to Gal Hammer from comment #5)
> 
> > Can you back your claim of 511 ports support?
> > 
> > When you tested it, didn't you see error messages in the terminal window?
> 
> when I boot with 512 max_ports,qemu has following message:
> 
> /usr/libexec/qemu-kvm -device
> virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,
> max_ports=512 -chardev socket,path=/tmp/tt0,server,nowait,id=chardev0
> -device
> virtserialport,nr=31,chardev=chardev0,name=com.redhat.rhevm.vdsm,bus=virtio-
> serial0.0,id=port0
> qemu-kvm: -device
> virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,
> max_ports=512: maximum ports supported: 511
> 

This looks like a bug introduced with commit b829c2a9 (max virtio queues was increased from 64 to 1024).

> In guest,cmd just return:"The system cannot find the file specified.0
> file(s) copied."No other errors.

Amit? Can you please take a final verdict? Thanks.

Is this specific to windows guests?  Does linux guest behave differently?

(In reply to Amit Shah from comment #8)
> Is this specific to windows guests?  Does linux guest behave differently?

As I said in comment#2,"I tried on rhel guest,data can be transferred with nr=32."
So it's windows guests only.

The limit on the Windows side has been removed in virtio-win commit fc18def with some prep work in commits 48db372 and df9f26e.

Hi,

Reproduced this issue on virtio-win-prewhql-126 version (win10-32 guest)
Verified this issue on virtio-win-prewhql-133 version

Steps same as comment#0

Actual Results:
on virtio-win-prewhql-126 (un-fixed) version, cannot transfer when with "nr>=31"
on virtio-win-prewhql-133 fix version, test with nr=1/30/31/510, all can transfer data successfully.

Base on above, this issue has been fixed already.

Thanks
Yu Wang

change status to verified according to comment#11

Hi Ladi, 

QE found another issue, if we add more than 32 ports for 1 pci, can only transfer data between guest and host via the first 32 ports, when transfer data >= 33 port .

1 boot guest with 37 ports on the same serial controller.

2.on host:
# nc -U /tmp/helloworld36
3.on guest,copy a txt via port36
cmd --> # copy 1.txt \\.\Global.\com.redhat.rhevm.vdsm36


Actual results:
After step3,guest cmd return:The system cannot find the file specified.0 file(s) copied.

Could you check if it is a bug or the limit for windows/qemu/driver?

Thanks
Yu Wang

Hi Yu Wang,

(In reply to Yu Wang from comment #13)
> Hi Ladi, 
> 
> QE found another issue, if we add more than 32 ports for 1 pci, can only
> transfer data between guest and host via the first 32 ports, when transfer
> data >= 33 port .
> 
> 1 boot guest with 37 ports on the same serial controller.
> 
> 2.on host:
> # nc -U /tmp/helloworld36
> 3.on guest,copy a txt via port36
> cmd --> # copy 1.txt \\.\Global.\com.redhat.rhevm.vdsm36
> 
> 
> Actual results:
> After step3,guest cmd return:The system cannot find the file specified.0
> file(s) copied.
> 
> Could you check if it is a bug or the limit for windows/qemu/driver?

Nice catch!

This is a limitation of QEMU. The control queue is created with size 32 so if the device tries to send more than 32 VIRTIO_CONSOLE_PORT_ADD control messages at once, some are dropped and the driver is never notified.

The limit used to be 16 and was increased to 32 in bug 656198.

The relevant code in virtio-serial-bus.c:

    /* TODO: host to guest notifications can get dropped
     * if the queue fills up. Implement queueing in host,
     * this might also make it possible to reduce the control
     * queue size: as guest preposts buffers there,
     * this will save 4Kbyte of guest memory per entry. */

    /* control queue: host to guest */
    vser->c_ivq = virtio_add_queue(vdev, 32, control_in);
    /* control queue: guest to host */
    vser->c_ovq = virtio_add_queue(vdev, 32, control_out);


Can you please open a separate 7.4 QEMU BZ for this? The workaround is to create the VM with <= 32 ports and add the rest in smaller batches later using QMP/HMP.

Thanks!
Ladi

(In reply to Ladi Prosek from comment #14)

Hi Ladi
> 
> Nice catch!
> 
> This is a limitation of QEMU. The control queue is created with size 32 so
> if the device tries to send more than 32 VIRTIO_CONSOLE_PORT_ADD control
> messages at once, some are dropped and the driver is never notified.
> 
> The limit used to be 16 and was increased to 32 in bug 656198.
> 
> The relevant code in virtio-serial-bus.c:
> 
>     /* TODO: host to guest notifications can get dropped
>      * if the queue fills up. Implement queueing in host,
>      * this might also make it possible to reduce the control
>      * queue size: as guest preposts buffers there,
>      * this will save 4Kbyte of guest memory per entry. */
> 
>     /* control queue: host to guest */
>     vser->c_ivq = virtio_add_queue(vdev, 32, control_in);
>     /* control queue: guest to host */
>     vser->c_ovq = virtio_add_queue(vdev, 32, control_out);
> 
> 
> Can you please open a separate 7.4 QEMU BZ for this? The workaround is to
> create the VM with <= 32 ports and add the rest in smaller batches later
> using QMP/HMP.

Thanks for your generous help, I create another bug, bug1432297.

Thanks!
Yu Wang

> 
> Thanks!
> Ladi

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2341