RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1370351 - [virtio-win][vioser] serial port can not transfer data when nr>=31
Summary: [virtio-win][vioser] serial port can not transfer data when nr>=31
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: virtio-win
Version: 7.3
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Ladi Prosek
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1401400
TreeView+ depends on / blocked
 
Reported: 2016-08-26 02:33 UTC by lijin
Modified: 2017-08-01 12:53 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
This update removes the hard-coded limit of 31 supported virtual serial ports. As a result, ports 31 and above can be used on Windows guests, and the number of ports is only limited by the host.
Clone Of:
Environment:
Last Closed: 2017-08-01 12:53:08 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2341 0 normal SHIPPED_LIVE virtio-win bug fix and enhancement update 2017-08-01 16:52:38 UTC

Description lijin 2016-08-26 02:33:04 UTC 
Description of problem:
when set nr of virtio-serial-pci larger than 30,virtserialport can not transfer data.

Version-Release number of selected component (if applicable):
virtio-win-prewhql-126
qemu-kvm-rhev-2.6.0-21.el7.x86_64
kernel-3.10.0-494.el7.x86_64
seabios-1.9.1-4.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.boot windows10 guest with virtserialport,set max_ports=511 and nr=31:
-device virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,max_ports=511 -chardev socket,path=/tmp/tt0,server,nowait,id=chardev0 -device virtserialport,nr=31,chardev=chardev0,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port0 \

2.on host:
# nc -U /tmp/tt0

3.on guest,copy a txt via port0
cmd --> # copy 1.txt \\.\Global.\com.redhat.rhevm.vdsm

4.shutdown guest and boot with nr=30

5.repeat step2 and step3

Actual results:
After step3,guest cmd return:The system cannot find the file specified.0 file(s) copied.
After step5,guest cmd return:1 file(s) copied;And host can receive the message.


Expected results:
serial port can transfer data when nr>=31

Additional info:
In rhel7.2,the max_ports of virtio-serial-pci can be set to 31,nr can be set to 30
In rhel7.3,the max_ports of virtio-serial-pci can be set to 511,nr can be set to 510
Try with virtio-win rhel7.2 version(virtio-win-prewhql-108),this issue can be reproduced,so it's not a regression bug.
Comment 1 Gal Hammer 2016-08-26 14:26:32 UTC 
AFAIK there is no support for than 32 ports and the max value of max_ports is 31 (one port is reserved for the virtio-serial's internal console queue).

Does this bug reproduced with max_ports=31?
Comment 2 lijin 2016-08-29 05:24:23 UTC 
(In reply to Gal Hammer from comment #1)
> AFAIK there is no support for than 32 ports and the max value of max_ports
> is 31 (one port is reserved for the virtio-serial's internal console queue).

max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
Do you mean max_ports=31 is the windows limit?Is there any reason for this?
I tried on rhel guest,data can be transferred with nr=32.

> Does this bug reproduced with max_ports=31?
If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range port id specified, max. allowed: 30)
Comment 3 Gal Hammer 2016-08-30 08:37:32 UTC 
(In reply to lijin from comment #2)
> (In reply to Gal Hammer from comment #1)
> > AFAIK there is no support for than 32 ports and the max value of max_ports
> > is 31 (one port is reserved for the virtio-serial's internal console queue).
> 
> max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
> Do you mean max_ports=31 is the windows limit?Is there any reason for this?
> I tried on rhel guest,data can be transferred with nr=32.

No. 32 ports is the current QEMU limit.

https://fedoraproject.org/wiki/Features/VirtioSerial : "max_ports: This option can be used to set the maximum number of allowed ports for this device. The default is 31 (the current maximum)."

> > Does this bug reproduced with max_ports=31?
> If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range
> port id specified, max. allowed: 30)

Does it reproduced with port id 30? or any other value within the 32 ports limit?

Thanks.
Comment 4 lijin 2016-08-30 09:06:24 UTC 
(In reply to Gal Hammer from comment #3)
> (In reply to lijin from comment #2)
> > (In reply to Gal Hammer from comment #1)
> > > AFAIK there is no support for than 32 ports and the max value of max_ports
> > > is 31 (one port is reserved for the virtio-serial's internal console queue).
> > 
> > max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
> > Do you mean max_ports=31 is the windows limit?Is there any reason for this?
> > I tried on rhel guest,data can be transferred with nr=32.
> 
> No. 32 ports is the current QEMU limit.
> 
> https://fedoraproject.org/wiki/Features/VirtioSerial : "max_ports: This
> option can be used to set the maximum number of allowed ports for this
> device. The default is 31 (the current maximum)."

Apparently,this document is out of date.
rhel7.3 qemu limit is 511,not 31,I have tried on qemu-kvm-rhev-2.6.0-21.el7.x86_64

> > > Does this bug reproduced with max_ports=31?
> > If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range
> > port id specified, max. allowed: 30)
> 
> Does it reproduced with port id 30? or any other value within the 32 ports
> limit?

it works normally when nr<30
Comment 5 Gal Hammer 2016-08-30 09:16:12 UTC 
(In reply to lijin from comment #4)
> (In reply to Gal Hammer from comment #3)
> > (In reply to lijin from comment #2)
> > > (In reply to Gal Hammer from comment #1)
> > > > AFAIK there is no support for than 32 ports and the max value of max_ports
> > > > is 31 (one port is reserved for the virtio-serial's internal console queue).
> > > 
> > > max_ports in qemu limit is 511,I can boot up guest with max_ports=511;
> > > Do you mean max_ports=31 is the windows limit?Is there any reason for this?
> > > I tried on rhel guest,data can be transferred with nr=32.
> > 
> > No. 32 ports is the current QEMU limit.
> > 
> > https://fedoraproject.org/wiki/Features/VirtioSerial : "max_ports: This
> > option can be used to set the maximum number of allowed ports for this
> > device. The default is 31 (the current maximum)."
> 
> Apparently,this document is out of date.
> rhel7.3 qemu limit is 511,not 31,I have tried on
> qemu-kvm-rhev-2.6.0-21.el7.x86_64

Can you back your claim of 511 ports support?

When you tested it, didn't you see error messages in the terminal window?

> > > > Does this bug reproduced with max_ports=31?
> > > If max_ports=31,the nr can NOT set to values larger than 30:(Out-of-range
> > > port id specified, max. allowed: 30)
> > 
> > Does it reproduced with port id 30? or any other value within the 32 ports
> > limit?
> 
> it works normally when nr<30
Comment 6 lijin 2016-08-30 09:34:01 UTC 
(In reply to Gal Hammer from comment #5)

> Can you back your claim of 511 ports support?
> 
> When you tested it, didn't you see error messages in the terminal window?

when I boot with 512 max_ports,qemu has following message:

/usr/libexec/qemu-kvm -device virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,max_ports=512 -chardev socket,path=/tmp/tt0,server,nowait,id=chardev0 -device virtserialport,nr=31,chardev=chardev0,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port0
qemu-kvm: -device virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,max_ports=512: maximum ports supported: 511


In guest,cmd just return:"The system cannot find the file specified.0 file(s) copied."No other errors.
Comment 7 Gal Hammer 2016-08-30 09:53:55 UTC 
(In reply to lijin from comment #6)
> (In reply to Gal Hammer from comment #5)
> 
> > Can you back your claim of 511 ports support?
> > 
> > When you tested it, didn't you see error messages in the terminal window?
> 
> when I boot with 512 max_ports,qemu has following message:
> 
> /usr/libexec/qemu-kvm -device
> virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,
> max_ports=512 -chardev socket,path=/tmp/tt0,server,nowait,id=chardev0
> -device
> virtserialport,nr=31,chardev=chardev0,name=com.redhat.rhevm.vdsm,bus=virtio-
> serial0.0,id=port0
> qemu-kvm: -device
> virtio-serial-pci,id=virtio-serial0,disable-legacy=off,disable-modern=off,
> max_ports=512: maximum ports supported: 511
> 

This looks like a bug introduced with commit b829c2a9 (max virtio queues was increased from 64 to 1024).

> In guest,cmd just return:"The system cannot find the file specified.0
> file(s) copied."No other errors.

Amit? Can you please take a final verdict? Thanks.
Comment 8 Amit Shah 2016-09-06 07:01:09 UTC 
Is this specific to windows guests?  Does linux guest behave differently?
Comment 9 lijin 2016-09-06 07:16:55 UTC 
(In reply to Amit Shah from comment #8)
> Is this specific to windows guests?  Does linux guest behave differently?

As I said in comment#2,"I tried on rhel guest,data can be transferred with nr=32."
So it's windows guests only.
Comment 10 Ladi Prosek 2016-09-06 09:29:23 UTC 
The limit on the Windows side has been removed in virtio-win commit fc18def with some prep work in commits 48db372 and df9f26e.
Comment 11 Yu Wang 2017-02-28 05:43:00 UTC 
Hi,

Reproduced this issue on virtio-win-prewhql-126 version (win10-32 guest)
Verified this issue on virtio-win-prewhql-133 version

Steps same as comment#0

Actual Results:
on virtio-win-prewhql-126 (un-fixed) version, cannot transfer when with "nr>=31"
on virtio-win-prewhql-133 fix version, test with nr=1/30/31/510, all can transfer data successfully.

Base on above, this issue has been fixed already.

Thanks
Yu Wang
Comment 12 lijin 2017-02-28 05:48:26 UTC 
change status to verified according to comment#11
Comment 13 Yu Wang 2017-03-13 03:41:47 UTC 
Hi Ladi, 

QE found another issue, if we add more than 32 ports for 1 pci, can only transfer data between guest and host via the first 32 ports, when transfer data >= 33 port .

1 boot guest with 37 ports on the same serial controller.

2.on host:
# nc -U /tmp/helloworld36
3.on guest,copy a txt via port36
cmd --> # copy 1.txt \\.\Global.\com.redhat.rhevm.vdsm36


Actual results:
After step3,guest cmd return:The system cannot find the file specified.0 file(s) copied.

Could you check if it is a bug or the limit for windows/qemu/driver?

Thanks
Yu Wang
Comment 14 Ladi Prosek 2017-03-14 12:33:25 UTC 
Hi Yu Wang,

(In reply to Yu Wang from comment #13)
> Hi Ladi, 
> 
> QE found another issue, if we add more than 32 ports for 1 pci, can only
> transfer data between guest and host via the first 32 ports, when transfer
> data >= 33 port .
> 
> 1 boot guest with 37 ports on the same serial controller.
> 
> 2.on host:
> # nc -U /tmp/helloworld36
> 3.on guest,copy a txt via port36
> cmd --> # copy 1.txt \\.\Global.\com.redhat.rhevm.vdsm36
> 
> 
> Actual results:
> After step3,guest cmd return:The system cannot find the file specified.0
> file(s) copied.
> 
> Could you check if it is a bug or the limit for windows/qemu/driver?

Nice catch!

This is a limitation of QEMU. The control queue is created with size 32 so if the device tries to send more than 32 VIRTIO_CONSOLE_PORT_ADD control messages at once, some are dropped and the driver is never notified.

The limit used to be 16 and was increased to 32 in bug 656198.

The relevant code in virtio-serial-bus.c:

    /* TODO: host to guest notifications can get dropped
     * if the queue fills up. Implement queueing in host,
     * this might also make it possible to reduce the control
     * queue size: as guest preposts buffers there,
     * this will save 4Kbyte of guest memory per entry. */

    /* control queue: host to guest */
    vser->c_ivq = virtio_add_queue(vdev, 32, control_in);
    /* control queue: guest to host */
    vser->c_ovq = virtio_add_queue(vdev, 32, control_out);


Can you please open a separate 7.4 QEMU BZ for this? The workaround is to create the VM with <= 32 ports and add the rest in smaller batches later using QMP/HMP.

Thanks!
Ladi
Comment 15 Yu Wang 2017-03-15 02:54:44 UTC 
(In reply to Ladi Prosek from comment #14)

Hi Ladi
> 
> Nice catch!
> 
> This is a limitation of QEMU. The control queue is created with size 32 so
> if the device tries to send more than 32 VIRTIO_CONSOLE_PORT_ADD control
> messages at once, some are dropped and the driver is never notified.
> 
> The limit used to be 16 and was increased to 32 in bug 656198.
> 
> The relevant code in virtio-serial-bus.c:
> 
>     /* TODO: host to guest notifications can get dropped
>      * if the queue fills up. Implement queueing in host,
>      * this might also make it possible to reduce the control
>      * queue size: as guest preposts buffers there,
>      * this will save 4Kbyte of guest memory per entry. */
> 
>     /* control queue: host to guest */
>     vser->c_ivq = virtio_add_queue(vdev, 32, control_in);
>     /* control queue: guest to host */
>     vser->c_ovq = virtio_add_queue(vdev, 32, control_out);
> 
> 
> Can you please open a separate 7.4 QEMU BZ for this? The workaround is to
> create the VM with <= 32 ports and add the rest in smaller batches later
> using QMP/HMP.

Thanks for your generous help, I create another bug, bug1432297.

Thanks!
Yu Wang

> 
> Thanks!
> Ladi
Comment 18 errata-xmlrpc 2017-08-01 12:53:08 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2341
Note You need to log in before you can comment on or make changes to this bug.