Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
When attempting to copy and past large amounts of text from the client to the guest, the spice-agent restarted too many times and finally stopped. This caused the agent to crash. Updates to both the vdagent and virtio-serial driver have imposed a limit on the amount of data able to be copied and pasted, meaning the text will not copy but neither will it crash the agent.
Created attachment 602828[details]
vdagent log for outcome 1
Description of problem:
copy-paste of data > 1MB from the guest to the client
2 outcomes may occur (in addition to the expected one):
(1) agent is killed by the guest, mouse is lost.
vdagent.log (with messages I added to the code for debugging) is attached
(2) corrupted msgs reach the server, and it detaches itself from the vdagent and switches to server mouse mode:
a message of "dispatch_vdi_port_data: invalid port" appears on the server, and
I added special code to verify that the message that precede the "invalid port" message is also corrupted.
Cause of the bug: the agent code doesn't maintain a queue of messages directed to the server/client. Instead, it holds a buffer of size 1MB. If other message than clipboard don't have enough space in the buffer, it is lost,
VDAgent::dispatch_message fails, and the agent is killed.
For clipboard msg, the clipboard data is stored till it is fully sent.
Detailed cause of outcome (1) : the write buffer was full with clipboard data; Then there was an attempt to write another message to the buffer, which failed.
In the case of the log attached, the message was a response to another VD_AGENT_CLIPBOARD_REQUEST - it looks like spice-gtk is sensative to key repeats, and my ctrl-v was sent twice.
Detailed cause of outcome (2): interleaving msgs
In the middle of writing a large clipboard data to the buffer (in several chunks), another message was successfully written to the buffer ==> data corruption.
Version-Release number of selected component (if applicable):
How reproducible:
I was able to always reproduce with copying pictures of size > 6MB
guest: Windows7-32 bit
client: remote-viewer (linux)
Steps to Reproduce:
1.
2.
3.
Actual results:
Expected results:
Additional info:
Thanks Yonit. I really appreciate the time you spent to locate this buggy code. It's strange the bug was a bit hidden when using the old spicec / vioserial. I'll add a protected msg queue as mentioned.
(In reply to comment #0)
> Created attachment 602828[details]
> vdagent log for outcome 1
>
> Description of problem:
> copy-paste of data > 1MB from the guest to the client
>
> 2 outcomes may occur (in addition to the expected one):
> (1) agent is killed by the guest, mouse is lost.
> vdagent.log (with messages I added to the code for debugging) is attached
> (2) corrupted msgs reach the server, and it detaches itself from the vdagent
> and switches to server mouse mode:
> a message of "dispatch_vdi_port_data: invalid port" appears on the server,
> and
> I added special code to verify that the message that precede the "invalid
> port" message is also corrupted.
>
> Cause of the bug: the agent code doesn't maintain a queue of messages
> directed to the server/client. Instead, it holds a buffer of size 1MB. If
> other message than clipboard don't have enough space in the buffer, it is
> lost,
> VDAgent::dispatch_message fails, and the agent is killed.
> For clipboard msg, the clipboard data is stored till it is fully sent.
>
> Detailed cause of outcome (1) : the write buffer was full with clipboard
> data; Then there was an attempt to write another message to the buffer,
> which failed.
> In the case of the log attached, the message was a response to another
> VD_AGENT_CLIPBOARD_REQUEST - it looks like spice-gtk is sensative to key
> repeats, and my ctrl-v was sent twice.
>
> Detailed cause of outcome (2): interleaving msgs
> In the middle of writing a large clipboard data to the buffer (in several
> chunks), another message was successfully written to the buffer ==> data
> corruption.
>
>
> How reproducible:
>
> I was able to always reproduce with copying pictures of size > 6MB
> guest: Windows7-32 bit
> client: remote-viewer (linux)
The attached patch is only part of the message corruption solution, which is still reproduced. It seems like the main suspect is virtio-serial windows driver, similar to bzs 858674 752338 858687.
Notes:
1. Corruption was unreproducible on Fedora 17 guest with virtio serial & spice agent.
2. Spice-server-only test (no qemu at all) generating stream of messages could not reproduce the corruption.
http://cgit.freedesktop.org/spice/spice/tree/server/tests/test_vdagent.c
3. Debug printfs added in qemu/spice-qemu-char.c spice_chr_write() shows a corrupted msgs
dbg spice_chr_write: 2048 port 1 size 2040 proto 1 type 10 op 0 sz 2020
dbg spice_chr_write: 2048 port 1 size 2040 proto 1 type 10 op 0 sz 2020
dbg spice_chr_write: 2048 port 30828432 size 4294965888 proto 1 type 10 op 0 sz 2020
dbg spice_chr_write: 2048 port 1 size 2040 proto 1 type 10 op 0 sz 2020
(./x86_64-softmmu/qemu-system-x86_64:19068): Spice-Warning **: reds.c:841:vdi_port_read_buf_process: invalid port
To isolate the issue we have also written a minimal dummy_agent which does nothing but simply emulate spice guest vdagent without any real functionality. It continuously reads (async) all incoming messages (e.g. mouse moves from spice client) while writing (async) an infinite stream of chunks, each chunk containing a Spice agent message of an illegal type. For each received message, a "." is printed to stdout. For each [batch_count, default 1000] * 2KB chunks written "w%u" is printed to stdout and we Sleep for [interval_ms, default 10] ms, not to starve the reads. With dummy_agent, corruption is easily reproduced after few hundreds of MBs are written.
http://git.engineering.redhat.com/?p=users/agilboa/dummy_agent.git
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
http://rhn.redhat.com/errata/RHBA-2012-1502.html
Created attachment 602828 [details] vdagent log for outcome 1 Description of problem: copy-paste of data > 1MB from the guest to the client 2 outcomes may occur (in addition to the expected one): (1) agent is killed by the guest, mouse is lost. vdagent.log (with messages I added to the code for debugging) is attached (2) corrupted msgs reach the server, and it detaches itself from the vdagent and switches to server mouse mode: a message of "dispatch_vdi_port_data: invalid port" appears on the server, and I added special code to verify that the message that precede the "invalid port" message is also corrupted. Cause of the bug: the agent code doesn't maintain a queue of messages directed to the server/client. Instead, it holds a buffer of size 1MB. If other message than clipboard don't have enough space in the buffer, it is lost, VDAgent::dispatch_message fails, and the agent is killed. For clipboard msg, the clipboard data is stored till it is fully sent. Detailed cause of outcome (1) : the write buffer was full with clipboard data; Then there was an attempt to write another message to the buffer, which failed. In the case of the log attached, the message was a response to another VD_AGENT_CLIPBOARD_REQUEST - it looks like spice-gtk is sensative to key repeats, and my ctrl-v was sent twice. Detailed cause of outcome (2): interleaving msgs In the middle of writing a large clipboard data to the buffer (in several chunks), another message was successfully written to the buffer ==> data corruption. Version-Release number of selected component (if applicable): How reproducible: I was able to always reproduce with copying pictures of size > 6MB guest: Windows7-32 bit client: remote-viewer (linux) Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: