Bug 1271915 - Press keys in text mode got sticky output
Press keys in text mode got sticky output
Status: ASSIGNED
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kbd (Show other bugs)
7.2
All Unspecified
medium Severity medium
: rc
: 7.5
Assigned To: Vitezslav Crhonek
qe-baseos-daemons
:
Depends On:
Blocks: 1444027
  Show dependency treegraph
 
Reported: 2015-10-15 02:14 EDT by mazhang
Modified: 2017-11-14 04:30 EST (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Guest screenshot (19.78 KB, image/png)
2015-10-15 02:14 EDT, mazhang
no flags Details

  None (edit)
Description mazhang 2015-10-15 02:14:59 EDT
Created attachment 1083137 [details]
Guest screenshot

Description of problem:


Version-Release number of selected component (if applicable):

Host:
3.10.0-323.el7.ppc64le
qemu-kvm-rhev-2.3.0-30.el7.ppc64le

Guest:
3.10.0-323.el7.ppc64le

How reproducible:
60%

Steps to Reproduce:
1.Start guest:
/usr/libexec/qemu-kvm -name RHEL-7.2 -machine pseries,accel=kvm,usb=off -m 2048 -realtime mlock=off -smp 8,sockets=2,cores=2,threads=2 \
-rtc base=localtime,clock=host,driftfix=slew \
-monitor stdio \
-boot menu=on \
-device usb-ehci,id=usb,bus=pci.0,addr=0x1 \
-drive file=/home/mazhang/RHEL-Server-7.2-ppc64le-virtio.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2,cache=none,rerror=stop,werror=stop,aio=native \
-device virtio-blk-pci,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bus=pci.0,addr=0x3,bootindex=1 \
-vnc :99 -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x5 \
-device usb-kbd,id=input0 \
-device usb-mouse,id=input1 \
-device usb-tablet,id=input2 \
-msg timestamp=on \
-netdev tap,id=hostnet0,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:c4:e7:93,bus=pci.0,addr=0x6 \
-serial unix:/tmp/console0,server,nowait \

2. Change runlevel to 3 or just Ctrl-Alt-F2 switch to text mode.
# systemctl set-default multi-user.target
Removed symlink /etc/systemd/system/default.target.
Created symlink from /etc/systemd/system/default.target to /usr/lib/systemd/system/multi-user.target.

3. Input some commands in guest.

Actual results:
Got sticky output, for example, if press "ls" got "lssssss", see attachment.

Expected results:
Output normal.

Additional info:
With GUI mode not hit this problem.
Comment 2 David Gibson 2016-02-22 04:03:43 EST
Does this problem also occur if using OHCI or XHCI USB controllers instead of EHCI?
Comment 3 mazhang 2016-02-22 21:22:38 EST
(In reply to David Gibson from comment #2)
> Does this problem also occur if using OHCI or XHCI USB controllers instead
> of EHCI?

OHCI and XHCI controllers still hit this problem.

Host:
3.10.0-327.8.1.el7.ppc64le
qemu-kvm-rhev-2.3.0-31.el7_2.5.ppc64le

Guest:
3.10.0-344.el7.ppc64le
Comment 4 Laurent Vivier 2016-04-26 09:46:39 EDT
I'm not able to reproduce this problem.

What is your VNC client?

Did you try with the VNC client of virt-manager?

As USB guest drivers are very time sensitive, do you know if the host has an heavy CPU load?

Tested with:

Host:

3.10.0-327.13.1.el7.ppc64le
qemu-kvm-rhev-2.3.0-31.el7_2.12 / qemu-kvm-rhev-2.3.0-31.el7_2.5

Guest:

3.10.0-327.el7.ppc64le / 3.10.0-327.13.1.el7.ppc64le / 3.10.0-382.el7.ppc64le / 4.6.0-rc3
Comment 5 mazhang 2016-04-27 04:49:04 EDT
(In reply to Laurent Vivier from comment #4)
> I'm not able to reproduce this problem.
> 
> What is your VNC client?

My vnc client is remote-viewer which provide by virt-viewer-2.0-6.el7.x86_64

> 
> Did you try with the VNC client of virt-manager?

No, I'm in Beijing office, the host in Boston, so virt-manager is too slow for me, can't test by virt-manager.
I'm not sure if the high network latency cause this problem.

> 
> As USB guest drivers are very time sensitive, do you know if the host has an
> heavy CPU load?

Both host and client running on low CPU usage. 

> 
> Tested with:
> 
> Host:
> 
> 3.10.0-327.13.1.el7.ppc64le
> qemu-kvm-rhev-2.3.0-31.el7_2.12 / qemu-kvm-rhev-2.3.0-31.el7_2.5
> 
> Guest:
> 
> 3.10.0-327.el7.ppc64le / 3.10.0-327.13.1.el7.ppc64le /
> 3.10.0-382.el7.ppc64le / 4.6.0-rc3


I can reproduce with following configuration.

Host:
3.10.0-327.20.1.el7.ppc64le
qemu-kvm-rhev-2.3.0-31.el7_2.12.ppc64le

Guest:
3.10.0-327.18.2.el7.ppc64le

Client:
virt-viewer-2.0-6.el7.x86_64
Comment 6 Laurent Vivier 2016-04-27 05:25:45 EDT
(In reply to mazhang from comment #5)
> Client:
> virt-viewer-2.0-6.el7.x86_64

OK, I'm able to reproduce it with virt-viewer-3.0-1.fc23.x86_64. It is interesting to see remote-viewer doesn't manage ssh tunneling

So, I'm also able to reproduce the problem with "vinagre" (the VNC viewer I was using) by disabling the ssh tunneling.

Could you try to use "vinagre" with the ssh tunneling enabled?

HOWTO:

    Remote->Connect

        Protocol: VNC
        Host: localhost:99
        X Use Host "<your host>" as a SSH tunnel

This really looks like a protocol latency issue (I'm also using remote machines in Raleigh whereas I'm in the French Alps...), not a USB/ppc64le one.
Comment 7 Laurent Vivier 2016-04-27 12:52:00 EDT
I've added some traces in QEMU:
- at VNC server level, the event is not duplicated
  (one event for key down, one event for key up)
- at HID device emulation level, the event received from VNC server is only
  queued once (one event for key down, one event for key up)

The event seems duplicated later.

As said previously, in my case, the problem doesn't happen with SSH tunneling.
Comment 8 mazhang 2016-04-28 02:30:57 EDT
(In reply to Laurent Vivier from comment #6)
> (In reply to mazhang from comment #5)
> > Client:
> > virt-viewer-2.0-6.el7.x86_64
> 
> OK, I'm able to reproduce it with virt-viewer-3.0-1.fc23.x86_64. It is
> interesting to see remote-viewer doesn't manage ssh tunneling
> 
> So, I'm also able to reproduce the problem with "vinagre" (the VNC viewer I
> was using) by disabling the ssh tunneling.
> 
> Could you try to use "vinagre" with the ssh tunneling enabled?
> 

This problem doesn't happen with ssh tunneling enabled.

Steps:
1. Boot vm on remote host.
2. Install vinagre.
3. Create a ssh tunnel.
    ssh -f -L 5999:localhost:5999 root@10.19.112.66 -N -p 22

4. Connect vm.

> HOWTO:
> 
>     Remote->Connect
> 
>         Protocol: VNC
>         Host: localhost:99
>         X Use Host "<your host>" as a SSH tunnel
> 
> This really looks like a protocol latency issue (I'm also using remote
> machines in Raleigh whereas I'm in the French Alps...), not a USB/ppc64le
> one.
Comment 9 Laurent Vivier 2016-04-28 15:48:49 EDT
I think this problem appears because of the latency of the network:

the delay between the key down event and the key up event is greater than the time defined for the auto-repeat delay.

The auto-repeat delay is normally defined to 250 ms, and in my case the latency (delay of the ping) between my VNC client (in France) and the VNC server (in USA) is 400 ms. I think you are also in this case.

I think SSH allows to avoid this problem by coalescing down and up events. 

To check that this is the real problem, I was not able to change the auto-repeat delay of the keyboard. The kbdrate doesn't work on POWER[1], and I didn't find a way to send an (EV_REP, REP_DELAY) to the keyboard driver.

So I think this problem is not ppc64le specific and not really a bug.

I reassign this bug to the kbd component just to know how to change the keyboard rate...

[1] "Cannot open /dev/port: No such file or directory"
Comment 10 Vitezslav Crhonek 2016-05-03 09:45:44 EDT
(In reply to Laurent Vivier from comment #9)
> 
> To check that this is the real problem, I was not able to change the
> auto-repeat delay of the keyboard. The kbdrate doesn't work on POWER[1], and
> I didn't find a way to send an (EV_REP, REP_DELAY) to the keyboard driver.
> 
> So I think this problem is not ppc64le specific and not really a bug.
> 
> I reassign this bug to the kbd component just to know how to change the
> keyboard rate...
> 
> [1] "Cannot open /dev/port: No such file or directory"

I don't think it's possible set it using kbdrate.

kbdrate uses KDKBDREP ioctl call (or KIOCSRATE on SPARC), but you have to call it from console, not from pseudo terminal, otherwise ioctl fails with ENOTTY.
(This should possibly work on different architectures, but I don't have physical access to them, so I can't test it.)

If ioctl call doesn't succeed kbdrate reads/writes /dev/port, which is arch dependent and can lead to unexpected results (e.g. time out on s390x, nothing visible but "weird" state of terminal on aarch64, ...).

We should probably disable /dev/port manipulation on these architectures (as it is on SPARC).
Comment 11 Laurent Vivier 2016-05-03 11:15:31 EDT
Seeking in the kernel sources, I've found an ioctl() allowing to change the delay.

The following program allows to change the delay for an USB keyboard (in the case of our VM).

Is there something similar in the kbd component?
Thanks.

// cc -o rate rate.c
// sudo ./rate 2500 330

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/kd.h>
#include <linux/input.h>

#define EVDEV   "/dev/input/by-id/usb-QEMU_QEMU_USB_Keyboard_42-event-kbd"
int main(int argc, char **argv)
{
        struct kbd_repeat repeat;
        int fd;
        int ret;

        if (argc < 3) {
                printf("%s <delay ms> <period ms>\n", argv[0]);
                exit(1);
        }

        fd = open(EVDEV, O_RDWR);
        if (fd == -1) {
                perror("Cannot open " EVDEV);
                exit(1);
        }
        ret = ioctl(fd, EVIOCGREP, &repeat);
        if (ret == -1) {
                perror("Cannot ioctl(EVIOCGREP) " EVDEV);
                exit(1);
        }
        printf("Delay %d Period %d\n", repeat.delay, repeat.period);
        repeat.delay = atol(argv[1]);
        repeat.period = atol(argv[2]);
        ret = ioctl(fd, EVIOCSREP, &repeat);
        if (ret == -1) {
                perror("Cannot ioctl(EVIOCSREP) " EVDEV);
                exit(1);
        }
        printf("Delay %d Period %d\n", repeat.delay, repeat.period);
        close(fd);
}
Comment 12 Vitezslav Crhonek 2016-05-05 04:23:25 EDT
(In reply to Laurent Vivier from comment #11)
> Is there something similar in the kbd component?
> Thanks.

No, there's nothing similar, just kbdrate.
Comment 15 Laurent Vivier 2017-11-14 04:30:27 EST
*** Bug 1510414 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.