Bug 440456 - strange behavior with dogtail.rawinput.pressKey()
Summary: strange behavior with dogtail.rawinput.pressKey()
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: dogtail   
(Show other bugs)
Version: 5.2
Hardware: All
OS: Linux
Target Milestone: rc
: ---
Assignee: Zack Cerza
QA Contact: desktop-bugs@redhat.com
Depends On:
TreeView+ depends on / blocked
Reported: 2008-04-03 16:51 UTC by Alexander Todorov
Modified: 2013-04-12 19:37 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2009-10-29 17:07:03 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

Description Alexander Todorov 2008-04-03 16:51:09 UTC
Description of problem:
when using dogtail.rawinput.typeText() or pressKey() all colons are interpreted
as semi-colons from the virt-viewer application.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Login to GNOME (X running should be enough probably)
2. su && virt-install to start a Xen HVM installation
3. Wait for the VNC console to appear (this is the domU console)
4. From another console start python
5. use dogtail.rawinput.pressKey(':') which should generate a key press event at
the VNC console. The virt-viewer needs to be on focus to receive the key stroke
instead of some other application. To achieve this you can use:
time.sleep(5); pressKey(':')  and then swith to virt-viewer so that it is in
focus when the pressKey function is executed.
Actual results:
instead of colon ':' there appears a semi-colon ';'

Expected results:
A colon appears.

Additional info:
This is different from bug #428431 and bug #410291 in the following:
here we're using Dogtail to send keyboard strokes to the virt-viewer application
while it is running on the normal desktop (X server). The previous bugs used to
run the same steps to reproduce but on Xvnc instead of X.

See comments from Michal Babej on other bug:
VNC server doesn't support XKB extension, and
contains this piece of code:

static gboolean
dec_lock_modifiers (SpiDEController *controller, unsigned modifiers)
        return XkbLockModifiers (spi_get_display (), XkbUseCoreKbd, 
                          modifiers, modifiers);

So locking any modifier keys doesn't work through VNC.

What we suppose is the problem is the following:
X server supports XKB extension and because the virt-viewer is a normal GTK
application it should not have problems with handling the generated keyboard
strokes. However virt-viewer uses gtk-vnc widget which we think doesn't know how
to handle the keystrokes. i.e. X reports to at-spi that it supports XKB but
vnc-widget can't handle it. On the other hand when running the same test under
Xvnc server we don't have the problem as Xvnc doesn't support XKB and at-spi
uses other code to generate the key events.

Comment 1 RHEL Product and Program Management 2008-06-04 22:45:30 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update

Comment 5 Daniel Berrange 2008-09-23 14:55:21 UTC
Note that ':' is a shift'd key on US layout keyboards. So, to generate this key symbol requires

Key Press: 'Shift'
Key Press: ';'
Key Release ';'
Key Release 'Shift'

I can see this coming into the KeyEvent handler in gtk-vnc

(virt-viewer:9479): gtk-vnc-DEBUG: Key event 50 0 0 65505
(virt-viewer:9479): gtk-vnc-DEBUG: Key event 47 1 0 58
(virt-viewer:9479): gtk-vnc-DEBUG: Key event 47 1 0 58
(virt-viewer:9479): gtk-vnc-DEBUG: Key event 50 1 0 65505

'47' corresponds to ';' and '50' corresponds to 'Shift'.

When your dogtail example runs we only get

(virt-viewer:9479): gtk-vnc-DEBUG: Key event 47 1 0 58
(virt-viewer:9479): gtk-vnc-DEBUG: Key event 47 1 0 58

missing the Shift key press, so  it gets interpreted as non-shifted, hence the ';'.

Ergo, this is a dogtail bug

Comment 6 Alexander Todorov 2008-09-24 07:11:13 UTC
does this has something to do with at-spi bug #428431 ?

Additional info:
if you try the test described in comment #0 against some other application it works as expected, i.e. Dogtail is sending the Shift-; sequence correctly. 

I still believe there issue is something related to the vnc widget. Please see comment #0 for my initial assumptions.

Comment 7 Daniel Berrange 2008-09-24 09:16:23 UTC
I don't know what 'lock modifiers' is doing, but the core issue is that VNC as a protocol has no concept of modifiers, so whether you lock them on or off has no bearing on things. THe server interprets the SHIFT key press/release event and as such GTK-VNC has to send the SHIFT key press/release to the server. Since dogtail is not giving GTK-VNC any SHIFT key press/release the ':' keysym gets interpreted as ';' on the server. Either Dogtail itself, or the app using dogtail should send the SHIFT key press/release events to make this work

Comment 8 Michal Babej 2008-09-25 16:34:06 UTC
XkbLockModifiers is a call in XKB extension api.

Dogtail is using at-spi to generate keyboard events.

The fixed at-spi (1.7.11-3.el5) contains code which decides at run-time whether it can use XKB extension, and it will only generate shift key events when there's _no_ XKB extension present.

If you used the fixed at-spi, the fact that couldn't see (in comment #5) keyb events for shift means that at-spi detected XKB while the X server in fact didn't have that extension, or the XEvents didn't get through. Either way it's *not* a bug in dogtail, nor in at-spi.

May i ask for output of "xdpyinfo | grep XKEY" in gtk-vnc ? Thanks.

Comment 9 Daniel Berrange 2008-09-25 16:41:52 UTC
> May i ask for output of "xdpyinfo | grep XKEY" in gtk-vnc ?

That is a non-sensical question. GTK-VNC is a GTK widget, not a shell, and the guest X server plays no part in this problem

The problem is in the events being fed to GTK-VNC via the host OS X server, from dogtail. The X server in the guest OS has no part in this. THe problem is on the host side.

Comment 10 Michal Babej 2008-09-25 17:54:28 UTC
Yes the guest X server play a small part in this problem. The xorg where both gtk-vnc and dogtail were run did have XKB.

The generated KeyPress/KeyRelease events for xev window in Xorg + XKB look like this:

KeyPress event, serial 26, synthetic NO, window 0x3400001,
    root 0x45, subw 0x0, time 2590415758, (101,-11), root:(106,510),
    state 0x1, keycode 47 (keysym 0x3a, colon), same_screen YES,
    XLookupString gives 1 bytes: (3a) ":"
    XmbLookupString gives 1 bytes: (3a) ":"
    XFilterEvent returns: False

KeyRelease event, serial 29, synthetic NO, window 0x3400001,
    root 0x45, subw 0x0, time 2590415761, (101,-11), root:(106,510),
    state 0x1, keycode 47 (keysym 0x3a, colon), same_screen YES,
    XLookupString gives 1 bytes: (3a) ":"
    XFilterEvent returns: False

As you can see, there are no Shift keypress/release events in this case, while the symbol lookup is still ":" not ";" - if you don't trust me, try it out. And it also works for any application i've tried so far but virt-viewer.

Alex: the workaround might be disabling xkeyboard extension, for the time when you're testing.

Comment 13 Zack Cerza 2009-10-29 16:51:27 UTC
This is really just a limitation of VNC as far as I can tell; NAK :(

Comment 14 RHEL Product and Program Management 2009-10-29 17:07:03 UTC
Development Management has reviewed and declined this request.  You may appeal
this decision by reopening this request.

Note You need to log in before you can comment on or make changes to this bug.