Bug 450918

Created attachment 308971 [details]
A screenshot of the messages displayed on the console when using the mouse.

The same problem has been encountered on various VMware products, such as
VMware Workstation 6.x and Vmware Server 3.x, when install RHEL4.7Beta AS/WS/ES
VMs.  RHEL4.7Beta WS has 2.6.9-70.EL kernel.

Without touching the mouse, users may install the RHEL4.7Beta guest OS 
in the text mode and login into the text console without X running.  
As long as moving the mouse, the messages of "atkbd.c: Unknown key pressed ..."
are displayed on the console and the keyboard no longer works properly.

RHEL 4.7 beta does not follow ps/2 mouse protocol, and starts sending invalid
commands almost immediately. Most systems eventually setup Intellimouse that
supports scroll wheel by sending set sample rate 200, 100, 80 (mouse commands f3
c8 f3 64 f3 50)

mouse command sequence from RHEL 4.7:
-------------------------------------
disable (f5)
reset (ff)
read id (f2) (which returns 00)
unknown command ed ?
unknown command 00 ?
unknown command ed ?
unknown command 00 ?
set rate 0 (f3 00)
enable (f4)
(nothing more)

mouse command sequence from RHEL 5.0:
-------------------------------------
disable (f5)
reset (ff)
read id (f2) (returns 00)
read id (f2) (returns 00)
set defaults (f6)
set rate 10 (f3 0a)
set resolution 0 (e8 00)
set rate 20 (f3 14)
set rate 60 (f3 3c)
set rate 40 (f3 28)
set rate 20 (f3 14)
set rate 20 (f3 14)
set rate 60 (f3 3c)
set rate 40 (f3 28)
set rate 20 (f3 14)
set rate 20 (f3 14)
read id (f2) (returns 00)
set resolution 0 (e8 00)
set resolution 0 (e8 00)
set resolution 0 (e8 00)
set resolution 0 (e8 00)
status request (e9) returns (00 00 14)
set defaults (f6)
set resolution 0 (e8 00)
set scaling 1:1 (e6)
set scaling 1:1 (e6)
set scaling 1:1 (e6)
status request (e9) returns (00 00 64)
set resolution 0 (e8 00)
set scaling 2:1 (e7)
set scaling 2:1 (e7)
set scaling 2:1 (e7)
status request (e9) returns (10 00 64)
set resolution 3 (e8 03)
set scaling 1:1 (e6)
set scaling 1:1 (e6)
set scaling 1:1 (e6)
status request (e9) returns (00 03 64)
set resolution 0 (e8 00)
set scaling 1:1 (e6)
set scaling 1:1 (e6)
set scaling 1:1 (e6)
status request (e9) returns (00 00 64)
unknown command e1 ?
reset (ff)
set rate 200 (f3 c8)
set rate 100 (f3 64)
set rate 80 (f3 50)
read id (f2) (returns 03) (Intellimouse supported - 4-byte packets)
...

Sure looks like something basic changed in RHEL 4.7 ps/2 mouse support.

Stratus:  Did this problem occur when installing earlier RHEL 4.7 Beta or
development builds?  Does RHEL 4.6 install correctly?  

This problem was encountered install RHEL AS 4.0 Update 7 beta using snapshots 
one and two on both 32-bit and 64-bit systems.  This problem does not occur 
using RHEL AS 4.0 Update 6.

Thanks for the update.   Did Stratus test any of the RHEL 4.7 development builds
, e.g. pre-Beta release timeframe?  I will need to narrow down the kernel
revision information as I do not have vmware server available to test on, the
problem will need to be resolved by inspection.

Note the command sequence sent to the mouse appears to be keyboard commands:

RHEL 4.7 command sequence sent to mouse
---------------------------------------
disable (f5)   // same for mouse and keyboard
reset (ff)     // same for mouse and keyboard
read id (f2) (which returns 00)  // same for mouse and keyboard
unknown command ed  // unknown mouse - set leds for keyboard
unknown command 00  // unknown mouse - led data for keyboard
unknown command ed  // unknown mouse - set leds for keyboard
unknown command 00  // unknown mouse - led data for keyboard
set rate 0 (f3 00)  // same for mouse and keyboard
enable (f4)  // same for mouse and keyboard 
(nothing more)

So was there some simplification/consolidation in keyboard and mouse initialization?

In an attempt to differentiate whether this is a kernel issue, vs X input
methods, etc... can you try a RHEL4.6 install then drop in the 4.7 kernel?  Or
similarly, try a RHEL4.7 beta install and swap in the 4.6 kernel.

This appears to be a kernel problem.  I took a virtual RHEL AS 4.6 system 
running a 2.6.9-67.0.7ELsmp kernel, and installed RPM's kernel-smp-2.6.9-
72.EL.x86_64.rpm and kernel-smp-devel-2.6.9-72.EL.x86_64.rpm.  The exact same 
problem is encountered.  Even if I moved the mouse during the text mode booting 
process, I would experience problems.  So this does not appear to be an X 
problem, but rather a kernel problem.

As an additional test, I took a virtual system running RHEL 4.7 beta snapshot 1 
and installed the U6 kernel of 2.6.9-67.0.7ELsmp (kernel and kernel-devel).  I 
rebooted the virtual system on the U6 kernel and everything worked fine.  There 
were no mouse or console graphic problems with the U6 kernel running on the U7 
beta release.  This helps to prove that the problem seems to be kernel related.

Kurt/Brett,

I have built a test kernel with one of the patches removed. That's the only
patch which touches the  
mouse/keyboard related code. We suspect it might have introduced the regression.
To confirm, can you please try following test kernels and see if it resolves the
problem.

Test kernels are available at following site.

http://people.redhat.com/vgoyal/.vmware/

I applied the test kernel from the comment#17 in a RHEL4.7-WSbeta VM.
Boot the VM with this kernel, the mouse problem is gone. 

I applied the test kernel from the comment#17 in a RHEL4.7-WSbeta VM.
Boot the VM with this kernel, the mouse problem is gone. 

Thanks Wei. Good to know that problem is gone after reverting one patch. We will
dive deeper to find out why it is happening.

+1 on comment#19 ... booting with kernel-smp-2.6.9-76.EL.vgoyal.vmware.i686.rpm
does not cause the console keypress output when moving the mouse around on the
guest display.

The messages from Comment #1 are from atkbd_interrupt which would be called
from serio_interrupt based upon what port was passed in from i8042_interrupt.
And in this case it seems i8042_interrupt is treating the mouse movements
as keyboard data. I will need to build a debug kernel to determine how the
system is getting into this state.

Tested the kernel on Jan's setup with no failures to report.

With regard to comment#22, I think this occurs because the ps/2 mouse is being
initialized as a keyboard as shown in comment #11.

We have found a VMware problem that is related to this issue.  When the invalid
"keyboard" commands are sent to the PS/2 mouse, the mouse responds with ACK
(0xfa) when it should respond with NACK (0xfe).  The ACK allows the invalid
command sequence to continue.  Nevertheless, keyboard commands should not be
routed to the mouse.

When removing the polling timer, we did change some of the init code,
I have added some debug code to the kernel-smp-2.6.9-75.EL.TEST.bz450918.1 kernel,
would you please supply the dmesg output after booting up? 

http://people.redhat.com/dmilburn/

Created attachment 310711 [details]
dmesg (kernel-smp-2.6.9-75.EL.TEST.bz450918.1)

Created attachment 310714 [details]
dmesg for TEST.bz450918.kernel

This looks odd which is consistent with Comment#25

input: AT Raw Set 2 keyboard on isa0060/serio1
input: AT Translated Set 2 keyboard on isa0060/serio0

James, would you please attach the dmesg output from Comment#21? Thank you

James or Wei,

Would you please repeat the test on kernel-smp-2.6.9-75.EL.TEST.bz450918.2 and
supply the dmesg output? Thank you.

http://people.redhat.com/dmilburn/

Created attachment 310736 [details]
dmesg (kernel-smp-2.6.9-75.EL.TEST.bz450918.2)

Created attachment 310738 [details]
dmesg output for kernel.TEST.bz450918.2

I thought we may not be hitting all the init code since we moved it from
i8042_open into i8042_init, but that doesn't look to be the case.

The serio driver tries to match each port with the approriate driver, and 
it looks like the atkbd driver is claiming serio1 as a keyboard device.
I will need to add some debug in the atkbd driver's init and probe code to
furthur debug.

input: AT Raw Set 2 keyboard on isa0060/serio1 <===

Please retest with kernel-smp-2.6.9-75.EL.TEST.bz450918.3 and supply the dmesg
output, this should give us a better idea on why the atkbd driver is claiming
serio1, thanks.

http://people.redhat.com/dmilburn/

(In reply to comment #37)
> Please retest with kernel-smp-2.6.9-75.EL.TEST.bz450918.3 and supply the dmesg

Boot the VM with this kernel, it doesn't response any keypress or mouse movement.
I can't login to get dmesg.  

Created attachment 310867 [details]
dmesg output for TEST.bz450918.3 kernel.

Wei, looks like we were updating the BZ at the same time, I thought since you
were having problems with the .3 kernel that there maybe too many debug
statements. After Comment #38, I went ahead and built another .4 kernel with
less debugging, though it still looks at the atkbd commands/responses. Also,
I didn't see (input: AT Raw Set 2 keyboard on isa0060/serio1) in Comment #39,
can you please boot the .4 kernel and see if that works better?

http://people.redhat.com/dmilburn/

David,

For the comment#39, the keyboard/mouse still didn't work. I managed 
to get the content of dmesg buffer via some other way:)

I'll try your .4 kernel.

The keyboard/mouse still doesn't work with .4 kernel as with .3 kernel.
I didn't see "input: AT Raw Set 2 keyboard on isa0060/serio1" or
"input: AT Translated Set 2 keyboard on isa0060/serio0" either in 'messages'.

In dmesg for .3 kernel, there is the error

drivers/input/serio/i8042.c: f2 -> i8042 (kbd-data) [4614]
ATKBD_PROBE Get ID command FAILED set LEDS -1515913216

Wei, I think the debugging is causing commands to fail, I was specifically
looking for the get ID command, but according to the output sendbyte would
have failed, plus it looks like that was for serio0. So, we really need to
get back to where you can login normally, I will need to narrow down the debug
some more.

This bugzilla has Keywords: Regression.  

Since no regressions are allowed between releases, 
it is also being proposed as a blocker for this release.  

Please resolve ASAP.

Wei, I have another test kernel (.5) available with minimal debug, you should be
able to login normally and get the dmesg output, this should show the atkbd probe
concerning the get id command for serio0 and serio1. Thanks.

http://people.redhat.com/dmilburn/

Created attachment 310945 [details]
dmesg output for TEST.bz450918.5 kernel.

The keyboard works with .5 kernel. I got dmesg output and attach it here.

Ok, thanks, the keyboard driver is sending the ATKBD_CMD_GETID to serio1
(before it sends the cmd the driver invalidates the params with 0xa5). And
the command does fail, but, the keyboard driver still treats serio1 as a
keyboard, tries to set the LEDS, returns 0 to atkbd_connect which sets it up as
a keyboard device

"input: AT Raw Set 2 keyboard on isa0060/serio1"

The upstream atkbd driver behaves the same way on the GETID failure, it is also
odd that the patch backed out in Comment#17 didn't change any of the atkbd code.

====Comment #51 dmesg ouput=========================
ATKBD_CONNECT: atkbd->write 1
ATKBD_PROBE serio isa0060/serio1 atkbd_reset 0
ATKBD_COMMAND sending ATKBD_CMD_GETID
ATKBD_COMMAND, ATKBD_CMD_GETID ret 0xa5
ATKBD_PROBE Get ID command FAILED set LEDS     <==ATKBD_CMD_FAILED in probe  
input: AT Raw Set 2 keyboard on isa0060/serio1 <==Still treats as keyboard
ATKBD_CONNECT
ATKBD_CONNECT: TYPE 0x6000000
ATKBD_CONNECT: atkbd->write 1
ATKBD_PROBE serio isa0060/serio0 atkbd_reset 0
ATKBD_COMMAND sending ATKBD_CMD_GETID
ATKBD_COMMAND, ATKBD_CMD_GETID ret 0xab   <==cmd succeeds, ret 0xab
ATKBD_COMMAND, KEYBOARD FOUND 0xab
ATKBD_PROBE serio isa0060/serio0 param[0] 0xab FOUND KEYBOARD
ATKBD_PROBE atkbd->id 43841 atkbd->translated 1
input: AT Translated Set 2 keyboard on isa0060/serio0
=============================================================

Normally, as long as the GETID command succeeds and returns
anything but 0xac or 0xab then the keyboard driver probe would know for sure
that the device isn't a keyboard.

=====dmesg from a x86_64 box====================
ATKBD_PROBE serio isa0060/serio1 atkbd_reset 0
ATKBD_COMMAND sending ATKBD_CMD_GETID
ATKBD_COMMAND, ATKBD_CMD_GETID ret 0x0
ATKBD_COMMAND, NOT A KEYBOARD, ret 0x0  <===succeed, not 0xac or 0xab in param
ATKBD_PROBE serio isa0060/serio1 param[0] 0x0 NOT A KEYBOARD
ATKBD_CONNECT
ATKBD_CONNECT: TYPE 0x6000000
ATKBD_CONNECT: atkbd->write 1
ATKBD_PROBE serio isa0060/serio0 atkbd_reset 0
ATKBD_COMMAND sending ATKBD_CMD_GETID
ATKBD_COMMAND, ATKBD_CMD_GETID ret 0xab <==Succeeded, ret 0xab
ATKBD_COMMAND, KEYBOARD FOUND 0xab
ATKBD_PROBE serio isa0060/serio0 param[0] 0xab FOUND KEYBOARD
ATKBD_PROBE atkbd->id 43841 atkbd->translated 1
input: AT Translated Set 2 keyboard on isa0060/serio0
input: ImPS/2 Logitech Wheel Mouse on isa0060/serio1

Committed in 77.EL . RPMS are available at http://people.redhat.com/vgoyal/rhel4/

For the time being this problem has been resolved by reverting the problematic
patch which was committed in 68.12 with following changelog entry.

-i8042: remove polling timer support (David Milburn) [340561]

Revert git commit id is commit 7913eba6ad6a49aa98d0b4ccf38945ca08d0697a

Note: bz 340561 is little misleading here. Above patch had actually solved bz 246233

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2008-0665.html