This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 986761 - guest kernel will print many "serial8250: too much work for irq3" when using kvm with isa-serial
guest kernel will print many "serial8250: too much work for irq3" when using ...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
6.5
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Seth Jennings
Virtualization Bugs
:
: 1133314 (view as bug list)
Depends On:
Blocks: 1159933 1124856 1132123 1209644
  Show dependency treegraph
 
Reported: 2013-07-21 23:26 EDT by Sibiao Luo
Modified: 2015-07-22 03:59 EDT (History)
20 users (show)

See Also:
Fixed In Version: kernel-2.6.32-514.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-07-22 03:59:57 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sibiao Luo 2013-07-21 23:26:46 EDT
Description of problem:
When using kvm with isa-serial via TCP or Unix socket, the guest kernel will print out many "serial8250: too much work for irq3".

Version-Release number of selected component (if applicable):
host info:
kernel-2.6.32-398.el6.x86_64
qemu-kvm-0.12.1.2-2.378.el6.x86_64
guest info:
kernel-2.6.32-398.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1.boot guest with 4 isa-serial via TCP or Unix socket.
e.g:...-chardev socket,id=isa-serial-1,path=/tmp/isa-serial-1,server,nowait -device isa-serial,chardev=isa-serial-1 -chardev socket,id=isa-serial-2,path=/tmp/isa-serial-2,server,nowait -device isa-serial,chardev=isa-serial-2 -chardev socket,id=isa-serial-3,path=/tmp/isa-serial-3,server,nowait -device isa-serial,chardev=isa-serial-3 -chardev socket,id=isa-serial-4,path=/tmp/isa-serial-4,server,nowait -device isa-serial,chardev=isa-serial-4
or
e.g:...-chardev socket,id=isa-serial-1,host=localhost,port=12341,server,nowait -device isa-serial,chardev=isa-serial-1 -chardev socket,id=isa-serial-2,host=localhost,port=12342,server,nowait -device isa-serial,chardev=isa-serial-2 -chardev socket,id=isa-serial-3,host=localhost,port=12343,server,nowait -device isa-serial,chardev=isa-serial-3 -chardev socket,id=isa-serial-4,host=localhost,port=12344,server,nowait -device isa-serial,chardev=isa-serial-4
2.configuration and reboot guest to make the configure effect.
 2.1. open /etc/inittab add:  s1:2345:respawn:/sbin/agetty -L -f /etc/issue 115200 ttyS0 vt100
 2.2. config guest kernel append:  console=ttyS0,115200
 2.3. open /etc/securetty  append: /dev/ttyS0
 2.4. # reboot
3.try to login using every isa serial(ttyS0-S3)
# nc -U /tmp/isa-serial-1
or
# nc localhost 12341

Actual results:
after step 3, can login guest via the 4 isa-serial and can do some basic actions(ls, mkdir, vim, find, rm...) successfully, but guest dmesg will output many "serial8250: too much work for irq3".
# dmesg | grep -i serial
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
serial8250: ttyS2 at I/O 0x3e8 (irq = 4) is a 16550A
serial8250: ttyS3 at I/O 0x2e8 (irq = 3) is a 16550A
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3
serial8250: too much work for irq3

# ls -lh /sys/bus/platform/devices/serial8250/
total 0
lrwxrwxrwx. 1 root root    0 Jul 22 19:00 driver -> ../../../bus/platform/drivers/serial8250
-r--r--r--. 1 root root 4.0K Jul 22 19:05 modalias
drwxr-xr-x. 2 root root    0 Jul 22 19:05 power
lrwxrwxrwx. 1 root root    0 Jul 22 19:00 subsystem -> ../../../bus/platform
drwxr-xr-x. 4 root root    0 Jul 22 19:00 tty
-rw-r--r--. 1 root root 4.0K Jul 22 19:00 uevent

Expected results:
should not flood with the message "serial8250: too much work for irq3".

Additional info:
# /usr/libexec/qemu-kvm -S -M rhel6.5.0 -cpu SandyBridge -enable-kvm -m 4096 -smp 4,sockets=2,cores=2,threads=1 -no-kvm-pit-reinjection -name sluo -uuid 43425b70-86e5-4664-bf2c-3b76699b8bec -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -drive file=/home/RHEL6.5-20130712.n.0-6.5-Server-x64.qcow2,if=none,id=drive-system-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,serial="QEMU-DISK1" -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-system-disk,id=system-disk,bootindex=1 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x5 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=2C:41:38:B6:40:21,bus=pci.0,addr=0x6,bootindex=2 -drive file=/home/my-data-disk.qcow2,if=none,id=drive-data-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop,serial="QEMU-DISK2" -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk,bootindex=3 -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -vnc :1 -spice port=5931,disable-ticketing -monitor stdio -chardev socket,id=isa-serial-1,path=/tmp/isa-serial-1,server,nowait -device isa-serial,chardev=isa-serial-1 -chardev socket,id=isa-serial-2,path=/tmp/isa-serial-2,server,nowait -device isa-serial,chardev=isa-serial-2 -chardev socket,id=isa-serial-3,path=/tmp/isa-serial-3,server,nowait -device isa-serial,chardev=isa-serial-3 -chardev socket,id=isa-serial-4,path=/tmp/isa-serial-4,server,nowait -device isa-serial,chardev=isa-serial-4
Comment 4 Gerd Hoffmann 2014-07-02 03:35:14 EDT
Hmm, fails to reproduce.  Can you retest with latest rhel6.5 please?
Comment 5 Sibiao Luo 2014-07-02 05:53:47 EDT
(In reply to Gerd Hoffmann from comment #4)
> Hmm, fails to reproduce.  Can you retest with latest rhel6.5 please?

Not meet 'serial8250: too much work for irq4' any more now in guest dmesg, but just the first isa-serial work well, other isa-serial fail to login.

[root@amd-6168-256-1 ~]# nc -U /tmp/isa-serial-1

Red Hat Enterprise Linux Server release 6.5 (Santiago)
Kernel 2.6.32-452.el6.x86_64 on an x86_64

vm-83-227.englab.nay.redhat.com login: ^C

[root@vm-83-227 ~]# ^C
[root@amd-6168-256-1 ~]# nc -U /tmp/isa-serial-2

^C
[root@amd-6168-256-1 ~]# nc -U /tmp/isa-serial-3

^C
[root@amd-6168-256-1 ~]# nc -U /tmp/isa-serial-4

^C

host info:
# uname -r && rpm -q qemu-kvm
2.6.32-488.el6.x86_64
qemu-kvm-0.12.1.2-2.428.el6.x86_64

Best Regards,
sluo
Comment 6 Gerd Hoffmann 2014-07-02 07:08:26 EDT
(In reply to Sibiao Luo from comment #5)
> (In reply to Gerd Hoffmann from comment #4)
> > Hmm, fails to reproduce.  Can you retest with latest rhel6.5 please?
> 
> Not meet 'serial8250: too much work for irq4' any more now in guest dmesg,

Good.

> but just the first isa-serial work well, other isa-serial fail to login.

Most likely a guest setup issue, no getty running.

Note that inittab update as described in the initial comment doesn't work
on rhel6 as it uses upstart not sysvinit.  On the console rhel6 will start
a getty automatically, thats why ttyS0 works nevertheless.
Comment 13 Seth Jennings 2014-09-02 10:57:47 EDT
Seems to be addressed by upstream commit included in 3.0:

commit e7328ae1848966181a7ac47e8ae6cddbd2cf55f3
Author: Jiri Slaby <jirislaby@gmail.com>
Commit: Greg Kroah-Hartman <gregkh@suse.de>

    serial: 8250, increase PASS_LIMIT
    
    With virtual machines like qemu, it's pretty common to see "too much
    work for irq4" messages nowadays. This happens when a bunch of output
    is printed on the emulated serial console. This is caused by too low
    PASS_LIMIT. When ISR loops more than the limit, it spits the message.
    
    I've been using a kernel with doubled the limit and I couldn't see no
    problems. Maybe it's time to get rid of the message now?
    
    Signed-off-by: Jiri Slaby <jirislaby@gmail.com>
    Cc: Alan Cox <alan@linux.intel.com>
    Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

diff --git a/drivers/tty/serial/8250.c b/drivers/tty/serial/8250.c
index b40f7b9..f11df87 100644
--- a/drivers/tty/serial/8250.c
+++ b/drivers/tty/serial/8250.c
@@ -81,7 +81,7 @@ static unsigned int skip_txen_test; /* force skip of txen test at init time */
 #define DEBUG_INTR(fmt...)     do { } while (0)
 #endif
 
-#define PASS_LIMIT     256
+#define PASS_LIMIT     512
 
 #define BOTH_EMPTY     (UART_LSR_TEMT | UART_LSR_THRE)
Comment 16 Rafael Aquini 2014-11-26 10:02:53 EST
Patch(es) available on kernel-2.6.32-514.el6
Comment 19 Chao Yang 2015-02-12 01:47:29 EST
Reproduced with kernel-2.6.32-504.el6.x86_64. 

Steps:

1. start a rhel6.6 guest with 'console=tty0 console=ttyS0,115200' appended in kernel line

2. run:
dd if=/dev/urandom of=/dev/console

Actual Result:

Lots of below message in guest dmesg:
...
serial8250: too much work for irq4
...


Verified pass with 2.6.32-530.el6.x86_64. No such message in guest dmesg while dd to /dev/console.  So this issue has been fixed.

CLI:
/usr/libexec/qemu-kvm -name rhel6.6 -S -M rhel6.6.0 -cpu SandyBridge,+invtsc,+erms,+smep,+fsgsbase,+rdrand,+f16c,+osxsave,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme -enable-kvm -m 2048 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -monitor stdio -boot menu=on -rtc base=utc -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/home/rhel6.6.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:3c:b3:07,bus=pci.0 -device usb-tablet,id=input0 -spice port=5900,disable-ticketing,seamless-migration=on -vga qxl -global qxl-vga.ram_size=67108864 -global qxl-vga.vram_size=67108864 -msg timestamp=on -chardev socket,path=/tmp/seril,server,nowait,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
Comment 20 Gerd Hoffmann 2015-02-13 03:19:10 EST
*** Bug 1133314 has been marked as a duplicate of this bug. ***
Comment 22 Lisa Mitchell 2015-06-24 12:39:58 EDT
Pasting in comment from NEC, as they verified the fix:

Hello Lisa,

> BZ 986761,   tied to QXCR1001367026:  NEC:"serial8250: too much work for irq4" 
> msg is shown on VM console
> Is fixed in RHEL 6.7  all releases since the Alpha, from the version 
> of kernel posted with the patch.
> Can NEC verify that it is fixed on your systems, since you found the defect?

The fix just increases PASS_LIMIT, I think it's not real fix.
So, we can reproduce this problem if we write a lot of message into console.
However, this fix which comes from upstream is acceptable for NEC. 
Please close QXCR1001367026.

Test1.
  dd if=/dev/zero of=/dev/console
  
  No problem

Test2.
  dd if=/dev/zero of=/dev/console bs=1 &
  dd if=/dev/zero of=/dev/console bs=1 &
           :

  If several applications wrote message into the console at the same time,
  The problem can reproduce, of course. The limit is not enough in this case.

This problem affects only Application which can open /dev/console.
All kernel messages are not affected.

Thanks,
Taichi


I am marking this as verified from HP standpoint.
Comment 24 errata-xmlrpc 2015-07-22 03:59:57 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1272.html

Note You need to log in before you can comment on or make changes to this bug.