Bug 484175

Summary: sysrq-t produces corrupted output
Product: [Fedora] Fedora Reporter: Nathanael Noblet <nathanael>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 10CC: kernel-maint, kmcmartin
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-11-18 10:22:03 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
dmesg / messages - corrupted output
none
F10 i386 syslog
none
Copy of /var/log/dmesg (f10 i386)
none
F10 x86_64 copy of /var/log/dmesg
none
dmesg captured output none

Description Nathanael Noblet 2009-02-05 02:44:32 EST
Created attachment 330968 [details]
dmesg / messages - corrupted output

Description of problem:
When issuing the command (as root) echo 't'>/proc/sysrq-trigger, the output in the messages log is corrupted.

Version-Release number of selected component (if applicable):
all kernels F9-F10 released (including some koji 2.6.28 versions).

How reproducible:
Always

Steps to Reproduce:
1. Boot machine
2. Login
3. su - ->echo 't'>/proc/sysrq-trigger
  
Actual results:

see corruption in outputted thread/process traces

Expected results:

clean output.

Additional info:

This is a long long story. I found this issue when dealing with system instability problems. I had an Asus Mobo, and 2x nvidia video (eventually ATI as well) cards driving 3 displays. I had to use the proprietary driver. I experienced many lockups, mostly when there was heavy network I/O (many bittorrents etc). While tring to find the source of the lockups I was told to look at sysrq-trigger because the lockups only killed the display and I could ssh into the machine and reboot that way. The idea was to look for zombie/dead processes or something. When I did that I found the corruption. I replaced the video cards + mobo with a GigaByte, and ATI video card. The corruption did not go away. I have since done the following.

- verified the integrity of the kernel packages installed
- installed from scratch (previously upgraded from F9 via pre-upgrade)
- installed to another spare HD using defaults for everything

I also booted into a liveCD version of F10 which did *NOT* exhibit this behaviour. The liveCD was an i686 version whereas everything else has been x86_64 to date. I will likely try a new F10 install of i686 to that spare HD to see if that at least narrows it down to arch.

I administer 6 F10 machines, and 4 CentOS machines, 3 of the fedora boxes have this corruption, no CentOS do. All of them are completely different machines, different mobo/ram combinations etc.. But they are all x86_64 except a couple of the CentOS.
Comment 1 Kyle McMartin 2009-02-05 03:25:32 EST
Sorry, it looks like you forgot to attach the output of 'dmesg' and just included the syslog output?

cheers, Kyle
Comment 2 Nathanael Noblet 2009-02-05 11:24:12 EST
oh sorry... Ok well here is the output of the same commands on a fresh install (not updated but that has yet to matter) of F10 - i386.
Also this time when I issued echo 't'>/proc/sysrq-trigger, I got odd output on the terminal like so:

[root@localhost ~]# echo 't'>/proc/sysrq-trigger 

Message from syslogd@localhost at Feb  5 09:19:10 ...
 kernel:] ? tty_ioctlx6cf72>] vfs_ioctl+0x2204a16ioctl+0x24a/0x25d

Message from syslogd@localhost at Feb  5 09:19:10 ...
 kernel:] ? tty_ioctl+0x6cf
[root@localhost ~]# 


I will attach a dmesg and syslog output for the F10 i386 and x86_64
Comment 3 Nathanael Noblet 2009-02-05 11:25:06 EST
Created attachment 331026 [details]
F10 i386 syslog
Comment 4 Nathanael Noblet 2009-02-05 11:27:29 EST
Created attachment 331027 [details]
Copy of /var/log/dmesg (f10 i386)

This is the copy of the dmesg file, because the output of dmesg is simply a fraction of the output found in the /var/log/messages file - however it still has the corruption as far as I can tell.
Comment 5 Nathanael Noblet 2009-02-05 11:33:23 EST
Created attachment 331029 [details]
F10 x86_64 copy of /var/log/dmesg
Comment 6 Kyle McMartin 2009-02-05 13:07:09 EST
Sorry, I meant the dmesg after you tried sysrq-t. The goal is to figure out if the kernel is spewing things out wrong, or if syslog is broken.

regards, Kyle
Comment 7 Nathanael Noblet 2009-02-05 14:18:16 EST
Created attachment 331045 [details]
dmesg captured output

as you suspected the error isn't with the kernel but the output fed to syslog...
This is on my x86_64 machine. Again the liveCD version didn't exhibit this behaviour.. so somehow something changed from the liveCD (i686) and the i386 install DVD, and x86_64
Comment 8 Nathanael Noblet 2009-07-29 14:12:37 EDT
I no longer have this error with F11 kernel 2.6.29.6-213.fc11.x86_64
Comment 9 Bug Zapper 2009-11-18 05:59:21 EST
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping