Bug 65270

Summary: readprofile stopped working with 2.4.18-3 kernel
Product: [Retired] Red Hat Linux Reporter: paluch
Component: kernelAssignee: Arjan van de Ven <arjanv>
Status: CLOSED CURRENTRELEASE QA Contact: Ben Levenson <benl>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3   
Target Milestone: ---   
Target Release: ---   
Hardware: athlon   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2004-09-30 15:39:35 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
gdb session of crashed readprofile (with debug info)
none
map file of my 2.4.18 kernel, that causes readprofile crash.
none
patch to catch corrupted map file none

Description paluch 2002-05-21 08:17:37 UTC
From Bugzilla Helper:
User-Agent: Mozilla/4.79 [en] (X11; U; Linux 2.4.18 i686)

Description of problem:
readprofile stopped working with new 2.4.18 kernel. Here is a summary:
2.4.9-31 - works ok
2.4.18-3 - reports 0 calls and 0.00 time for all functions
2.4.18 "plain" - crashes in the middle of the list.


Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1.
boot kernel 2.1.18-3 with profile=2 cmd line option
2.
run
readprofile -m /boot/System.map

3.
	

Actual Results:  For 2.4.18-3:
[root@henryk ~]# uname -a
Linux henryk.blanice.cz 2.4.18-3 #1 Thu Apr 18 07:32:41 EDT 2002 i686 unknown
[root@henryk ~]# readprofile -m /boot/System.map
     0 total                                      0.0000

For my plain 2.4.18:
[root@henryk ~]# uname -a
Linux henryk.blanice.cz 2.4.18 #2 Wed Mar 6 19:51:50 CET 2002 i686 unknown
[root@henryk ~]# readprofile -m /boot/System.map
  3492 default_idle                              72.7500
     2 copy_thread                                0.0004
     2 lcall27                                    0.0061
...
     1 proc_pid_readdir                           0.0000
     2 sys_msgctl                                 0.0000
     4 insert_char                                0.0001
    18 do_format                                  0.0003
     1 proc_bus_pci_ioctl                         0.0002
Segmentation fault (core dumped)



Expected Results:  To behave exactly same as on 2.4.9-31 kernel:
[root@henryk ~]# uname -a
Linux henryk.blanice.cz 2.4.9-31 #1 Tue Feb 26 06:23:51 EST 2002 i686 unknown
[root@henryk ~]# readprofile -m /boot/System.map
  9534 default_idle                             198.6250
     1 __switch_to                                0.0045
     1 setup_sigcontext                           0.0035
     2 do_signal                                  0.0030
     4 system_call                                0.0714
     1 ret_from_exception                         0.0400
    24 probe_irq_on                               0.0882
...



Additional info:

Here is ltrace and excerpt for /boot/System.map when readprofile crashes:
[root@henryk ~]# uname -a
Linux henryk.blanice.cz 2.4.18 #2 Wed Mar 6 19:51:50 CET 2002 i686 unknown
[root@henryk ~]# readprofile -m /boot/System.map
  3492 default_idle                              72.7500
     2 copy_thread                                0.0004
     2 lcall27                                    0.0061
...
     1 proc_pid_readdir                           0.0000
     2 sys_msgctl                                 0.0000
     4 insert_char                                0.0001
    18 do_format                                  0.0003
     1 proc_bus_pci_ioctl                         0.0002
Segmentation fault (core dumped)

ltrace of readprofile:

strcpy(0xbffffa50, "rtnl_unlock")                 = 0xbffffa50
fgets("c0199980 T rtattr_parse\n", 128, 0x0804bb98) = 0xbffff950
sscanf(0xbffff950, 0x08049900, 0xbffff938, 0xbffff940, 0xbffff9d0) = 3
strcpy(0xbffffa50, "rtattr_parse")                = 0xbffffa50
fgets("c020aa40 t pci_fixup_ide_trash\n", 128, 0x0804bb98) = 0xbffff950
sscanf(0xbffff950, 0x08049900, 0xbffff938, 0xbffff940, 0xbffff9d0) = 3
--- SIGSEGV (Segmentation fault) ---
+++ killed by SIGSEGV +++

excerpt from: /boot/System.map
...
c0199980 T rtattr_parse
c020aa40 t pci_fixup_ide_trash
c020aa70 t pci_fixup_latency
...

Comment 1 Arjan van de Ven 2002-06-20 15:57:45 UTC
please try adding "nmi_watchdog=1" as well

Comment 2 paluch 2002-06-20 18:29:03 UTC
Unfortunately I have no luck enabling NMI watched. Got message that local APIC
is enabled but no message about testing NMI and NMI interrupt count was 0.
(tried 2.4.9-31 & 2.4.18-3 from RedHat and also "myself" 2.4.18).
Here is a sample dmesg:

Linux version 2.4.18-3 (bhcompile.redhat.com) (gcc version 2.96
20000731 (Red Hat Linux 7.3 2.96-110)) #1 Thu Apr 18 07:32:41 EDT 2002
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000000fff0000 (usable)
 BIOS-e820: 000000000fff0000 - 000000000fff3000 (ACPI NVS)
 BIOS-e820: 000000000fff3000 - 0000000010000000 (ACPI data)
 BIOS-e820: 00000000ffff0000 - 0000000100000000 (reserved)
On node 0 totalpages: 65520
zone(0): 4096 pages.
zone(1): 61424 pages.
zone(2): 0 pages.
Kernel command line: ide0=ata66 ro root=/dev/hda6  profile=2 nmi_watchdog=1
single
ide_setup: ide0=ata66
Local APIC disabled by BIOS -- reenabling.
Found and enabled local APIC!
Initializing CPU#0
Detected 699.670 MHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 1395.91 BogoMIPS
Memory: 254248k/262080k available (1125k kernel code, 7444k reserved, 802k data,
296k init, 0k highmem)
Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
CPU: Before vendor init, caps: 0183fbff c1c7fbff 00000000, vendor = 2
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 64K (64 bytes/line)
CPU: After vendor init, caps: 0183fbff c1c7fbff 00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU:     After generic, caps: 0183fbff c1c7fbff 00000000 00000000
CPU:             Common caps: 0183fbff c1c7fbff 00000000 00000000
CPU: AMD Duron(tm) Processor stepping 01
Enabling fast FPU save and restore... done.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
mtrr: v1.40 (20010327) Richard Gooch (rgooch.au)
mtrr: detected mtrr type: Intel
PCI: PCI BIOS revision 2.10 entry at 0xfb250, last bus=1
PCI: Using configuration type 1
PCI: Probing PCI hardware
Disabling VIA memory write queue (PCI ID 0305, rev 02): [55] 89 & 1f -> 09
Applying VIA southbridge workaround.
PCI: Disabling Via external APIC routing
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
apm: BIOS version 1.2 Flags 0x07 (Driver version 1.16)
Starting kswapd
VFS: Diskquotas version dquot_6.5.0 initialized
pty: 2048 Unix98 ptys configured
Serial driver version 5.05c (2001-07-08) with MANY_PORTS MULTIPORT SHARE_IRQ
SERIAL_PCI ISAPNP enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
ttyS01 at 0x02f8 (irq = 3) is a 16550A
Real Time Clock Driver v1.10e
block: 480 slots per queue, batch=120
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
VP_IDE: IDE controller on PCI bus 00 dev 39
VP_IDE: chipset revision 6
VP_IDE: not 100% native mode: will probe irqs later
VP_IDE: VIA vt82c686b (rev 40) IDE UDMA100 controller on pci00:07.1
VP_IDE: ATA-66/100 forced bit set (WARNING)!!
    ide0: BM-DMA at 0xd000-0xd007, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xd008-0xd00f, BIOS settings: hdc:DMA, hdd:pio
hda: IBM-DTLA-307030, ATA DISK drive
hdc: Hewlett-Packard CD-Writer Plus 8100, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
hda: 60036480 sectors (30739 MB) w/1916KiB Cache, CHS=3737/255/63, UDMA(100)
ide-floppy driver 0.99.newide
Partition check:
 hda: hda1 hda2 < hda5 hda6 > hda3
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
ide-floppy driver 0.99.newide
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 2048 buckets, 16Kbytes
TCP: Hash tables configured (established 16384 bind 16384)
Linux IP multicast router 0.06 plus PIM-SM
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
RAMDISK: Compressed image found at block 0
Freeing initrd memory: 120k freed
VFS: Mounted root (ext2 filesystem).
Journalled Block Device driver loaded
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Freeing unused kernel memory: 296k freed
Adding Swap: 522076k swap-space (priority -1)
EXT3 FS 2.4-0.9.17, 10 Jan 2002 on ide0(3,6), internal journal
MSDOS FS: IO charset iso8859-2
MSDOS FS: Using codepage 852
ide-floppy driver 0.99.newide
hdc: ATAPI 24X CD-ROM CD-R/RW drive, 1024kB Cache, DMA
Uniform CD-ROM driver Revision: 3.12
hdc: DMA disabled

and /proc/cpuinfo:
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 6
model           : 3
model name      : AMD Duron(tm) Processor
stepping        : 1
cpu MHz         : 699.677
cache size      : 64 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat
pse36 mmx fxsr syscall mmxext 3dnowext 3dnow
bogomips        : 1395.91

------------------------------------------------------------------
The behaviour of readprofile dit not changed (exactly same as described above).

At least I tried to recompile readprofile with debug info -g. Please see further
attachments.


Comment 3 paluch 2002-06-20 18:30:21 UTC
Created attachment 61925 [details]
gdb session of crashed readprofile (with debug info)

Comment 4 paluch 2002-06-20 18:34:24 UTC
Created attachment 61926 [details]
map file of my 2.4.18 kernel, that causes readprofile crash.

Comment 5 paluch 2002-06-20 18:56:08 UTC
Created attachment 61928 [details]
patch to catch corrupted map file

Comment 6 paluch 2002-06-20 19:01:58 UTC
Added patch, that catches buffer overrun in while loop. But not yet sure what is
the real problem. Maybe - readprofile does not deal with functions in modules
(they seem to not be in /boot/System.map - possible cause of readprofile crash?)

Also - empty output for 2.4.18-3 kernel is still mysterious.

Comment 7 Bugzilla owner 2004-09-30 15:39:35 UTC
Thanks for the bug report. However, Red Hat no longer maintains this version of
the product. Please upgrade to the latest version and open a new bug if the problem
persists.

The Fedora Legacy project (http://fedoralegacy.org/) maintains some older releases, 
and if you believe this bug is interesting to them, please report the problem in
the bug tracker at: http://bugzilla.fedora.us/