Description of problem: I was testing an upgrade to 5.5.5-m3 on two servers and received a kernel panic in both cases. -- Mount options LABEL=mojo /local/mysql/mojo xfs noatime,nobarrier,allocsize=1g 1 2 LABEL=data0 /local/mysql/data xfs noatime,allocsize=1g 1 2 Binary Logs are going to mojo. MYSQL data (i.e. .frm, .idb, etc) are going to data. Sun hardware: 4140 2xINTEL SSD 160GB g2 Using LVM OS: Kernel: Linux m100452.ningops.com 2.6.18-128.1.10.0.1.el5 #1 SMP Thu May 7 17:49:20 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 5.3 (Tikanga) From MySQL version: MySQL-server-community-5.5.3_m3-1.rhel5.x86_64.rpm TO MySQL version: MySQL-server-5.5.5_m3-1.rhel5.x86_64.rpm How reproducible: Steps to Reproduce: 1. Load the following RPMS: MySQL-client-community-5.5.3_m3-1.rhel5 MySQL-devel-community-5.5.3_m3-1.rhel5 MySQL-shared-community-5.5.3_m3-1.rhel5 MySQL-server-community-5.5.3_m3-1.rhel5 2.Start mysql 3.Load some data 4. Stop mysql 5. remove these rpms MySQL-client-community-5.5.3_m3-1.rhel5 MySQL-devel-community-5.5.3_m3-1.rhel5 MySQL-shared-community-5.5.3_m3-1.rhel5 MySQL-server-community-5.5.3_m3-1.rhel5 6. load the following: MySQL-devel-5.5.5_m3-1.rhel5.x86_64.rpm MySQL-client-5.5.5_m3-1.rhel5.x86_64.rpm MySQL-shared-5.5.5_m3-1.rhel5.x86_64.rpm MySQL-server-5.5.5_m3-1.rhel5.x86_64.rpm 7. start mysql /etc/init.d/mysql start 8. try to run mysql_upgrade 9. wait for the system to panic Actual results: Kernel Panic: Starting mysql: Starting MySQL.......Access to block zero: fs <dm-0> inode: 11745961 start_block : 0 start_off : 200 blkcnt : 0 extent-state : 0 .----------- [cut here ] --------- [please bite here ] --------- Kernel BUG at ...ld/BUILD/xfs-kmod-0.4/_kmod_build_/support/debug.c:57 invalid opcode: 0000 [1] SMP last sysfs file: /devices/pci0000:80/0000:80:0f.0/0000:83:00.0/0000:84:04.0/0000:86:00.1/irq CPU 6 Modules linked in: nfs lockd fscache nfs_acl sunrpc bonding xfs(U) dm_round_robin dm_multipath scsi_dh video hwmon backlight sbs i2c_ec button battery asus_acpi acpi_memhotplug ac parport_pc lp parport joydev e1000e i2c_nforce2 sg pcspkr i2c_core dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod sata_nv libata shpchp mptsas mptscsih mptbase scsi_transport_sas ext3 jbd uhci_hcd ohci_hcd ehci_hcd qla2xxx(U) sd_mod scsi_mod qla2xxx_conf(U) intermodule(U) Pid: 2863, comm: xfsdatad/6 Tainted: G 2.6.18-128.1.10.0.1.el5 #1 RIP: 0010:[<ffffffff884d26b3>] [<ffffffff884d26b3>] :xfs:cmn_err+0xda/0x11f RSP: 0018:ffff810826569c00 EFLAGS: 00010246 RAX: 0000000000000071 RBX: ffffffff884d59cf RCX: ffffffff802f8aa8 .RDX: ffffffff802f8aa8 RSI: 0000000000000297 RDI: ffffffff884ece4c RBP: 0000000000000000 R08: ffffffff802f8aa8 R09: 0000000000000046 R10: ffff8108265698a0 R11: 0000000000000280 R12: 0000000000000297 R13: ffff81041f90f800 R14: 0000000000000000 R15: ffff810826569d30 FS: 000000004ae9d940(0000) GS:ffff8104362d98c0(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: 00000000009c1ee2 CR3: 0000000827e7c000 CR4: 00000000000006e0 Process xfsdatad/6 (pid: 2863, threadinfo ffff810826568000, task ffff810422c7c080) Stack: 0000003000000030 ffff810826569d00 ffff810826569c20 0000000000000250 ffff810827f43500 ffff810827f43500 ffff81042732fa20 0000000000b33aa9 0000000000000000 0000000000000200 0000000000000250 ffffffff884c8e8a Call Trace: [<ffffffff884c8e8a>] :xfs:kmem_zone_alloc+0x56/0xa3 [<ffffffff8002fd81>] __up_write+0x27/0xf2 [<ffffffff884b1b7e>] :xfs:xfs_iomap_write_unwritten+0x1cf/0x221 . [<ffffffff884b1700>] :xfs:xfs_iomap+0x320/0x4eb [<ffffffff884c949a>] :xfs:xfs_end_bio_unwritten+0x0/0x38 [<ffffffff884c94c9>] :xfs:xfs_end_bio_unwritten+0x2f/0x38 [<ffffffff8004d15c>] run_workqueue+0x94/0xe4 [<ffffffff800499dd>] worker_thread+0x0/0x122 [<ffffffff8009d9e3>] keventd_create_kthread+0x0/0xc4 [<ffffffff80049acd>] worker_thread+0xf0/0x122 [<ffffffff8008a4cf>] default_wake_function+0x0/0xe [<ffffffff8009d9e3>] keventd_create_kthread+0x0/0xc4 [<ffffffff8009d9e3>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032383>] kthread+0xfe/0x132 [<ffffffff8005dfb1>] child_rip+0xa/0x11 [<ffffffff8009d9e3>] keventd_create_kthread+0x0/0xc4 [<ffffffff80032285>] kthread+0x0/0x132 [<ffffffff8005dfa7>] child_rip+0x0/0x11 Code: 0f 0b 68 54 82 4d 88 c2 39 00 eb 2b 48 c7 c6 a3 82 4d 88 48 RIP [<ffffffff884d26b3>] :xfs:cmn_err+0xda/0x11f RSP <ffff810826569c00> <0>Kernel panic - not syncing: Fatal exception Expected results: mysql starts Additional info:
looks like starting the mysql server with innodb_use_native_aio=0 fixed the issue.
> Kernel BUG at ...ld/BUILD/xfs-kmod-0.4/_kmod_build_/support/debug.c:57 You hit an assert in an old unsupported external kernel module, built with debugging turned on; this is not a RHEL bug. (RHEL5 does now have real xfs support, talk to RH support folks if you need it)