Hide Forgot
Description of problem: When run kdump tier1 test crasherloop on tyan-gt24-08.rhts.eng.bos.redhat.com, kdump failed to boot up second kernel. ============================================================================ logger: /usr/bin/rhts-test-runner.sh rhts-test-checkin 127.0.0.1:7099 tyan-gt24-08.rhts.eng.bos.redhat.com 68744 /kernel/kdump/crash-crasher 9000 1535398 04/01/11 05:06:19 JobID:68744 Test:/kernel/kdump/crash-crasher Response:1 logger: /usr/bin/rhts-test-runner.sh rhts-test-update 127.0.0.1:7099 1535398 start rh-tests-kernel-kdump-crash-crasher 04/01/11 05:06:19 testID:1535398 start: /mnt/tests/kernel/kdump/crash-crasher /mnt/tests/kernel/kdump/crash-crasher loaded crasher module logger: 2011-04-01 05:06:29 /usr/bin/rhts-test-runner.sh 7430 240 hearbeat... logger: 2011-04-01 05:07:29 /usr/bin/rhts-test-runner.sh 7430 300 hearbeat... readahead-collector: sorting [-- MARK -- Fri Apr 1 05:10:00 2011] INFO: task jbd2/dm-0-8:482 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. jbd2/dm-0-8 D 0000000000000007 0 482 2 0x00000000 ffff88013839fb20 0000000000000046 0000000000000000 ffff88013839fae4 ffff88013839fa90 ffffffff81012969 ffff88013839fad0 0000000000000282 ffff8801383a1a78 ffff88013839ffd8 000000000000f558 ffff8801383a1a78 Call Trace: [<ffffffff81012969>] ? read_tsc+0x9/0x20 [<ffffffff81098d29>] ? ktime_get_ts+0xa9/0xe0 [<ffffffff8110d300>] ? sync_page+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff8110d33d>] sync_page+0x3d/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff8110d4f3>] wait_on_page_bit+0x73/0x80 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811232a5>] ? pagevec_lookup_tag+0x25/0x40 [<ffffffff8110d90b>] wait_on_page_writeback_range+0xfb/0x190 [<ffffffff8124846f>] ? submit_bio+0x8f/0x120 [<ffffffff8110d9cf>] filemap_fdatawait+0x2f/0x40 [<ffffffffa01a9e80>] jbd2_journal_commit_transaction+0x7f0/0x1490 [jbd2] [<ffffffff8107a19b>] ? try_to_del_timer_sync+0x7b/0xe0 [<ffffffffa01af958>] kjournald2+0xb8/0x220 [jbd2] [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40 [<ffffffffa01af8a0>] ? kjournald2+0x0/0x220 [jbd2] [<ffffffff8108de16>] kthread+0x96/0xa0 [<ffffffff8100c1ca>] child_rip+0xa/0x20 [<ffffffff8108dd80>] ? kthread+0x0/0xa0 [<ffffffff8100c1c0>] ? child_rip+0x0/0x20 INFO: task beah-beaker-bac:1904 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. beah-beaker-b D 0000000000000001 0 1904 1 0x00000080 ffff88013a34bcc8 0000000000000082 0000000000000000 ffff88013a34bc8c 0000000000000001 ffff88013fc24300 ffff880143c15f40 00000000ffffd567 ffff880138d1db38 ffff88013a34bfd8 000000000000f558 ffff880138d1db38 Call Trace: [<ffffffff8110d300>] ? sync_page+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff8110d33d>] sync_page+0x3d/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff8110d4f3>] wait_on_page_bit+0x73/0x80 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811232a5>] ? pagevec_lookup_tag+0x25/0x40 [<ffffffff8110d90b>] wait_on_page_writeback_range+0xfb/0x190 [<ffffffff8110dad8>] filemap_write_and_wait_range+0x78/0x90 [<ffffffff8119f73e>] vfs_fsync_range+0x7e/0xe0 [<ffffffff8119f80d>] vfs_fsync+0x1d/0x20 [<ffffffff8119f84e>] do_fsync+0x3e/0x60 [<ffffffff8119f883>] sys_fdatasync+0x13/0x20 [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b INFO: task rhts-test-runne:14266 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rhts-test-run D 0000000000000000 0 14266 14265 0x00000080 ffff88023ab4d9a8 0000000000000086 0000000000000000 ffff88023ab4d96c ffff88013b628ec0 ffff88013fc24100 ffff880143c15f40 00000001000002e3 ffff88023931c6b8 ffff88023ab4dfd8 000000000000f558 ffff88023931c6b8 Call Trace: [<ffffffffa01a90dd>] do_get_write_access+0x29d/0x500 [jbd2] [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffffa01a9491>] jbd2_journal_get_write_access+0x31/0x50 [jbd2] [<ffffffffa01fd398>] __ext4_journal_get_write_access+0x38/0x80 [ext4] [<ffffffffa01d9863>] ext4_reserve_inode_write+0x73/0xa0 [ext4] [<ffffffffa01d98dc>] ext4_mark_inode_dirty+0x4c/0x1d0 [ext4] [<ffffffffa01d9bd0>] ext4_dirty_inode+0x40/0x60 [ext4] [<ffffffff8119a35b>] __mark_inode_dirty+0x3b/0x160 [<ffffffff8118ac42>] file_update_time+0xf2/0x170 [<ffffffff8104589d>] ? ptep_set_access_flags+0x6d/0x70 [<ffffffff8110f520>] __generic_file_aio_write+0x220/0x480 [<ffffffff811376cd>] ? handle_pte_fault+0x2cd/0xb50 [<ffffffff8110f7ef>] generic_file_aio_write+0x6f/0xe0 [<ffffffffa01d32a1>] ext4_file_write+0x61/0x1e0 [ext4] [<ffffffff811711ca>] do_sync_write+0xfa/0x140 [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40 [<ffffffff81210c6b>] ? selinux_file_permission+0xfb/0x150 [<ffffffff812040d6>] ? security_file_permission+0x16/0x20 [<ffffffff811714c8>] vfs_write+0xb8/0x1a0 [<ffffffff810d1af2>] ? audit_syscall_entry+0x272/0x2a0 [<ffffffff81171f01>] sys_write+0x51/0x90 [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b INFO: task sh:15127 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. sh D 0000000000000002 0 15127 15121 0x00000080 ffff880137bd39b8 0000000000000082 0000000000000000 ffff880137bd397c 0000000000011210 ffff88013fc24500 ffff880143c15f40 000000010000498f ffff880137b5c678 ffff880137bd3fd8 000000000000f558 ffff880137b5c678 Call Trace: [<ffffffff811a29a0>] ? sync_buffer+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff811a29e0>] sync_buffer+0x40/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff811a29a0>] ? sync_buffer+0x0/0x50 [<ffffffff814dade8>] out_of_line_wait_on_bit+0x78/0x90 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811a2996>] __wait_on_buffer+0x26/0x30 [<ffffffffa01e105a>] ext4_find_entry+0x1ba/0x490 [ext4] [<ffffffffa01e137d>] ext4_lookup+0x4d/0x140 [ext4] [<ffffffff8117f42b>] do_lookup+0x18b/0x220 [<ffffffff8117fa29>] __link_path_walk+0x569/0x820 [<ffffffff811803aa>] path_walk+0x6a/0xe0 [<ffffffff8118057b>] do_path_lookup+0x5b/0xa0 [<ffffffff81172ad1>] ? get_empty_filp+0xa1/0x170 [<ffffffff811811eb>] do_filp_open+0xfb/0xd90 [<ffffffff811754d0>] ? chrdev_open+0x0/0x230 [<ffffffff8116e97f>] ? __dentry_open+0x23f/0x360 [<ffffffff8112e15f>] ? kmemdup+0x3f/0x50 [<ffffffff81204296>] ? security_prepare_creds+0x16/0x20 [<ffffffff8109575f>] ? prepare_creds+0x8f/0xb0 [<ffffffff81179ce3>] open_exec+0x33/0x100 [<ffffffff81179e9f>] do_execve+0xef/0x310 [<ffffffff8126d89a>] ? strncpy_from_user+0x4a/0x90 [<ffffffff810095da>] sys_execve+0x4a/0x80 [<ffffffff8100b5ca>] stub_execve+0x6a/0xc0 INFO: task jbd2/dm-0-8:482 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. jbd2/dm-0-8 D 0000000000000007 0 482 2 0x00000000 ffff88013839fb20 0000000000000046 0000000000000000 ffff88013839fae4 ffff88013839fa90 ffffffff81012969 ffff88013839fad0 0000000000000282 ffff8801383a1a78 ffff88013839ffd8 000000000000f558 ffff8801383a1a78 Call Trace: [<ffffffff81012969>] ? read_tsc+0x9/0x20 [<ffffffff81098d29>] ? ktime_get_ts+0xa9/0xe0 [<ffffffff8110d300>] ? sync_page+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff8110d33d>] sync_page+0x3d/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff8110d4f3>] wait_on_page_bit+0x73/0x80 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811232a5>] ? pagevec_lookup_tag+0x25/0x40 [<ffffffff8110d90b>] wait_on_page_writeback_range+0xfb/0x190 [<ffffffff8124846f>] ? submit_bio+0x8f/0x120 [<ffffffff8110d9cf>] filemap_fdatawait+0x2f/0x40 [<ffffffffa01a9e80>] jbd2_journal_commit_transaction+0x7f0/0x1490 [jbd2] [<ffffffff8107a19b>] ? try_to_del_timer_sync+0x7b/0xe0 [<ffffffffa01af958>] kjournald2+0xb8/0x220 [jbd2] [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40 [<ffffffffa01af8a0>] ? kjournald2+0x0/0x220 [jbd2] [<ffffffff8108de16>] kthread+0x96/0xa0 [<ffffffff8100c1ca>] child_rip+0xa/0x20 [<ffffffff8108dd80>] ? kthread+0x0/0xa0 [<ffffffff8100c1c0>] ? child_rip+0x0/0x20 INFO: task beah-beaker-bac:1904 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. beah-beaker-b D 0000000000000001 0 1904 1 0x00000080 ffff88013a34bcc8 0000000000000082 0000000000000000 ffff88013a34bc8c 0000000000000001 ffff88013fc24300 ffff880143c15f40 00000000ffffd567 ffff880138d1db38 ffff88013a34bfd8 000000000000f558 ffff880138d1db38 Call Trace: [<ffffffff8110d300>] ? sync_page+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff8110d33d>] sync_page+0x3d/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff8110d4f3>] wait_on_page_bit+0x73/0x80 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811232a5>] ? pagevec_lookup_tag+0x25/0x40 [<ffffffff8110d90b>] wait_on_page_writeback_range+0xfb/0x190 [<ffffffff8110dad8>] filemap_write_and_wait_range+0x78/0x90 [<ffffffff8119f73e>] vfs_fsync_range+0x7e/0xe0 [<ffffffff8119f80d>] vfs_fsync+0x1d/0x20 [<ffffffff8119f84e>] do_fsync+0x3e/0x60 [<ffffffff8119f883>] sys_fdatasync+0x13/0x20 [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b INFO: task rhts-test-runne:14266 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rhts-test-run D 0000000000000000 0 14266 14265 0x00000080 ffff88023ab4d9a8 0000000000000086 0000000000000000 ffff88023ab4d96c ffff88013b628ec0 ffff88013fc24100 ffff880143c15f40 00000001000002e3 ffff88023931c6b8 ffff88023ab4dfd8 000000000000f558 ffff88023931c6b8 Call Trace: [<ffffffffa01a90dd>] do_get_write_access+0x29d/0x500 [jbd2] [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffffa01a9491>] jbd2_journal_get_write_access+0x31/0x50 [jbd2] [<ffffffffa01fd398>] __ext4_journal_get_write_access+0x38/0x80 [ext4] [<ffffffffa01d9863>] ext4_reserve_inode_write+0x73/0xa0 [ext4] [<ffffffffa01d98dc>] ext4_mark_inode_dirty+0x4c/0x1d0 [ext4] [<ffffffffa01d9bd0>] ext4_dirty_inode+0x40/0x60 [ext4] [<ffffffff8119a35b>] __mark_inode_dirty+0x3b/0x160 [<ffffffff8118ac42>] file_update_time+0xf2/0x170 [<ffffffff8104589d>] ? ptep_set_access_flags+0x6d/0x70 [<ffffffff8110f520>] __generic_file_aio_write+0x220/0x480 [<ffffffff811376cd>] ? handle_pte_fault+0x2cd/0xb50 [<ffffffff8110f7ef>] generic_file_aio_write+0x6f/0xe0 [<ffffffffa01d32a1>] ext4_file_write+0x61/0x1e0 [ext4] [<ffffffff811711ca>] do_sync_write+0xfa/0x140 [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40 [<ffffffff81210c6b>] ? selinux_file_permission+0xfb/0x150 [<ffffffff812040d6>] ? security_file_permission+0x16/0x20 [<ffffffff811714c8>] vfs_write+0xb8/0x1a0 [<ffffffff810d1af2>] ? audit_syscall_entry+0x272/0x2a0 [<ffffffff81171f01>] sys_write+0x51/0x90 [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b INFO: task sh:15127 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. sh D 0000000000000002 0 15127 15121 0x00000080 ffff880137bd39b8 0000000000000082 0000000000000000 ffff880137bd397c 0000000000011210 ffff88013fc24500 ffff880143c15f40 000000010000498f ffff880137b5c678 ffff880137bd3fd8 000000000000f558 ffff880137b5c678 Call Trace: [<ffffffff811a29a0>] ? sync_buffer+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff811a29e0>] sync_buffer+0x40/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff811a29a0>] ? sync_buffer+0x0/0x50 [<ffffffff814dade8>] out_of_line_wait_on_bit+0x78/0x90 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811a2996>] __wait_on_buffer+0x26/0x30 [<ffffffffa01e105a>] ext4_find_entry+0x1ba/0x490 [ext4] [<ffffffffa01e137d>] ext4_lookup+0x4d/0x140 [ext4] [<ffffffff8117f42b>] do_lookup+0x18b/0x220 [<ffffffff8117fa29>] __link_path_walk+0x569/0x820 [<ffffffff811803aa>] path_walk+0x6a/0xe0 [<ffffffff8118057b>] do_path_lookup+0x5b/0xa0 [<ffffffff81172ad1>] ? get_empty_filp+0xa1/0x170 [<ffffffff811811eb>] do_filp_open+0xfb/0xd90 [<ffffffff811754d0>] ? chrdev_open+0x0/0x230 [<ffffffff8116e97f>] ? __dentry_open+0x23f/0x360 [<ffffffff8112e15f>] ? kmemdup+0x3f/0x50 [<ffffffff81204296>] ? security_prepare_creds+0x16/0x20 [<ffffffff8109575f>] ? prepare_creds+0x8f/0xb0 [<ffffffff81179ce3>] open_exec+0x33/0x100 [<ffffffff81179e9f>] do_execve+0xef/0x310 [<ffffffff8126d89a>] ? strncpy_from_user+0x4a/0x90 [<ffffffff810095da>] sys_execve+0x4a/0x80 [<ffffffff8100b5ca>] stub_execve+0x6a/0xc0 INFO: task jbd2/dm-0-8:482 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. jbd2/dm-0-8 D 0000000000000007 0 482 2 0x00000000 ffff88013839fb20 0000000000000046 0000000000000000 ffff88013839fae4 ffff88013839fa90 ffffffff81012969 ffff88013839fad0 0000000000000282 ffff8801383a1a78 ffff88013839ffd8 000000000000f558 ffff8801383a1a78 Call Trace: [<ffffffff81012969>] ? read_tsc+0x9/0x20 [<ffffffff81098d29>] ? ktime_get_ts+0xa9/0xe0 [<ffffffff8110d300>] ? sync_page+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff8110d33d>] sync_page+0x3d/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff8110d4f3>] wait_on_page_bit+0x73/0x80 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811232a5>] ? pagevec_lookup_tag+0x25/0x40 [<ffffffff8110d90b>] wait_on_page_writeback_range+0xfb/0x190 [<ffffffff8124846f>] ? submit_bio+0x8f/0x120 [<ffffffff8110d9cf>] filemap_fdatawait+0x2f/0x40 [<ffffffffa01a9e80>] jbd2_journal_commit_transaction+0x7f0/0x1490 [jbd2] [<ffffffff8107a19b>] ? try_to_del_timer_sync+0x7b/0xe0 [<ffffffffa01af958>] kjournald2+0xb8/0x220 [jbd2] [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40 [<ffffffffa01af8a0>] ? kjournald2+0x0/0x220 [jbd2] [<ffffffff8108de16>] kthread+0x96/0xa0 [<ffffffff8100c1ca>] child_rip+0xa/0x20 [<ffffffff8108dd80>] ? kthread+0x0/0xa0 [<ffffffff8100c1c0>] ? child_rip+0x0/0x20 INFO: task beah-beaker-bac:1904 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. beah-beaker-b D 0000000000000001 0 1904 1 0x00000080 ffff88013a34bcc8 0000000000000082 0000000000000000 ffff88013a34bc8c 0000000000000001 ffff88013fc24300 ffff880143c15f40 00000000ffffd567 ffff880138d1db38 ffff88013a34bfd8 000000000000f558 ffff880138d1db38 Call Trace: [<ffffffff8110d300>] ? sync_page+0x0/0x50 [<ffffffff814da4d3>] io_schedule+0x73/0xc0 [<ffffffff8110d33d>] sync_page+0x3d/0x50 [<ffffffff814dad3f>] __wait_on_bit+0x5f/0x90 [<ffffffff8110d4f3>] wait_on_page_bit+0x73/0x80 [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50 [<ffffffff811232a5>] ? pagevec_lookup_tag+0x25/0x40 [<ffffffff8110d90b>] wait_on_page_writeback_range+0xfb/0x190 [<ffffffff8110dad8>] filemap_write_and_wait_range+0x78/0x90 [<ffffffff8119f73e>] vfs_fsync_range+0x7e/0xe0 [<ffffffff8119f80d>] vfs_fsync+0x1d/0x20 [<ffffffff8119f84e>] do_fsync+0x3e/0x60 [<ffffffff8119f883>] sys_fdatasync+0x13/0x20 [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b [-- MARK -- Fri Apr 1 05:15:00 2011] [-- MARK -- Fri Apr 1 05:20:00 2011] Version-Release number of selected component (if applicable): kernel-2.6.32-128.el6.x86_64 kexec-tools-2.0.0-179.el6.x86_64 How reproducible: tyan-gt24-08.rhts.eng.bos.redhat.com Steps to Reproduce: 1.Install RHEL6.1-20110330.2 2.Upgrade kexec-tools to 2.0.0-179.el6 3.Trigger crash via crasherloop Actual results: kdump failed Expected results: kdump save vmcore Additional info: https://beaker.engineering.redhat.com/recipes/141037 http://beaker-archive.app.eng.bos.redhat.com/beaker-logs/2011/04/687/68744/141037///console.log
Since RHEL 6.1 External Beta has begun, and this bug remains unresolved, it has been rejected as it is not proposed as exception or blocker. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux.