Hide Forgot
kdump failed to start on Fedora16 alpha PPC64 installed P7 Juno-L system.While running "service kdump start" noticed following error messages in /var/log/messages. #service kdump start Starting kdump (via systemctl): Job failed. See system logs and 'systemctl status' for details. [FAILED] # tail -f /var/log/messages Oct 28 08:39:37 elm17f131 kdump: kexec: failed to load kdump kernel Oct 28 08:39:37 elm17f131 kdump: failed to start up Oct 28 08:39:37 elm17f131 systemd[1]: kdump.service: control process exited, code=exited status=1 Oct 28 08:39:37 elm17f131 systemd[1]: Unit kdump.service entered failed state. --- Steps to reproduce --- 1) Install F16 alpha PPC64 2) append crashkernel entry in /etc.yaboot.conf file as below and reboot the system image=vmlinuz-3.1.0-0.rc9.git0.2.fc16.kh.ppc64 label=linux read-only initrd=/initramfs-3.1.0-0.rc9.git0.2.fc16.kh.ppc64.img append="rd.md=0 rd.lvm=0 LANG=en_US.UTF-8 KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd.luks=0 root=UUID=e5b062f8-09da-401b-a96a-29592371d35e rd.dm=0 crashkernel=128M@64M" 4) Now start the kdump service service kdump start You will notice failure info in terminal console & /var/log/messages --- More information --- #uname -a Linux elm17f131.xxx.xxx.com 3.1.0-0.rc9.git0.2.fc16.kh.ppc64 #1 SMP Wed Oct 12 22:41:01 UTC 2011 ppc64 ppc64 ppc64 GNU/Linux # cat /proc/cmdline ro rd.md=0 rd.lvm=0 LANG=en_US.UTF-8 KEYTABLE=us SYSFONT=latarcyrheb-sun16 rd.luks=0 root=UUID=e5b062f8-09da-401b-a96a-29592371d35e rd.dm=0 crashkernel=128M@64M # rpm -qa | grep -i kdump system-config-kdump-2.0.5-4.fc15.noarch # rpm -qa | grep -i kexec kexec-tools-2.0.0-47.fc16.ppc64 # grep -i DUMP /boot/config-3.1.0-0.rc9.git0.2.fc16.kh.ppc64 CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y CONFIG_CRASH_DUMP=y CONFIG_PHYP_DUMP=y CONFIG_SCSI_IPR_DUMP=y CONFIG_JOYSTICK_JOYDUMP=m ======= I enabled -x in kdump init script and noticed the following. ++ sed s/smp//g + kdump_kver=3.1.0-0.rc9.git0.2.fc16.kh.ppc64 + kdump_kernel=/boot/vmlinux-3.1.0-0.rc9.git0.2.fc16.kh.ppc64kdump + kdump_initrd=/boot/initrd-3.1.0-0.rc9.git0.2.fc16.kh.ppc64kdump.img + '[' '!' -f /boot/vmlinux-3.1.0-0.rc9.git0.2.fc16.kh.ppc64kdump ']' + echo -n 'No kdump kernel image found.' No kdump kernel image found.+ warning ======= Following are the information from the F16 PPC64 installed system: [root@elm17f131 ~]# grep -i CRASH_DUMP /boot/config-3.1.0-0.rc9.git0.2.fc16.kh.ppc64 CONFIG_CRASH_DUMP=y [root@elm17f131 ~]# grep -i RELOCATABLE /boot/config-3.1.0-0.rc9.git0.2.fc16.kh.ppc64 CONFIG_RELOCATABLE=y [root@elm17f131 ~]# rpm -qa | grep -i kdump system-config-kdump-2.0.5-4.fc15.noarch [root@elm17f131 ~]# rpm -qa | grep -i kexec kexec-tools-2.0.0-47.fc16.ppc64
------- Comment From aruna.ibm.com 2011-11-04 02:56 EDT------- Hi , Initrd and kernel for kdump are missing under /boot. Thanks, Aruna
------- Comment From aruna.ibm.com 2011-11-04 04:23 EDT------- (In reply to comment #11) > Hi , > > Initrd and kernel for kdump are missing under /boot. > > Thanks, > Aruna Sorry for the confusion, i mean the logs searches in that path and says image found. Thanks, Aruna.
------- Comment From aruna.ibm.com 2011-11-04 06:37 EDT------- Hi, I edited /etc/sysconfig/kdump to KDUMP_BOOTDIR="/boot" #What is the image type used for kdump KDUMP_IMG="vmlinuz" ---> which was having "vmlinux" before #What is the images extension. Relocatable kernels don't have one KDUMP_IMG_EXT="" ----> this was having "kdump" as extension. now the previous error messages goes away. dmesg | grep crash [ 0.000000] Reserving 256MB of memory at 64MB for crashkernel (System RAM: 4096MB) even the memory for crashkernel is reserved. but still these messages are seen dmesg | grep kdump [ 8.937996] kdump[636]: kexec: failed to load kdump kernel [ 8.940715] kdump[638]: failed to start up can you please let us know why kdump is trying to start from systemctl. [root@elm17f131 ~]# service kdump start Starting kdump (via systemctl): Warning: Unit file of created job changed on disk, 'systemctl --system daemon-reload' recommended. Job failed. See system logs and 'systemctl status' for details. [FAILED] Thanks, Aruna
------- Comment From baude.com 2011-11-07 11:28 EDT------- The updated packages will be pushed to the mirrors when updates are activated. Should be reasonable to expect this soon.
------- Comment From baude.com 2011-12-12 11:22 EDT------- Please provide your yaboot.conf and the amount of memory on this box.
------- Comment From anton.com 2011-12-13 17:31 EDT------- Can we retest without the @XX argument? We now automatically select the base address, and if that fails we have a problem. This is the patch in 3.1: commit 8aa6d359298ad284a202dc43f103e2f8100a6e82 Author: Anton Blanchard <anton> Date: Sun Jul 31 19:27:35 2011 +0000 powerpc: Move kdump default base address to half RMO size on 64bit We are seeing boot failures on some very large boxes even with commit b5416ca9f824 (powerpc: Move kdump default base address to 64MB on 64bit). This patch halves the RMO so both kernels get about the same amount of RMO memory. On large machines this region will be at least 256MB, so each kernel will get 128MB. We cap it at 256MB (small SLB size) since some early allocations need to be in the bolted SLB region. We could relax this on machines with 1TB SLBs in a future patch. Signed-off-by: Anton Blanchard <anton> Signed-off-by: Benjamin Herrenschmidt <benh.org>
------- Comment From anton.com 2011-12-14 18:00 EDT------- We understand the issue now. This box has a 128MB RMO. We split it 64M/64M for the first stage and kdump kernels. RTAS is at top of the RMO (18MB @ 96MB). This means we have 45MB in total for the kdump kernel. The kernel is 20MB and the initrd is 25MB in FC16. We run out of space in the RMO. Since we have a workaround (specifying a base of 32MB), we can work on this upstream and target FC17. Tony Breeds has already submitted the kernel patch: http://patchwork.ozlabs.org/patch/131293/ kexec-tools will need a modification to allow the initrd to sit above the RMO assuming a recent kernel with the above fix.
*** Bug 752124 has been marked as a duplicate of this bug. ***
Created attachment 560832 [details] Proposed patch
------- Comment From vaish123.ibm.com 2012-03-13 05:58 EDT------- Making the Comment #39 external, Applied above patch mentioned in comment #37 & verified this issue on latest fedora GA build on Firebord-L P7 LPAR , this time it did not reproduce the issue. kdump service got started as expected. # systemctl start kdump.service # systemctl status kdump.service kdump.service - Crash recovery kernel arming Loaded: loaded (/lib/systemd/system/kdump.service; static) Active: active (exited) since Fri, 02 Jan 1970 00:44:14 -0500; 12s ago Process: 824 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) CGroup: name=systemd:/system/kdump.service --- log from /var/log/messages --- Jan 2 00:43:58 ltcfbl6fb13 kdumpctl[824]: Detected /etc/kdump.conf or /boot/vmlinuz-3.1.5-6.fc16.ppc64 change Jan 2 00:43:58 ltcfbl6fb13 kdumpctl[824]: Rebuilding /boot/initramfs-3.1.5-6.fc16.ppc64kdump.img Jan 2 00:44:13 ltcfbl6fb13 kdump: kexec: loaded kdump kernel Jan 2 00:44:13 ltcfbl6fb13 kdumpctl[824]: Starting kdump: Jan 2 00:44:13 ltcfbl6fb13 kdump: started up --- cat /proc/cmdline --- root=/dev/mapper/vg_ltcfbl6fb13-lv_root ro rd.md=0 rd.dm=0 rhgb console=hvc0 KEYTABLE=us quiet SYSFONT=True rd.luks=0 rd.lvm.lv=vg_ltcfbl6fb13/lv_root rd.lvm.lv=vg_ltcfbl6fb13/lv_swap LANG=en_US.UTF-8 crashkernel=128M@64M --- uname -a --- Linux xxx 3.1.5-6.fc16.ppc64 #1 SMP Tue Dec 20 21:24:05 UTC 2011 ppc64 ppc64 ppc64 GNU/Linux Thanks... Manas
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
------- Comment From maknayak.com 2012-04-23 13:51 EDT------- verified this issue on fedora 17 PPC64 alpha build on Juno IOCL system , and could reproduce the same problem. Looks like F17 alpha did not pick-up the patch yet . Applied the patch mentioned in comment #37 & this issue got disappeared.kdump service got started as expected. --- uname -a --- Linux elm17f131.beaverton.ibm.com 3.3.0-1.fc17.ppc64 #1 SMP Wed Mar 21 18:52:34 MST 2012 ppc64 ppc64 ppc64 GNU/Linux [root@elm17f131 ~]# systemctl status kdump.service kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; static) Active: active (exited) since Mon, 23 Apr 2012 13:38:33 -0400; 5s ago Process: 633 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) CGroup: name=systemd:/system/kdump.service When do we expect this patch to be in F17 PPC64 ? Thanks... Manas
F17/ppc alpha had an older kexec-tools than what this bug indicates the problem was fixed in. I just built a vastly newer one (2.0.3-45.fc17), please give it a try: http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=507366 thanks!
------- Comment From maknayak.com 2012-04-24 14:16 EDT------- (In reply to comment #43) > F17/ppc alpha had an older kexec-tools than what this bug indicates the > problem was fixed in. > > I just built a vastly newer one (2.0.3-45.fc17), please give it a try: > http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=507366 > > thanks! I tried with the above latest build from the given link on F17 Alpha PPC64 build and it worked as expected. Here is the my finding form the verification: After installed with newer version: [root@elm17f128 ~]# rpm -Uvh kexec-tools-2.0.3-45.fc17.ppc64.rpm Preparing... ########################################### [100%] 1:kexec-tools ########################################### [100%] [root@elm17f128 ~]# grep KDUMP_IMG /etc/sysconfig/kdump KDUMP_IMG="vmlinuz" KDUMP_IMG_EXT="" [root@elm17f128 ~]# systemctl start kdump.service [root@elm17f128 ~]# systemctl status kdump.service kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; disabled) Active: active (exited) since Tue, 24 Apr 2012 13:59:17 -0400; 11s ago Process: 620 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) CGroup: name=systemd:/system/kdump.service [root@elm17f128 ~]# systemctl enable kdump.service ln -s '/usr/lib/systemd/system/kdump.service' '/etc/systemd/system/multi-user.target.wants/kdump.service' Latest kexec build has fixed the issue. Thanks... Manas
------- Comment From clnperez.com 2012-04-24 16:48 EDT------- Thanks all. Should we keep this open and check again to see if this fix makes it into F17 Beta?
------- Comment From maknayak.com 2012-05-21 16:10 EDT------- (In reply to comment #46) > Thanks all. Should we keep this open and check again to see if this fix > makes it into F17 Beta? Verified this issue on Fedora 17 beta ppc64 and it has the fix.Kdump gets started without any issue. I think we can close this issue now. # uname -a Linux miz03.austin.ibm.com 3.3.4-4.fc17.ppc64 #1 SMP Tue May 8 17:19:04 MST 2012 ppc64 ppc64 ppc64 GNU/Linux BOOT_IMAGE=/vmlinuz-3.3.4-4.fc17.ppc64 root=/dev/mapper/vg_miz03-lv_root ro rd.md=0 rd.dm=0 KEYTABLE=us SYSFONT=True rd.lvm.lv=vg_miz03/lv_swap rd.luks=0 rd.lvm.lv=vg_miz03/lv_root LANG=en_US.UTF-8 rhgb quiet crashkernel=128M@64M [root@miz03 ~]# rpm -qa | grep -i kdump system-config-kdump-2.0.5-7.fc17.noarch [root@miz03 ~]# rpm -qa | grep -i kexec kexec-tools-2.0.3-45.fc17.ppc64 Thanks... Manas
------- Comment From clnperez.com 2012-05-21 17:59 EDT------- Closing on our end. This is fixed by the Beta.
This message is a reminder that Fedora 16 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 16. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '16'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 16's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 16 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
closing here as well (comment #17)