Hide Forgot
Description of problem: The reboot test fails to verify the kdump-generated vmcore file for the nfs version of the test. Version-Release number of selected component (if applicable): v7 1.3 R43
I suspect issues in either communication with the server, or disk space on the server. 1) How much space is available on /var/v7/export/v7-net/var/crash on the server? Does this directory exist, and are there any core files? 2) could you try removing the path setting from /etc/kdump.conf (apparently it's set to v7-net/var/crash), and rerun the test with --device nfs 4) is selinux set to premissive on the server? 3) please attach /etc/kdump.conf after running the test.
(In reply to comment #1) > I suspect issues in either communication with the server, or disk space > on the server. > > 1) How much space is available on /var/v7/export/v7-net/var/crash > on the server? Does this directory exist, and are there any core files? > ==> it has 87GB available on server. I've seen /var/v7/export/var/crash on server and nothing in there. > 2) could you try removing the path setting from /etc/kdump.conf (apparently > it's set to v7-net/var/crash), and rerun the test with --device nfs > ==> uncommented path /var/crash in /etc/kdump.conf on SUT and reran the test: v7 run --test=reboot --device nfs --server 10.1.1.29 ==> attached /etc/kdump.conf on SUT ==> checked /var/crash/ after the test. No core files. (NOTE: if run reboot with --device local. It will produce a core file) > 4) is selinux set to premissive on the server? ==> it was set "disabled". I tried set it to "permissive" and reran the test. Does it make any different between permissive and disabled ? > > 3) please attach /etc/kdump.conf after running the test. ==> Please see attachment (kdump.conf on SUT). ==> attached results.xml, ==> attached /etc/selinux/config on server
Created attachment 500100 [details] kdump.conf of SUT
Created attachment 500101 [details] results.xml after running reboot nfs test
Created attachment 500102 [details] /etc/selinux/config file on server
(In reply to comment #2) > > 2) could you try removing the path setting from /etc/kdump.conf (apparently > > it's set to v7-net/var/crash), and rerun the test with --device nfs > > > ==> uncommented path /var/crash in /etc/kdump.conf on SUT and reran the test: > v7 run --test=reboot --device nfs --server 10.1.1.29 > ==> attached /etc/kdump.conf on SUT Attachment 500101 [details] shows path set to /var/crash (not commented out). Are you sure you commented it out before the run?
I did both (commented it out and just left it as it is), the results are the same since the default path is /var/crash. I've seen the mount from the SUT to the server. Howver, the system could not generate a core via network when the system crashed. [root@eagle3 ~]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/vg_eagle3-lv_root 51606140 8678736 40305964 18% / tmpfs 7753920 0 7753920 0% /dev/shm /dev/sda2 495844 87739 382505 19% /boot /dev/mapper/vg_eagle3-lv_home 218677124 191696 207377284 1% /home 10.1.1.29:/var/v7/export 100792320 8036352 87635968 9% /tmp/v7-reboot-_nPt3i/v7-net [root@eagle3 ~]#
Please test if you can produce an image manually. With kdump configured to use 10.1.1.29:/var/v7/export, use the following commands to cause a kernel panic: > echo 1 > /proc/sys/kernel/sysrq > echo \"c\" > /proc/sysrq-trigger Then mount the directory and check for the image file.
(In reply to comment #8) > Please test if you can produce an image manually. With kdump configured to use > 10.1.1.29:/var/v7/export, use the following commands to cause a kernel panic: > > > echo 1 > /proc/sys/kernel/sysrq > > echo \"c\" > /proc/sysrq-trigger > > Then mount the directory and check for the image file. Executed the above commands, the system crashed, and booted up. But couldn't see any images generated. Please see the console while crashing (console.log attached).
Created attachment 500517 [details] console log when system crashing.
------- Comment From hienn.com 2011-06-01 14:34 EDT------- *** Bug 71840 has been marked as a duplicate of this bug. ***
------- Comment From hienn.com 2011-06-06 12:05 EDT------- Any update ?
(In reply to comment #9) > (In reply to comment #8) > > Please test if you can produce an image manually. With kdump configured to use > > 10.1.1.29:/var/v7/export, use the following commands to cause a kernel panic: > > > > > echo 1 > /proc/sys/kernel/sysrq > > > echo \"c\" > /proc/sysrq-trigger > > > > Then mount the directory and check for the image file. > > Executed the above commands, the system crashed, and booted up. But couldn't > see any images generated. > Please see the console while crashing (console.log attached). This doesn't seem to be a v7 bug, as you can't get a kdump image over NFS manually.
------- Comment From hienn.com 2011-06-09 18:52 EDT------- To fix the kdump and make it to work on NFS. The crashkernel reserved memory appended to the boot command line should change to 512M@64M. After that, the reboot test with nfs works fine.
------- Comment From lxie.com 2011-07-13 13:38 EDT------- (In reply to comment #17) > reboot with nfs network works. So let's close the bug. (In reply to comment #17) > reboot with nfs network works. So let's close the bug.
------- Comment From hienn.com 2011-08-03 13:00 EDT------- the problem came back when testing on RH Test Suite v7-1.3-46 (latest). - Part of Console log: Making device-mapper control node Scanning logical volumes Reading all physical volumes. This may take a while... Found volume group "vg_eagle3" using metadata type lvm2 Activating logical volumes 3 logical volume(s) in volume group "vg_eagle3" now active Free memory/Total memory (free %): 140480 / 219840 ( 63.901 ) mapping eth0 to eth0 ifup: option with empty value " gateway" eth0 failed to come up Restarting system. Will attach entire console log, sos-report, test output.log
Created attachment 516548 [details] entire console log ------- Comment (attachment only) From hienn.com 2011-08-03 13:01 EDT-------
Created attachment 516549 [details] sosreport ------- Comment (attachment only) From hienn.com 2011-08-03 13:02 EDT-------
Created attachment 516550 [details] test output.log ------- Comment (attachment only) From hienn.com 2011-08-03 13:03 EDT-------
------- Comment From hienn.com 2011-08-04 19:41 EDT------- The kdump on local works fine. But network kdump has problem. Please look at the console log, I think the problem is ethernet interface failed to come up : mapping eth0 to eth0 ifup: option with empty value " gateway" eth0 failed to come up
------- Comment From hienn.com 2011-08-17 11:42 EDT------- Let's close this bug. Please refer to bug#74129 for latest status of kdump and RH HWCert reboot tests. *** This bug has been marked as a duplicate of bug 74129 ***