I've setup kdump to capture a kernel dump over ssh for bug 613429 which is a hard crash.
Unfortunately kdump doesn't seem to work.
When I trigger a manual crash with an echo to the usual place, I get a blank screen. Numlock still responds. Is this a kernel modesetting problem?
I wait a few minutes, and see no file appear on the kdump server.
The root partition of the crashing box is encrypted - but I am not dumping to a local disk.
My network card is under the control of NetworkManager.
Relevant lines from kdump.conf:
core_collector makedumpfile -c -d 31
First, the kexec-tools you used have lots of known problems, so I am not surprise that you may hit one of those. It might be worth trying kexec-tools-2.0.0-33.fc14 from koji which fixed those.
Second, to get the meaningful logs for us to debug, try to setup a serial console and paste the kdump process here. Alternatively, try to disable KMS by adding nomodeset to the kernel command line and trigger the crash from a VT (not from X), and attach the screenshot which has the kdump process messages.
cai, you need to stop telling people that . Theres an update working through bohdi with all the missing updates for F-13 that people can use.
Serial logs and a sosreport, as you said will be helpful here. disabling KMS isn't going hurt, but its not likley to fix this issue. The logs are really what we need.
Okay I will wait for the update.
How can I get you the logs though? This is a desktop, I don't know of any recent desktops that have a serial connection.
The disk is encrypted so I doubt saving there will work.
Should I just wait?
plug a usbserial dongle into it, that should work just fine.
(In reply to comment #4)
> plug a usbserial dongle into it, that should work just fine.
How does this work?
I plug a usb to serial converter into my computer. Then where does the serial end go? Into another converter that converts to usb?
You plug in a NULL modem cable and connect it to a second system running kermit or minimcom or other serial communications port. Then you set up the system your debugging to add a console to ttyS0 so that kernel messages are output to that port. You can use the secondary system to record the kernel messages and attach them here.
How can I use a NULL modem cable to do this? Where does ttyS0 come from, I don't have any real serial connection. Please can you give me different instructions?
I have a modern pc with no serial anything that is crashing.
I have a modern laptop with no serial anything.
How do I get a crash dump onto the laptop?
I've told you how. If you need additional instructions on how to connect a serial console to your pc, you can consult this:
It doesn't matter that you have a system with no serial ports, thats why they make these:
The bottom line is, if you want to log a boot sequence, this is how you do it, regardless of OS. Your only other option is to manually record and transcribe what you see on the screen.
Trust me, the serial console is the easy way to go.
I can't see where the serial part comes in. I have:
PC (usb) <---some connection---> LAPTOP (usb)
So I plug one of those USB serial converters into my PC, and I end up with:
PC (usb:serial) <--- serial connection ---> LAPTOP
What happens on the laptop side? Do you want me to buy a second usb:serial connector? i.e.
PC (usb:serial) <--- serial connection ---> LAPTOP (serial:usb)
You're making this far more difficult than it needs to be
You're having trouble diagnosing a problem with a kernel boot, and the problem is so early in the boot cycle the system can't record/log messages on its own to any recordable media, so you need to attach an external recording device. Since you don't have a system with a built in management board thats capable of preforming this function, you need a secondary system and a way for the two of them to communicate. That method is a serial NULL modem connection.
Even if you're using USB, the communication protocol is going to be RS232 serial.
Now, to use RS232 serial communications over a system with no real serial ports, you'll have to make a serial port. On systems with USB, thats easy to do. You just buy a usb to serial converter for each system that you want use a serial port on.
Then you buy a NULL modem cable to connect the two serial ports together:
From there you configure the system thats crashing to send boot messages to the serial port (using the serial-console.txt document I described in comment 8), and then you use some serial comms software (like minicom or c-kermit) to watch and record those messages on the system that you're using to monitor.
I understand all of what you wrote from the first time you wrote it.
The problem was that you seemed to assume that I have a serial connection to a machine with a serial connector on it. I don't have such a thing.
> Now, to use RS232 serial communications over a system with no real serial
> ports, you'll have to make a serial port. On systems with USB, thats easy to
> do. You just buy a usb to serial converter for each system that you want use a
> serial port on.
I can't justify buying two USB to serial connectors just to get a kernel error, apologies.
Sorry I should add: thanks very much for taking the time to answer my questions.
let me know if anything changes. I can help you fix this if you manage to obtain the funds to purchase the debug equipment.
Another possibility to capture the second kernel information is via netconsole if you don't want to buy serial dongles. Some information here and there,
(In reply to comment #0)
> The root partition of the crashing box is encrypted - but I am not dumping to a
> local disk.
This might be problematic even if you don't mount it from my observation. Can you try without encrypted rootfs? We are working to get the doc in place to write up the limitation of encrypted rootfs with kdump.
(In reply to comment #15)
> you try without encrypted rootfs? We are working to get the doc in place to
> write up the limitation of encrypted rootfs with kdump.
I can't reinstall the whole box without encryption. Sorry about that.
I should be able to use netconsole on this box, but in my experience it hardly ever works :/
(In reply to comment #16)
> (In reply to comment #15)
> > Can
> > you try without encrypted rootfs? We are working to get the doc in place to
> > write up the limitation of encrypted rootfs with kdump.
> I can't reinstall the whole box without encryption. Sorry about that.
Can you try to add rd_NO_LUKS to you kdump kernel command line like this to skip encrypted fs detection?
# cat /etc/sysconfig/kdump