Bug 1353836

Summary: kexec boot kernel guest had Call Trace
Product: Red Hat Enterprise Linux 7 Reporter: weliao <weliao>
Component: seabiosAssignee: Bandan Das <bdas>
Status: CLOSED CURRENTRELEASE QA Contact: Xueqiang Wei <xuwei>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: bdas, chayang, coli, juzhang, knoel, kraxel, virt-maint, xfu, xuwei
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-15 01:27:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
result1
none
result2 none

Description weliao 2016-07-08 07:15:50 UTC
Description of problem:
Guest run kexec -l followed by kexec -e, had Call Trace.

Version-Release number of selected component (if applicable):
Host:
3.10.0-461.el7.x86_64
qemu-kvm-rhev-2.6.0-11.el7.x86_64
seabios-bin-1.9.1-4.el7.noarch
Guest:
3.10.0-456.el7.x86_64
How reproducible:
50%

Steps to Reproduce:
1.Launch a guest.
/usr/libexec/qemu-kvm -name guest=7.3,debug-threads=on   -machine pc -cpu Opteron_G5 -m 2048 -realtime mlock=off -smp 4,sockets=1,cores=4,threads=1 -uuid 41b02868-1e28-4b59-8cc9-0e7516167755 -no-user-config -nodefaults -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/home/RHEL-Server-7.3-64-virtio.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,snapshot=off -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:eb:79:88,bus=pci.0,addr=0x13  -device usb-tablet,id=input0 -spice port=5900,addr=0.0.0.0,disable-ticketing,streaming-video=all,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -monitor stdio -qmp tcp:0:4444,server,nowait -serial unix:/tmp/console,server,nowait
2.Guest run kexec
kexec -l /boot/vmlinuz-3.10.0-433.el7.x86_64 --initrd=/boot/initramfs-3.10.0-433.el7.x86_64.img --append=root=/dev/mapper/rhel_virtlab--66--85--156-root ro console=tty0 crashkernel=auto rd.lvm.lv=rhel_virtlab-66-85-156/root rd.lvm.lv=rhel_virtlab-66-85-156/swap biosdevname=0 net.ifnames=0 console=ttyS0,115200 LANG=en_US.UTF-8
kexec -e
3.

Actual results:
Guest had Call Trace after reboot.

Expected results:
Guest no Call Trace.

Additional info:
[    3.826285] usb 1-1: SerialNumber: 42
[    3.915514] irq 10: nobody cared (try booting with the "irqpoll" option)
[    3.916017] CPU: 0 PID: 295 Comm: dracut-pre-udev Not tainted 3.10.0-433.el7.x86_64 #1
[    3.916017] Hardware name: Red Hat KVM, BIOS 1.9.1-4.el7 04/01/2014
[    3.916017]  ffff88007f029a00 000000001d0e8b7c ffff88007fc03cb8 ffffffff816692bb
[    3.916017]  ffff88007fc03ce0 ffffffff811292b2 ffff88007f029a00 000000000000000a
[    3.916017]  0000000000000000 ffff88007fc03d20 ffffffff811296d2 ffffffff8108c968
[    3.916017] Call Trace:
[    3.916017]  <IRQ>  [<ffffffff816692bb>] dump_stack+0x19/0x1b
[    3.916017]  [<ffffffff811292b2>] __report_bad_irq+0x32/0xd0
[    3.916017]  [<ffffffff811296d2>] note_interrupt+0x132/0x1f0
[    3.916017]  [<ffffffff8108c968>] ? __do_softirq+0x98/0x280
[    3.916017]  [<ffffffff81126db1>] handle_irq_event_percpu+0xe1/0x1e0
[    3.916017]  [<ffffffff81126eed>] handle_irq_event+0x3d/0x60
[    3.916017]  [<ffffffff8112a38a>] handle_fasteoi_irq+0x5a/0x100
[    3.916017]  [<ffffffff8102d27f>] handle_irq+0xbf/0x150
[    3.916017]  [<ffffffff8167beef>] do_IRQ+0x4f/0xf0
[    3.916017]  [<ffffffff816710ad>] common_interrupt+0x6d/0x6d
[    3.916017]  [<ffffffff8108c968>] ? __do_softirq+0x98/0x280
[    3.916017]  [<ffffffff8167b35c>] call_softirq+0x1c/0x30
[    3.916017]  [<ffffffff8102d375>] do_softirq+0x65/0xa0
[    3.916017]  [<ffffffff8108cd55>] irq_exit+0x115/0x120
[    3.916017]  [<ffffffff8167bef8>] do_IRQ+0x58/0xf0
[    3.916017]  [<ffffffff816710ad>] common_interrupt+0x6d/0x6d
[    3.916017]  <EOI>  [<ffffffff811801f0>] ? get_page_from_freelist+0x360/0x9a0
[    3.916017]  [<ffffffff811809c7>] __alloc_pages_nodemask+0x197/0xba0
[    3.916017]  [<ffffffff813116c2>] ? put_dec+0x72/0x90
[    3.916017]  [<ffffffff813126b3>] ? number.isra.2+0x323/0x360
[    3.916017]  [<ffffffff811c768a>] alloc_pages_vma+0x9a/0x150
[    3.916017]  [<ffffffff811a1546>] wp_page_copy.isra.57+0x86/0x470
[    3.916017]  [<ffffffff811a326f>] do_wp_page+0x9f/0x520
[    3.916017]  [<ffffffff811a0de4>] ? do_read_fault.isra.43+0xe4/0x130
[    3.916017]  [<ffffffff811a5964>] handle_mm_fault+0x6e4/0xfa0
[    3.916017]  [<ffffffff812069e5>] ? __d_free+0x35/0x40
[    3.916017]  [<ffffffff812075be>] ? dentry_kill+0x14e/0x1b0
[    3.916017]  [<ffffffff81674d4f>] __do_page_fault+0x13f/0x440
[    3.916017]  [<ffffffff81675136>] trace_do_page_fault+0x56/0x150
[    3.916017]  [<ffffffff8167481b>] do_async_page_fault+0x1b/0xd0
[    3.916017]  [<ffffffff816713f8>] async_page_fault+0x28/0x30
[    3.916017] handlers:
[    3.916017] [<ffffffff814617c0>] usb_hcd_irq
[    3.916017] [<ffffffff814617c0>] usb_hcd_irq
[    3.916017] Disabling IRQ #10

Comment 2 Bandan Das 2019-08-07 18:44:23 UTC
(In reply to weliao from comment #0)
> Description of problem:
> Guest run kexec -l followed by kexec -e, had Call Trace.
> 
> Version-Release number of selected component (if applicable):
> Host:
> 3.10.0-461.el7.x86_64
> qemu-kvm-rhev-2.6.0-11.el7.x86_64
> seabios-bin-1.9.1-4.el7.noarch
> Guest:
> 3.10.0-456.el7.x86_64
> How reproducible:
> 50%
> 
> Steps to Reproduce:
> 1.Launch a guest.
> /usr/libexec/qemu-kvm -name guest=7.3,debug-threads=on   -machine pc -cpu
> Opteron_G5 -m 2048 -realtime mlock=off -smp 4,sockets=1,cores=4,threads=1
> -uuid 41b02868-1e28-4b59-8cc9-0e7516167755 -no-user-config -nodefaults -rtc
> base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet
> -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1
> -boot menu=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device
> ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,
> addr=0x6 -device
> ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 -device
> ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 -device
> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive
> file=/home/RHEL-Server-7.3-64-virtio.qcow2,format=qcow2,if=none,id=drive-
> virtio-disk0,snapshot=off -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,
> id=virtio-disk0 -netdev tap,id=hostnet0,vhost=on -device
> virtio-net,netdev=hostnet0,id=net0,mac=52:54:00:eb:79:88,bus=pci.0,addr=0x13
> -device usb-tablet,id=input0 -spice
> port=5900,addr=0.0.0.0,disable-ticketing,streaming-video=all,seamless-
> migration=on -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,
> vgamem_mb=16,bus=pci.0,addr=0x2 -device
> intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
> spicevmc,id=charredir0,name=usbredir -device
> usb-redir,chardev=charredir0,id=redir0 -chardev
> spicevmc,id=charredir1,name=usbredir -device
> usb-redir,chardev=charredir1,id=redir1 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on -monitor
> stdio -qmp tcp:0:4444,server,nowait -serial unix:/tmp/console,server,nowait
> 2.Guest run kexec
> kexec -l /boot/vmlinuz-3.10.0-433.el7.x86_64
> --initrd=/boot/initramfs-3.10.0-433.el7.x86_64.img
> --append=root=/dev/mapper/rhel_virtlab--66--85--156-root ro console=tty0
> crashkernel=auto rd.lvm.lv=rhel_virtlab-66-85-156/root
> rd.lvm.lv=rhel_virtlab-66-85-156/swap biosdevname=0 net.ifnames=0
> console=ttyS0,115200 LANG=en_US.UTF-8
> kexec -e
> 3.
> 
> Actual results:
> Guest had Call Trace after reboot.
> 
> Expected results:
> Guest no Call Trace.
> 
> Additional info:
> [    3.826285] usb 1-1: SerialNumber: 42
> [    3.915514] irq 10: nobody cared (try booting with the "irqpoll" option)

This looks like a device that was not safely shutdown when kexec was called, and now,
the kexec kernel gets an interrupt for a driver that hasn't been initialized yet.

Can you please retry this and see if it still exists ?

Comment 3 Michael 2019-08-07 22:18:04 UTC
(In reply to Bandan Das from comment #2)
> 
> This looks like a device that was not safely shutdown when kexec was called,
> and now,
> the kexec kernel gets an interrupt for a driver that hasn't been initialized
> yet.
> 
> Can you please retry this and see if it still exists ?

That is no problem at all. I keep the needinfo status just avoid I forget this. I will update results once I have. 


Thanks

Comment 4 Michael 2019-08-14 09:02:09 UTC
(In reply to Bandan Das from comment #2)
> 
> This looks like a device that was not safely shutdown when kexec was called,
> and now,
> the kexec kernel gets an interrupt for a driver that hasn't been initialized
> yet.
> 
> Can you please retry this and see if it still exists ?

Hi Bandan:

I have tested this in lastest qemu and RHEL8.1.0 guest. 

I repeated exactly same steps in the original issue and I got different result. Please see the attachment. 


Version:
Host: Red Hat Enterprise Linux release 8.1 Beta (Ootpa)
kernel:4.18.0-131.el8.x86_64
qemu-kvm-4.0.0-6.module+el8.1.0+3736+a2aefea3.x86_64

guest:
kenrel:4.18.0-128.el8.x86_64

Comment 5 Michael 2019-08-14 09:02:44 UTC
Created attachment 1603642 [details]
result1

Comment 6 Michael 2019-08-14 09:03:10 UTC
Created attachment 1603643 [details]
result2