Bug 1104682
| Summary: | DomU boot failure: BUG: soft lockup - CPU#0 stuck for 22s! [systemd-udevd:161] | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Robby <robbyke> | ||||
| Component: | kernel | Assignee: | Dave Airlie <airlied> | ||||
| Status: | CLOSED UPSTREAM | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 21 | CC: | awilliam, drjones, gansalmon, itamar, jonathan, joonas.lehtonen, kernel-maint, ketuzsezr, madhu.chinakonda, mchehab, robatino, robbyke, robinlee.sysu, thucnh, vkuznets | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | RejectedBlocker | ||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2015-11-02 19:28:52 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Dave? FYI: The new update, kernel-3.14.5-200.fc20, has the same issue. kernel-3.14.8-200.fc20.x86_64 still has this issue. kernel-3.14.9-200.fc20.x86_64 still has this issue. Is there anything I can do to speed up the process for a fix? It is a problem for deployments of new VMs. For example doing a net-install will result in an unbootable system after setup finishes, unless you of course uncheck the option to download updates during setup. This causes the VMs to be stuck at default kernel version 3.11.10-301.fc20.x86_64 because the last known working kernel package (3.13.10-200.fc20.x86_64) is no longer available in the update repositories. I'm having the same issue with 3.15.3-200.fc20. After adding 'systemd.log_level=debug systemd.log_target=console' as a kernel parameter the system boots up but has network interface issues. Created attachment 914814 [details]
Call Trace of 3.15.3 booting up in a Xen DomU guest
After upgrading kernal to 3.15.3-200, there is a new boot entry 'Fedora (3.15.3-200.fc20.x86_64) 20 (Heisenbug)' added to to the boot menu but this new entry doesn't work. Fortunately, the original boot entry is still there and it work properly. Further more, the Eclipse Kepler with Aptana 3 plugin installed make my Fedora OS crashes. I'n still working it out, you have any idea please share with me. Thanks alot kernel-3.15.7-200.fc20.x86_64 still has this issue. So, what I was afraid for is happening: Fedora 21 alpha/beta versions are around the corner and this bug is still open. I have just tried Fedora 21 Alpha TC6 and TC7 (which have kernel 3.16.1) and they also do not boot. Someone on IRC in fedora-qa suggested to try a vanilla kernel. So, I did a fresh install of Fedora 20 and I grabbed 3.16.0 and 3.16.2 from kernel.org and used the /boot/config-3.11.10-301.fc20.x86_64 config file as a base config for compiling the vanilla kernels. After installing the vanilla kernels, they booted up fine. So, it seems that something the Fedora Project does to the kernel is causing this issue to occur. Proposed as a Blocker for 21-beta by Fedora user robbyke using the blocker tracking app because: One of the release criteria is that Fedora must successfully boot as a guest, which is currently not the case under Xen, details are in the bugreport which was originally filed for Fedora 20. I just tried the net-installer for Fedora 21 Beta Test Compose 4. The ISO boots up and I managed to do a minimal install, however, upon completion of the install and a reboot of the Xen DomU the problem reoccurs and the kernel fails to boot, leaving Dom0 stuck with 100% CPU usage. Konrad, have you seen anything like this in testing? Any idea what may be going on? Thanks! Odd, I did not see this when I tested F21 PV and PVHVM guests Can you provide me with an detailed list of how to reproduce this? What did you use (libvirt, virt-install, xl?)? What process is stuck at 100% in dom0? (QEMU?) How much memory was assigned to the guest? Is the VGA adapter that was put in the guest the standard VGA? Thank you. It is a fairly basic Xen Dom0 setup, ie: Installed a minimal version of Fedora 20 and then "yum install xen". Using xl to manage the DomUs. 1. Install minimal version of Fedora 20 2. yum install xen 3. Create volumes for the DomUs to use (with LVM) 4. Setup bridged networking 5. Create Xen DomU configuration files, one for install, and one for running (the difference being disk paths, and usbdevice = [ 'tablet' ] for the graphical install) 6. Using the install cfg, boot from Fedora 21 Beta TC4 net-install server ISO and start installing (minimal for testing purposes) 7. Reboot when setup is finished (the install cfg is setup to destroy the DomU on reboot, so "xl create" using the running cfg to boot the newly installed Fedora OS) 8. Watch it stall through VNC after showing grub, and also watch qemu-dm use 100% CPU in Dom0, needing to manually destroy the DomU The process that is stuck at 100% CPU is qemu-dm, as explained above. Memory assigned to the guest is 2048MB. Yes, standard VGA (vga = "stdvga"). Hope this helps. I can reproduce it iff vga="stdvga" is specified. If I don't have: vga=stdvga it boots fine. How significant is that to typical Xen usage? Sorry, I'm not that familiar with how it's commonly configured :) It seems that vga=stdvga replaces the Cirrus VGA driver :
00:02.0 VGA compatible controller: Cirrus Logic GD 5446 (prog-if 00 [VGA controller])
Subsystem: XenSource, Inc. Device 0001
Physical Slot: 2
Flags: bus master, fast devsel, latency 0
Memory at f0000000 (32-bit, prefetchable) [size=32M]
Memory at f3000000 (32-bit, non-prefetchable) [size=4K]
Expansion ROM at <unassigned> [disabled]
with an:
00:02.0 VGA compatible controller: Technical Corp. Device 1111 (prog-if 00 [VGA controller])
Subsystem: XenSource, Inc. Device 0001
Flags: bus master, fast devsel, latency 0
Memory at f1000000 (32-bit, prefetchable) [size=8M]
Expansion ROM at <unassigned> [disabled]
which ends up stuck in the TTM. The TTM layer thinks there is an SWIOTLB in use, but I am not sure if that is true as the guest did not boot with more than 4GB.
(In reply to Adam Williamson (Red Hat) from comment #16) > How significant is that to typical Xen usage? Sorry, I'm not that familiar > with how it's commonly configured :) Until today I hadn't realized it was there. I think 'virt-manager' might expose this option if you select the right 'Video driver', checking that. (In reply to Konrad Rzeszutek Wilk from comment #18) > (In reply to Adam Williamson (Red Hat) from comment #16) > > How significant is that to typical Xen usage? Sorry, I'm not that familiar > > with how it's commonly configured :) > > Until today I hadn't realized it was there. > > I think 'virt-manager' might expose this option if you select the right > 'Video driver', checking that. Nope. There are some options, but selecting any of them will always pick the Cirrus VGA. Discussed at 2014-10-22 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2014-10-22/f21-blocker-review.2014-10-22-16.03.log.txt . Rejected as a blocker: this only violates the criterion in the case of a rather unusual machine configuration option, which isn't common enough to be considered a blocking violation. Oh, well, at least I now know what is causing the problem and keeping me from updating the kernels on Fedora 20 DomUs. Thanks. Never thought that could be the problem, specially since it suddenly broke with a kernel update. Removing the vga option indeed fixes the issue. I manually created the configs and saw stdvga was the better option when it came to support for more videoram and higher resolutions. > Removing the vga option indeed fixes the issue. I manually created the
> configs and saw stdvga was the better option when it came to support for
> more videoram and higher resolutions.
videoram=X
in your guest config should do it.
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 20 kernel bugs. Fedora 20 has now been rebased to 3.17.2-200.fc20. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 21, and are still experiencing this issue, please change the version to Fedora 21. If you experience different issues, please open a new bug report for those. The issue is still present with this updated kernel as well. *********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 20 kernel bugs. Fedora 20 has now been rebased to 3.18.7-100.fc20. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 21, and are still experiencing this issue, please change the version to Fedora 21. If you experience different issues, please open a new bug report for those. This issue is still present in Fedora 21 with kernel 3.18.9-200.fc21.x86_64.
Serial output:
[237338.674252] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [systemd-udevd:418]
[237338.677508] Modules linked in: intel_rapl crct10dif_pclmul crc32_pclmul crc32c_intel bochs_drm(+) ghash_clmulni_intel ttm ppdev drm_kms_helper parport_pc drm parport serio_raw i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc xen_blkfront xen_netfront ata_generic pata_acpi
[237338.694140] CPU: 1 PID: 418 Comm: systemd-udevd Not tainted 3.18.9-200.fc21.x86_64 #1
[237338.697122] Hardware name: Xen HVM domU, BIOS 4.3.3 01/06/2015
[237338.699335] task: ffff880036a3ebf0 ti: ffff88007a664000 task.ti: ffff88007a664000
[237338.702215] RIP: 0010:[<ffffffff8139ba09>] [<ffffffff8139ba09>] memset+0x9/0xb0
[237338.705492] RSP: 0000:ffff88007a667610 EFLAGS: 00000206
[237338.707515] RAX: 0000000000000300 RBX: ffff88007a6677f0 RCX: 00000000002e98b7
[237338.710261] RDX: 0000000000300000 RSI: 0000000000000000 RDI: ffffc90000d96749
[237338.713056] RBP: ffff88007a667718 R08: 0000000000000001 R09: ffffc90000d80000
[237338.715971] R10: 0000000000000001 R11: 0000000000000000 R12: ffff88007a6237a0
[237338.718738] R13: ffffffff81c11c98 R14: 000000008f8601f4 R15: 0000000000000010
[237338.721563] FS: 00007f4d34806880(0000) GS:ffff88007f420000(0000) knlGS:0000000000000000
[237338.724749] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[237338.726943] CR2: 00007f4d31f2a9e8 CR3: 000000007a9e5000 CR4: 00000000001406e0
[237338.729705] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[237338.732461] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[237338.735318] Stack:
[237338.736501] ffffffffa0194ad9 0000000000000000 0000000236baa3d0 ffff880078f97438
[237338.740781] ffffffffa011d9f4 ffff880036baa3c0 ffff88007a6237a0 ffff880036baa000
[237338.745049] 0000000000000800 0000000000000300 ffff88007a6677f0 0000000000000800
[237338.749262] Call Trace:
[237338.750570] [<ffffffffa0194ad9>] ? ttm_bo_move_memcpy+0x5b9/0x630 [ttm]
[237338.753137] [<ffffffffa011d9f4>] ? drm_mm_insert_node_in_range_generic+0x344/0x3c0 [drm]
[237338.756325] [<ffffffffa01662d8>] bochs_bo_move+0x18/0x20 [bochs_drm]
[237338.758716] [<ffffffffa0191f02>] ttm_bo_handle_move_mem+0x2b2/0x660 [ttm]
[237338.761231] [<ffffffffa01929c9>] ? ttm_bo_mem_space+0x189/0x370 [ttm]
[237338.763628] [<ffffffffa019308b>] ttm_bo_validate+0x20b/0x220 [ttm]
[237338.766026] [<ffffffffa0166952>] bochs_bo_pin+0xf2/0x180 [bochs_drm]
[237338.768458] [<ffffffffa01677e5>] bochsfb_create+0xf5/0x3c0 [bochs_drm]
[237338.770922] [<ffffffffa01823c6>] drm_fb_helper_initial_config+0x216/0x3d0 [drm_kms_helper]
[237338.774323] [<ffffffffa0167b1a>] bochs_fbdev_init+0x6a/0x80 [bochs_drm]
[237338.777097] [<ffffffffa0166275>] bochs_load+0x95/0xb0 [bochs_drm]
[237338.779707] [<ffffffffa0118745>] drm_dev_register+0xb5/0x110 [drm]
[237338.782273] [<ffffffffa011b8ed>] drm_get_pci_dev+0x8d/0x200 [drm]
[237338.784982] [<ffffffffa01661b3>] bochs_pci_probe+0x93/0xc0 [bochs_drm]
[237338.787761] [<ffffffff813d34a5>] local_pci_probe+0x45/0xa0
[237338.790141] [<ffffffff81292332>] ? sysfs_do_create_link_sd.isra.2+0x72/0xc0
[237338.792913] [<ffffffff813d4809>] pci_device_probe+0xf9/0x150
[237338.795297] [<ffffffff814a9a73>] driver_probe_device+0xa3/0x400
[237338.797624] [<ffffffff814a9eab>] __driver_attach+0x9b/0xa0
[237338.799911] [<ffffffff814a9e10>] ? __device_attach+0x40/0x40
[237338.802209] [<ffffffff814a7743>] bus_for_each_dev+0x73/0xc0
[237338.804443] [<ffffffff814a94de>] driver_attach+0x1e/0x20
[237338.806559] [<ffffffff814a90a0>] bus_add_driver+0x180/0x250
[237338.808779] [<ffffffff814aa6a4>] driver_register+0x64/0xf0
[237338.811053] [<ffffffff813d2d3c>] __pci_register_driver+0x4c/0x50
[237338.813604] [<ffffffffa011bb5a>] drm_pci_init+0xfa/0x130 [drm]
[237338.815938] [<ffffffffa016e000>] ? 0xffffffffa016e000
[237338.818192] [<ffffffffa016e017>] bochs_init+0x17/0x1000 [bochs_drm]
[237338.820565] [<ffffffff81002148>] do_one_initcall+0xd8/0x210
[237338.822722] [<ffffffff811d7952>] ? __vunmap+0xa2/0x100
[237338.824813] [<ffffffff8111aa49>] load_module+0x1f59/0x2720
[237338.826955] [<ffffffff81116330>] ? store_uevent+0x70/0x70
[237338.829083] [<ffffffff8111b2dd>] SyS_init_module+0xcd/0x120
[237338.831235] [<ffffffff81762129>] system_call_fastpath+0x12/0x17
[237338.852585] Code: 16 fe 66 44 89 1f 66 44 89 54 17 fe eb 0c 48 83 fa 01 72 06 44 8a 1e 44 88 1f c3 90 90 90 90 90 90 90 49 89 f9 40 88 f0 48 89 d1 <f3> aa 4c 89 c8 c3 0f 1f 84 00 00 00 00 00 0f 1f 84 00 00 00 00
[237372.014321] INFO: rcu_sched self-detected stall on CPU { 1} (t=60001 jiffies g=483 c=482 q=0)
[237372.018102] INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=60004 jiffies, g=483, c=482, q=0)
[237372.018103] Task dump for CPU 1:
[237372.018104] systemd-udevd R running task 0 418 407 0x00000008
[237372.018106] ffff88007a667878 ffffffffa019308b ffff880078f97640 ffff88007b666a80
[237372.018107] ffff880078f97600 ffff880036baa3c0 0000000000000000 0000000000300000
[237372.018107] 0000000000000300 0000000200000000 ffff880000240004 0000000000000000
[237372.018108] Call Trace:
[237372.018114] [<ffffffffa019308b>] ? ttm_bo_validate+0x20b/0x220 [ttm]
[237372.018117] [<ffffffffa0166952>] bochs_bo_pin+0xf2/0x180 [bochs_drm]
[237372.018119] [<ffffffffa01677e5>] bochsfb_create+0xf5/0x3c0 [bochs_drm]
[237372.018123] [<ffffffffa01823c6>] drm_fb_helper_initial_config+0x216/0x3d0 [drm_kms_helper]
[237372.018125] [<ffffffffa0167b1a>] bochs_fbdev_init+0x6a/0x80 [bochs_drm]
[237372.018126] [<ffffffffa0166275>] bochs_load+0x95/0xb0 [bochs_drm]
[237372.018132] [<ffffffffa0118745>] drm_dev_register+0xb5/0x110 [drm]
[237372.018137] [<ffffffffa011b8ed>] drm_get_pci_dev+0x8d/0x200 [drm]
[237372.018138] [<ffffffffa01661b3>] bochs_pci_probe+0x93/0xc0 [bochs_drm]
[237372.018141] [<ffffffff813d34a5>] local_pci_probe+0x45/0xa0
[237372.018143] [<ffffffff81292332>] ? sysfs_do_create_link_sd.isra.2+0x72/0xc0
[237372.018144] [<ffffffff813d4809>] pci_device_probe+0xf9/0x150
[237372.018145] [<ffffffff814a9a73>] driver_probe_device+0xa3/0x400
[237372.018146] [<ffffffff814a9eab>] __driver_attach+0x9b/0xa0
[237372.018147] [<ffffffff814a9e10>] ? __device_attach+0x40/0x40
[237372.018149] [<ffffffff814a7743>] bus_for_each_dev+0x73/0xc0
[237372.018149] [<ffffffff814a94de>] driver_attach+0x1e/0x20
[237372.018150] [<ffffffff814a90a0>] bus_add_driver+0x180/0x250
[237372.018151] [<ffffffff814aa6a4>] driver_register+0x64/0xf0
[237372.018153] [<ffffffff813d2d3c>] __pci_register_driver+0x4c/0x50
[237372.018156] [<ffffffffa011bb5a>] drm_pci_init+0xfa/0x130 [drm]
[237372.018160] [<ffffffffa016e000>] ? 0xffffffffa016e000
[237372.018161] [<ffffffffa016e017>] bochs_init+0x17/0x1000 [bochs_drm]
[237372.018163] [<ffffffff81002148>] do_one_initcall+0xd8/0x210
[237372.018166] [<ffffffff811d7952>] ? __vunmap+0xa2/0x100
[237372.018167] [<ffffffff8111aa49>] load_module+0x1f59/0x2720
[237372.018169] [<ffffffff81116330>] ? store_uevent+0x70/0x70
[237372.018170] [<ffffffff8111b2dd>] SyS_init_module+0xcd/0x120
[237372.018174] [<ffffffff81762129>] system_call_fastpath+0x12/0x17
[237372.103229] Task dump for CPU 1:
[237372.103232] systemd-udevd R running task 0 418 407 0x00000008
[237372.103233] ffff880036a3ebf0 000000008f8601f4 ffff88007f423ba8 ffffffff810c44b8
[237372.103234] 0000000000000001 ffffffff81c58a80 ffff88007f423bc8 ffffffff810c7add
[237372.103236] ffff88007f423bc8 0000000000000002 ffff88007f423bf8 ffffffff810f5cc1
[237372.103236] Call Trace:
[237372.103241] <IRQ> [<ffffffff810c44b8>] sched_show_task+0xa8/0x110
[237372.103244] [<ffffffff810c7add>] dump_cpu_task+0x3d/0x50
[237372.103246] [<ffffffff810f5cc1>] rcu_dump_cpu_stacks+0x91/0xd0
[237372.103248] [<ffffffff810f9987>] rcu_check_callbacks+0x497/0x710
[237372.103250] [<ffffffff810fec3b>] update_process_times+0x4b/0x80
[237372.103252] [<ffffffff8110e755>] tick_sched_handle.isra.19+0x25/0x60
[237372.103252] [<ffffffff8110e7d5>] tick_sched_timer+0x45/0x80
[237372.103254] [<ffffffff810ff954>] __run_hrtimer+0x74/0x1d0
[237372.103254] [<ffffffff8110e790>] ? tick_sched_handle.isra.19+0x60/0x60
[237372.103256] [<ffffffff810ffd6f>] hrtimer_interrupt+0x10f/0x260
[237372.103258] [<ffffffff8100a17e>] xen_timer_interrupt+0x3e/0x180
[237372.103260] [<ffffffff8149311b>] ? add_interrupt_randomness+0x4b/0x200
[237372.103262] [<ffffffff810ee9e7>] handle_irq_event_percpu+0x77/0x1a0
[237372.103264] [<ffffffff810f22ce>] handle_percpu_irq+0x3e/0x60
[237372.103266] [<ffffffff810edf9c>] generic_handle_irq+0x2c/0x40
[237372.103269] [<ffffffff81455c44>] evtchn_2l_handle_events+0x2a4/0x2b0
[237372.103271] [<ffffffff8109d4be>] ? __do_softirq+0x21e/0x2b0
[237372.103272] [<ffffffff81453040>] __xen_evtchn_do_upcall+0x50/0xa0
[237372.103274] [<ffffffff81455217>] xen_evtchn_do_upcall+0x37/0x50
[237372.103276] [<ffffffff81763e0d>] xen_hvm_callback_vector+0x6d/0x80
[237372.103279] <EOI> [<ffffffff8139ba09>] ? memset+0x9/0xb0
[237372.103283] [<ffffffffa0194ad9>] ? ttm_bo_move_memcpy+0x5b9/0x630 [ttm]
[237372.103290] [<ffffffffa011d9f4>] ? drm_mm_insert_node_in_range_generic+0x344/0x3c0 [drm]
[237372.103293] [<ffffffffa01662d8>] bochs_bo_move+0x18/0x20 [bochs_drm]
[237372.103295] [<ffffffffa0191f02>] ttm_bo_handle_move_mem+0x2b2/0x660 [ttm]
[237372.103296] [<ffffffffa01929c9>] ? ttm_bo_mem_space+0x189/0x370 [ttm]
[237372.103298] [<ffffffffa019308b>] ttm_bo_validate+0x20b/0x220 [ttm]
[237372.103301] [<ffffffffa0166952>] bochs_bo_pin+0xf2/0x180 [bochs_drm]
[237372.103304] [<ffffffffa01677e5>] bochsfb_create+0xf5/0x3c0 [bochs_drm]
[237372.103307] [<ffffffffa01823c6>] drm_fb_helper_initial_config+0x216/0x3d0 [drm_kms_helper]
[237372.103309] [<ffffffffa0167b1a>] bochs_fbdev_init+0x6a/0x80 [bochs_drm]
[237372.103311] [<ffffffffa0166275>] bochs_load+0x95/0xb0 [bochs_drm]
[237372.103316] [<ffffffffa0118745>] drm_dev_register+0xb5/0x110 [drm]
[237372.103320] [<ffffffffa011b8ed>] drm_get_pci_dev+0x8d/0x200 [drm]
[237372.103322] [<ffffffffa01661b3>] bochs_pci_probe+0x93/0xc0 [bochs_drm]
[237372.103324] [<ffffffff813d34a5>] local_pci_probe+0x45/0xa0
[237372.103326] [<ffffffff81292332>] ? sysfs_do_create_link_sd.isra.2+0x72/0xc0
[237372.103327] [<ffffffff813d4809>] pci_device_probe+0xf9/0x150
[237372.103329] [<ffffffff814a9a73>] driver_probe_device+0xa3/0x400
[237372.103329] [<ffffffff814a9eab>] __driver_attach+0x9b/0xa0
[237372.103330] [<ffffffff814a9e10>] ? __device_attach+0x40/0x40
[237372.103332] [<ffffffff814a7743>] bus_for_each_dev+0x73/0xc0
[237372.103333] [<ffffffff814a94de>] driver_attach+0x1e/0x20
[237372.103334] [<ffffffff814a90a0>] bus_add_driver+0x180/0x250
[237372.103336] [<ffffffff814aa6a4>] driver_register+0x64/0xf0
[237372.103337] [<ffffffff813d2d3c>] __pci_register_driver+0x4c/0x50
[237372.103341] [<ffffffffa011bb5a>] drm_pci_init+0xfa/0x130 [drm]
[237372.103344] [<ffffffffa016e000>] ? 0xffffffffa016e000
[237372.103345] [<ffffffffa016e017>] bochs_init+0x17/0x1000 [bochs_drm]
[237372.103347] [<ffffffff81002148>] do_one_initcall+0xd8/0x210
[237372.103349] [<ffffffff811d7952>] ? __vunmap+0xa2/0x100
[237372.103351] [<ffffffff8111aa49>] load_module+0x1f59/0x2720
[237372.103352] [<ffffffff81116330>] ? store_uevent+0x70/0x70
[237372.103353] [<ffffffff8111b2dd>] SyS_init_module+0xcd/0x120
[237372.103356] [<ffffffff81762129>] system_call_fastpath+0x12/0x17
[237375.130174] sched: RT throttling activated
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 21 kernel bugs. Fedora 21 has now been rebased to 3.19.5-200.fc21. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 22, and are still experiencing this issue, please change the version to Fedora 22. If you experience different issues, please open a new bug report for those. Closing this. It needs to be fixed upstream if it still exists and the Fedora team will not be able to spend time on this specific issue. I met the same issue with 3.10.0-229.el7.x86_64 on CentOS. CentOS issue https://bugs.centos.org/view.php?id=9163 |
Description of problem: Running some Fedora virtual machines on a Fedora Xen host. After updating the Fedora virtual machines to the newest kernel the DomUs fail to boot. The last working kernel is 3.13.10-200.fc20.x86_64. Any newer kernels fail to boot, including 3.14.4-200.fc20.x86_64. However, the Xen host itself boots fine with the latest kernel. Version-Release number of selected component (if applicable): N/A How reproducible: Always. Note: Doing a net-install which fetches the latest versions of every package immediately results on failure to boot after the setup has finished. Steps to Reproduce: 1. yum update to get the latest kernel 2. reboot virtual machine 3. fails to boot Actual results: Fails to boot. Expected results: Should boot fine as it does with kernel 3.13.10-200.fc20.x86_64. Additional info: I managed to grab some output via serial: [408432.518947] BUG: soft lockup - CPU#0 stuck for 22s! [systemd-udevd:161] [408432.521416] Modules linked in: bochs_drm(+) drm_kms_helper xen_blkfront ttm drm i2c_core ata_generic pata_acpi [408432.528756] CPU: 0 PID: 161 Comm: systemd-udevd Not tainted 3.14.4-200.fc20.x86_64 #1 [408432.531998] Hardware name: Xen HVM domU, BIOS 4.3.2 05/18/2014 [408432.534388] task: ffff88000011a600 ti: ffff88002eeb0000 task.ti: ffff88002eeb0000 [408432.537521] RIP: 0010:[<ffffffff81361849>] [<ffffffff81361849>] memset+0x9/0xb0 [408432.540930] RSP: 0018:ffff88002eeb1690 EFLAGS: 00000206 [408432.543127] RAX: ffff88002eeb1800 RBX: 000000007e40290d RCX: 00000000002404e7 [408432.546156] RDX: 0000000000300000 RSI: 0000000000000000 RDI: ffffc90000cbfb19 [408432.549137] RBP: ffff88002eeb1788 R08: 00000000f17fffff R09: ffffc90000c00000 [408432.552166] R10: 0000000000000001 R11: ffffffff811b00cf R12: ffff88002ea61680 [408432.555165] R13: ffffc90000c00000 R14: 0000000000300000 R15: 8000000000000573 [408432.558151] FS: 00007f11b2387880(0000) GS:ffff88002f400000(0000) knlGS:0000000000000000 [408432.561411] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [408432.563763] CR2: 00007f456f70c000 CR3: 000000002eee1000 CR4: 00000000001406f0 [408432.566774] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [408432.569798] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [408432.572691] Stack: [408432.573887] ffffffffa0082569 00000000ffffffff 000000022eeb1708 ffff88002a67b838 [408432.578223] ffff88002c64d740 ffff88002eec6240 8000000000000163 000080d200000001 [408432.582628] ffff88002ed29ba0 000000007e40290d ffff88002eeb1840 0000000000000000 [408432.587183] Call Trace: [408432.588485] [<ffffffffa0082569>] ? ttm_bo_move_memcpy+0x5d9/0x6a0 [ttm] [408432.591002] [<ffffffffa00922e8>] bochs_bo_move+0x18/0x20 [bochs_drm] [408432.593355] [<ffffffffa007fd25>] ttm_bo_handle_move_mem+0x265/0x5b0 [ttm] [408432.595875] [<ffffffffa00806b9>] ? ttm_bo_mem_space+0x169/0x340 [ttm] [408432.598347] [<ffffffffa0080d47>] ttm_bo_validate+0x237/0x250 [ttm] [408432.600692] [<ffffffffa0092933>] bochs_bo_pin+0xf3/0x170 [bochs_drm] [408432.603078] [<ffffffffa009374e>] bochsfb_create+0xfe/0x3f0 [bochs_drm] [408432.605542] [<ffffffffa006cbf8>] drm_fb_helper_initial_config+0x2e8/0x520 [drm_kms_helper] [408432.608721] [<ffffffff811ccd06>] ? kmem_cache_alloc_trace+0x1d6/0x200 [408432.611206] [<ffffffffa0093aa8>] bochs_fbdev_init+0x68/0x80 [bochs_drm] [408432.613683] [<ffffffffa0092287>] bochs_load+0x97/0xb0 [bochs_drm] [408432.615979] [<ffffffffa002ab8b>] drm_dev_register+0x7b/0x160 [drm] [408432.618350] [<ffffffffa002d0c0>] drm_get_pci_dev+0xa0/0x220 [drm] [408432.620675] [<ffffffffa00921c9>] bochs_pci_probe+0x99/0xc0 [bochs_drm] [408432.623115] [<ffffffff81394c25>] local_pci_probe+0x45/0xa0 [408432.625391] [<ffffffff81396019>] pci_device_probe+0xf9/0x150 [408432.627583] [<ffffffff81465045>] driver_probe_device+0x125/0x3a0 [408432.629828] [<ffffffff81465393>] __driver_attach+0x93/0xa0 [408432.632080] [<ffffffff81465300>] ? __device_attach+0x40/0x40 [408432.634276] [<ffffffff81462e73>] bus_for_each_dev+0x73/0xc0 [408432.636552] [<ffffffff814649fe>] driver_attach+0x1e/0x20 [408432.638701] [<ffffffff814645b0>] bus_add_driver+0x180/0x250 [408432.640871] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408432.642906] [<ffffffff81465a14>] driver_register+0x64/0xf0 [408432.645070] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408432.647120] [<ffffffff813945cc>] __pci_register_driver+0x4c/0x50 [408432.649427] [<ffffffffa002d35a>] drm_pci_init+0x11a/0x130 [drm] [408432.651781] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408432.653735] [<ffffffffa00a3017>] bochs_init+0x17/0x1000 [bochs_drm] [408432.656060] [<ffffffff8100216a>] do_one_initcall+0xfa/0x1b0 [408432.658238] [<ffffffff8105a953>] ? set_memory_nx+0x43/0x50 [408432.660464] [<ffffffff811059ed>] load_module+0x1e2d/0x25c0 [408432.662664] [<ffffffff81101070>] ? store_uevent+0x70/0x70 [408432.664819] [<ffffffff811ef5a0>] ? kernel_read+0x50/0x80 [408432.666906] [<ffffffff81106336>] SyS_finit_module+0xa6/0xd0 [408432.669076] [<ffffffff816ff569>] system_call_fastpath+0x16/0x1b [408432.671254] Code: 16 fe 66 44 89 1f 66 44 89 54 17 fe eb 0c 48 83 fa 01 72 06 44 8a 1e 44 88 1f c3 90 90 90 90 90 90 90 49 89 f9 40 88 f0 48 89 d1 <f3> aa 4c 89 c8 c3 0f 1f 84 00 00 00 00 00 0f 1f 84 00 00 00 00 [408445.608891] INFO: rcu_sched self-detected stall on CPU { 0} (t=240003 jiffies g=18446744073709551451 c=18446744073709551450 q=14501) [408445.615083] sending NMI to all CPUs: [408445.616713] NMI backtrace for cpu 0 [408445.618306] CPU: 0 PID: 161 Comm: systemd-udevd Not tainted 3.14.4-200.fc20.x86_64 #1 [408445.621344] Hardware name: Xen HVM domU, BIOS 4.3.2 05/18/2014 [408445.623534] task: ffff88000011a600 ti: ffff88002eeb0000 task.ti: ffff88002eeb0000 [408445.626409] RIP: 0010:[<ffffffff81048a4c>] [<ffffffff81048a4c>] default_send_IPI_mask_sequence_phys+0xbc/0x100 [408445.630351] RSP: 0018:ffff88002f403c38 EFLAGS: 00000046 [408445.632385] RAX: 0000000000000400 RBX: 000000000000b022 RCX: 0000000000000031 [408445.635100] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000300 [408445.637861] RBP: ffff88002f403c70 R08: ffffffff81d023a0 R09: 0000000000000410 [408445.640702] R10: 0000000000000050 R11: ffff88002f4039ae R12: ffffffff81d023a0 [408445.643818] R13: 0000000000000400 R14: 0000000000000082 R15: 0000000000000002 [408445.647017] FS: 00007f11b2387880(0000) GS:ffff88002f400000(0000) knlGS:0000000000000000 [408445.650377] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [408445.652615] CR2: 00007f456f70c000 CR3: 000000002eee1000 CR4: 00000000001406f0 [408445.655575] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [408445.658517] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [408445.661546] Stack: [408445.662756] 000000002f403c90 0000000000000000 0000000000002710 ffffffff81c4e740 [408445.667107] ffffffff81d02420 ffffffff81c4e740 0000000000000000 ffff88002f403c80 [408445.671607] ffffffff8104dcc7 ffff88002f403c98 ffffffff81048c14 ffff88002f40eac0 [408445.676206] Call Trace: [408445.677808] <IRQ> [408445.678699] [<ffffffff8104dcc7>] physflat_send_IPI_all+0x17/0x20 [408445.682123] [<ffffffff81048c14>] arch_trigger_all_cpu_backtrace+0x64/0xa0 [408445.684740] [<ffffffff810eefe4>] rcu_check_callbacks+0x404/0x670 [408445.687274] [<ffffffff81098d67>] update_process_times+0x47/0x70 [408445.689723] [<ffffffff810f9fb5>] tick_sched_handle.isra.16+0x25/0x60 [408445.692267] [<ffffffff810fa031>] tick_sched_timer+0x41/0x60 [408445.694597] [<ffffffff810b1374>] __run_hrtimer+0x74/0x1d0 [408445.696843] [<ffffffff810f9ff0>] ? tick_sched_handle.isra.16+0x60/0x60 [408445.699559] [<ffffffff810b1bc7>] hrtimer_interrupt+0x107/0x250 [408445.701927] [<ffffffff8100a13e>] xen_timer_interrupt+0x3e/0x180 [408445.704168] [<ffffffff8144f421>] ? add_interrupt_randomness+0x41/0x1c0 [408445.706615] [<ffffffff810e573e>] handle_irq_event_percpu+0x3e/0x1d0 [408445.708973] [<ffffffff810e8cde>] handle_percpu_irq+0x3e/0x60 [408445.711220] [<ffffffff81413e79>] evtchn_2l_handle_events+0x2a9/0x2c0 [408445.713592] [<ffffffff8100a030>] ? xen_clocksource_read+0x20/0x30 [408445.715903] [<ffffffff814115a0>] __xen_evtchn_do_upcall+0x50/0x90 [408445.718226] [<ffffffff8141340b>] xen_evtchn_do_upcall+0x2b/0x50 [408445.720538] [<ffffffff8170102d>] xen_hvm_callback_vector+0x6d/0x80 [408445.722906] <EOI> [408445.723781] [<ffffffff811b00cf>] ? alloc_vmap_area+0x6f/0x310 [408445.727026] [<ffffffff81361849>] ? memset+0x9/0xb0 [408445.729069] [<ffffffffa0082569>] ? ttm_bo_move_memcpy+0x5d9/0x6a0 [ttm] [408445.731510] [<ffffffffa00922e8>] bochs_bo_move+0x18/0x20 [bochs_drm] [408445.733912] [<ffffffffa007fd25>] ttm_bo_handle_move_mem+0x265/0x5b0 [ttm] [408445.736343] [<ffffffffa00806b9>] ? ttm_bo_mem_space+0x169/0x340 [ttm] [408445.738715] [<ffffffffa0080d47>] ttm_bo_validate+0x237/0x250 [ttm] [408445.740999] [<ffffffffa0092933>] bochs_bo_pin+0xf3/0x170 [bochs_drm] [408445.743307] [<ffffffffa009374e>] bochsfb_create+0xfe/0x3f0 [bochs_drm] [408445.745678] [<ffffffffa006cbf8>] drm_fb_helper_initial_config+0x2e8/0x520 [drm_kms_helper] [408445.748857] [<ffffffff811ccd06>] ? kmem_cache_alloc_trace+0x1d6/0x200 [408445.751172] [<ffffffffa0093aa8>] bochs_fbdev_init+0x68/0x80 [bochs_drm] [408445.753632] [<ffffffffa0092287>] bochs_load+0x97/0xb0 [bochs_drm] [408445.756012] [<ffffffffa002ab8b>] drm_dev_register+0x7b/0x160 [drm] [408445.758354] [<ffffffffa002d0c0>] drm_get_pci_dev+0xa0/0x220 [drm] [408445.760679] [<ffffffffa00921c9>] bochs_pci_probe+0x99/0xc0 [bochs_drm] [408445.763137] [<ffffffff81394c25>] local_pci_probe+0x45/0xa0 [408445.765281] [<ffffffff81396019>] pci_device_probe+0xf9/0x150 [408445.767521] [<ffffffff81465045>] driver_probe_device+0x125/0x3a0 [408445.787645] [<ffffffff81465393>] __driver_attach+0x93/0xa0 [408445.789801] [<ffffffff81465300>] ? __device_attach+0x40/0x40 [408445.791931] [<ffffffff81462e73>] bus_for_each_dev+0x73/0xc0 [408445.794089] [<ffffffff814649fe>] driver_attach+0x1e/0x20 [408445.796301] [<ffffffff814645b0>] bus_add_driver+0x180/0x250 [408445.798475] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408445.800543] [<ffffffff81465a14>] driver_register+0x64/0xf0 [408445.802784] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408445.804837] [<ffffffff813945cc>] __pci_register_driver+0x4c/0x50 [408445.807105] [<ffffffffa002d35a>] drm_pci_init+0x11a/0x130 [drm] [408445.809317] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408445.811415] [<ffffffffa00a3017>] bochs_init+0x17/0x1000 [bochs_drm] [408445.813853] [<ffffffff8100216a>] do_one_initcall+0xfa/0x1b0 [408445.816113] [<ffffffff8105a953>] ? set_memory_nx+0x43/0x50 [408445.818353] [<ffffffff811059ed>] load_module+0x1e2d/0x25c0 [408445.820414] [<ffffffff81101070>] ? store_uevent+0x70/0x70 [408445.822506] [<ffffffff811ef5a0>] ? kernel_read+0x50/0x80 [408445.824560] [<ffffffff81106336>] SyS_finit_module+0xa6/0xd0 [408445.826769] [<ffffffff816ff569>] system_call_fastpath+0x16/0x1b [408445.829065] Code: 90 8b 0c 25 00 a3 5e ff 80 e5 10 75 f2 89 f0 c1 e0 18 89 04 25 10 a3 5e ff 41 83 ff 02 44 89 e8 41 0f 45 c7 89 04 25 00 a3 5e ff <eb> 8a 4c 89 f7 57 9d 0f 1f 44 00 00 48 83 c4 10 5b 41 5c 41 5d [408445.857745] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 241.030 msecs [408445.861254] NMI backtrace for cpu 1 [408445.862818] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.14.4-200.fc20.x86_64 #1 [408445.865550] Hardware name: Xen HVM domU, BIOS 4.3.2 05/18/2014 [408445.867608] task: ffff88002c93df00 ti: ffff88002c9da000 task.ti: ffff88002c9da000 [408445.870384] RIP: 0010:[<ffffffff81054566>] [<ffffffff81054566>] native_safe_halt+0x6/0x10 [408445.873677] RSP: 0018:ffff88002c9dbe90 EFLAGS: 00000286 [408445.875658] RAX: 00000000ffffffed RBX: ffff88002c9dbfd8 RCX: 0100000000000000 [408445.878342] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000046 [408445.881655] RBP: ffff88002c9dbe90 R08: 0000000000000000 R09: 0000000000000000 [408445.885068] R10: 0000000000000000 R11: 00000000000003ba R12: 0000000000000001 [408445.888460] R13: ffff88002c9dbfd8 R14: 0000000000000000 R15: ffff88002c9dbfd8 [408445.891885] FS: 0000000000000000(0000) GS:ffff88002f420000(0000) knlGS:0000000000000000 [408445.895549] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [408445.898118] CR2: 00007fbd4c2eb358 CR3: 000000002c60a000 CR4: 00000000001406e0 [408445.901469] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [408445.904906] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [408445.908267] Stack: [408445.909693] ffff88002c9dbeb0 ffffffff8101df5f ffff88002c9dbfd8 ffffffff81d023a0 [408445.914677] ffff88002c9dbec0 ffffffff8101e856 ffff88002c9dbf20 ffffffff810e4c45 [408445.919612] ffff88002c9dbfd8 ffff88002c9dbfd8 ffff88002c9dbfd8 7a2d7e4afb65b0dc [408445.924717] Call Trace: [408445.925963] [<ffffffff8101df5f>] default_idle+0x1f/0xc0 [408445.928102] [<ffffffff8101e856>] arch_cpu_idle+0x26/0x30 [408445.930071] [<ffffffff810e4c45>] cpu_startup_entry+0xf5/0x2a0 [408445.932141] [<ffffffff81044f8b>] start_secondary+0x23b/0x300 [408445.934185] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84 [408472.518842] BUG: soft lockup - CPU#0 stuck for 22s! [systemd-udevd:161] [408472.521276] Modules linked in: bochs_drm(+) drm_kms_helper xen_blkfront ttm drm i2c_core ata_generic pata_acpi [408472.528244] CPU: 0 PID: 161 Comm: systemd-udevd Not tainted 3.14.4-200.fc20.x86_64 #1 [408472.531234] Hardware name: Xen HVM domU, BIOS 4.3.2 05/18/2014 [408472.533909] task: ffff88000011a600 ti: ffff88002eeb0000 task.ti: ffff88002eeb0000 [408472.536849] RIP: 0010:[<ffffffff81361849>] [<ffffffff81361849>] memset+0x9/0xb0 [408472.540123] RSP: 0018:ffff88002eeb1690 EFLAGS: 00000206 [408472.542265] RAX: ffff88002eeb1800 RBX: 000000007e40290d RCX: 000000000021eb7b [408472.545119] RDX: 0000000000300000 RSI: 0000000000000000 RDI: ffffc90000ce1485 [408472.547991] RBP: ffff88002eeb1788 R08: 00000000f17fffff R09: ffffc90000c00000 [408472.550829] R10: 0000000000000001 R11: ffffffff811b00cf R12: ffff88002ea61680 [408472.553588] R13: ffffc90000c00000 R14: 0000000000300000 R15: 8000000000000573 [408472.556365] FS: 00007f11b2387880(0000) GS:ffff88002f400000(0000) knlGS:0000000000000000 [408472.559432] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [408472.561665] CR2: 00007f456f70c000 CR3: 000000002eee1000 CR4: 00000000001406f0 [408472.564544] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [408472.567441] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [408472.570345] Stack: [408472.571530] ffffffffa0082569 00000000ffffffff 000000022eeb1708 ffff88002a67b838 [408472.576151] ffff88002c64d740 ffff88002eec6240 8000000000000163 000080d200000001 [408472.580921] ffff88002ed29ba0 000000007e40290d ffff88002eeb1840 0000000000000000 [408472.585562] Call Trace: [408472.586956] [<ffffffffa0082569>] ? ttm_bo_move_memcpy+0x5d9/0x6a0 [ttm] [408472.589707] [<ffffffffa00922e8>] bochs_bo_move+0x18/0x20 [bochs_drm] [408472.592772] [<ffffffffa007fd25>] ttm_bo_handle_move_mem+0x265/0x5b0 [ttm] [408472.595186] [<ffffffffa00806b9>] ? ttm_bo_mem_space+0x169/0x340 [ttm] [408472.597791] [<ffffffffa0080d47>] ttm_bo_validate+0x237/0x250 [ttm] [408472.600177] [<ffffffffa0092933>] bochs_bo_pin+0xf3/0x170 [bochs_drm] [408472.602589] [<ffffffffa009374e>] bochsfb_create+0xfe/0x3f0 [bochs_drm] [408472.605020] [<ffffffffa006cbf8>] drm_fb_helper_initial_config+0x2e8/0x520 [drm_kms_helper] [408472.608331] [<ffffffff811ccd06>] ? kmem_cache_alloc_trace+0x1d6/0x200 [408472.610786] [<ffffffffa0093aa8>] bochs_fbdev_init+0x68/0x80 [bochs_drm] [408472.613318] [<ffffffffa0092287>] bochs_load+0x97/0xb0 [bochs_drm] [408472.615745] [<ffffffffa002ab8b>] drm_dev_register+0x7b/0x160 [drm] [408472.618422] [<ffffffffa002d0c0>] drm_get_pci_dev+0xa0/0x220 [drm] [408472.621004] [<ffffffffa00921c9>] bochs_pci_probe+0x99/0xc0 [bochs_drm] [408472.623495] [<ffffffff81394c25>] local_pci_probe+0x45/0xa0 [408472.625767] [<ffffffff81396019>] pci_device_probe+0xf9/0x150 [408472.628098] [<ffffffff81465045>] driver_probe_device+0x125/0x3a0 [408472.630499] [<ffffffff81465393>] __driver_attach+0x93/0xa0 [408472.632647] [<ffffffff81465300>] ? __device_attach+0x40/0x40 [408472.634808] [<ffffffff81462e73>] bus_for_each_dev+0x73/0xc0 [408472.637017] [<ffffffff814649fe>] driver_attach+0x1e/0x20 [408472.639168] [<ffffffff814645b0>] bus_add_driver+0x180/0x250 [408472.641356] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408472.643456] [<ffffffff81465a14>] driver_register+0x64/0xf0 [408472.645583] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408472.647636] [<ffffffff813945cc>] __pci_register_driver+0x4c/0x50 [408472.649932] [<ffffffffa002d35a>] drm_pci_init+0x11a/0x130 [drm] [408472.652238] [<ffffffffa00a3000>] ? 0xffffffffa00a2fff [408472.654245] [<ffffffffa00a3017>] bochs_init+0x17/0x1000 [bochs_drm] [408472.656614] [<ffffffff8100216a>] do_one_initcall+0xfa/0x1b0 [408472.658741] [<ffffffff8105a953>] ? set_memory_nx+0x43/0x50 [408472.660819] [<ffffffff811059ed>] load_module+0x1e2d/0x25c0 [408472.662898] [<ffffffff81101070>] ? store_uevent+0x70/0x70 [408472.665008] [<ffffffff811ef5a0>] ? kernel_read+0x50/0x80 [408472.667049] [<ffffffff81106336>] SyS_finit_module+0xa6/0xd0 [408472.669167] [<ffffffff816ff569>] system_call_fastpath+0x16/0x1b [408472.671378] Code: 16 fe 66 44 89 1f 66 44 89 54 17 fe eb 0c 48 83 fa 01 72 06 44 8a 1e 44 88 1f c3 90 90 90 90 90 90 90 49 89 f9 40 88 f0 48 89 d1 <f3> aa 4c 89 c8 c3 0f 1f 84 00 00 00 00 00 0f 1f 84 00 00 00 00