Description of problem: After an update to kernel 3.18.9-100.fc20 numerous Out of SW-IOMMU and Failed to map TX DMA messages occur. Networking bounces up and down (probably getting up again only by a local script testing and restarting failing network connections) Version-Release number of selected component (if applicable): 3.18.9-100.fc20 How reproducible: constant Steps to Reproduce: 1. boot the server and wait a few minutes 2. 3. Actual results: DMA: Out of SW-IOMMU space for 84 bytes at device 0000:01:00.0 Expected results: No outage Additional info: Mar 18 10:28:59 ns4 kernel: wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge) Mar 18 10:29:30 ns4 kernel: wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge) Mar 18 10:29:37 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Mar 18 10:29:37 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Mar 18 10:29:44 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Mar 18 10:29:44 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Mar 18 10:36:20 ns4 kernel: r8169 0000:02:00.0: swiotlb buffer is full (sz: 634 bytes) Mar 18 10:36:20 ns4 kernel: DMA: Out of SW-IOMMU space for 634 bytes at device 0000:02:00.0 Mar 18 10:36:20 ns4 kernel: r8169 0000:02:00.0 eth0: Failed to map TX DMA! Mar 18 10:36:20 ns4 kernel: r8169 0000:02:00.0: swiotlb buffer is full (sz: 1442 bytes) Mar 18 10:36:20 ns4 kernel: DMA: Out of SW-IOMMU space for 1442 bytes at device 0000:02:00.0 Mar 18 10:36:20 ns4 kernel: r8169 0000:02:00.0 eth0: Failed to map TX DMA! Mar 18 10:36:20 ns4 kernel: r8169 0000:02:00.0: swiotlb buffer is full (sz: 646 bytes) Mar 18 10:36:20 ns4 kernel: DMA: Out of SW-IOMMU space for 646 bytes at device 0000:02:00.0 Mar 18 10:36:20 ns4 kernel: r8169 0000:02:00.0 eth0: Failed to map TX DMA! Mar 18 10:36:20 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 84 bytes) [ 0.104033] pci 0000:01:00.0: [10ec:8176] type 00 class 0x028000 [ 0.104077] pci 0000:01:00.0: reg 0x10: [io 0xe000-0xe0ff] [ 0.104146] pci 0000:01:00.0: reg 0x18: [mem 0xf7b00000-0xf7b03fff 64bit] [ 0.104398] pci 0000:01:00.0: supports D1 D2 [ 0.104400] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold [ 0.104455] pci 0000:01:00.0: System wakeup disabled by ACPI [ 2.007904] r8169 0000:02:00.0: irq 28 for MSI/MSI-X [ 2.008420] r8169 0000:02:00.0 eth0: RTL8168g/8111g at 0xffffc90000c6a000, 80:ee:73:83:4e:b2, XID 0c000800 IRQ 28 [ 2.008423] r8169 0000:02:00.0 eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko] Fallback to kernel 3.18.7-100.fc20 appears to solve the problem
Fallback to 3.18.7-100.fc20 was not sufficient :( [ 3866.624983] rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) [ 3866.624989] DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 [ 3875.311219] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge) [ 3906.342003] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge) [ 3967.251814] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge) [ 4028.289697] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge) [ 4039.880797] rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) [ 4039.880802] DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 [ 4089.199564] wlan0: No active IBSS STAs - trying to scan for other IBSS networks with same SSID (merge)
It started after 3.17.7-200.fc20.x86_64
After upgrading to Fedora 21 the problem is confirmed in 3.19.1-201.fc21.x86_64 #1 SMP Wed Mar 18 04:29:24 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux as well Now with a total kernel crash / freeze requiring a cold reboot ! Mar 31 23:34:01 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Mar 31 23:34:01 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Mar 31 23:34:01 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Mar 31 23:34:01 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Mar 31 23:34:01 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Mar
Attempting fallback to kernel 3.17.7-200.fc20.x86_64 This is a total showstopper. Apr 1 00:35:47 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Apr 1 00:35:47 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Apr 1 00:35:47 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Apr 1 00:35:47 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Apr 1 00:35:47 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Apr 1 00:35:47 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Apr 1 00:35:47 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Apr 1 00:35:47 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Apr 1 00:35:47 ns4 kernel: rtl8192ce 0000:01:00.0: swiotlb buffer is full (sz: 9100 bytes) Apr 1 00:35:47 ns4 kernel: DMA: Out of SW-IOMMU space for 9100 bytes at device 0000:01:00.0 Apr 1 00:35:54 ns4 kernel: BUG: unable to handle kernel NULL pointer dereference at 000000000000011d Apr 1 00:35:54 ns4 kernel: IP: [<ffffffff81260237>] ep_send_events_proc+0xa7/0x1c0 Apr 1 00:35:54 ns4 kernel: PGD d1eaa067 PUD d1eab067 PMD 0 Apr 1 00:35:54 ns4 kernel: Oops: 0000 [#1] SMP Apr 1 00:35:54 ns4 kernel: Modules linked in: xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 sit authenc cmac rmd160 crypto_null ip_vti ip_tunnel af_key ah6 ah4 esp6 esp4 xfrm4_mode_beet xfrm4_tunnel tunnel4 xfrm4_mode_tunnel xfrm4_mode_transport xfrm6_mode_transport xfrm6_mode_ro xfrm6_mode_beet xfrm6_mode_tunnel ipcomp ipcomp6 xfrm6_tunnel tunnel6 xfrm_ipcomp camellia_generic camellia_x86_64 cast6_generic cast5_generic cast_common deflate cts gcm ccm serpent_sse2_x86_64 serpent_generic blowfish_generic blowfish_x86_64 blowfish_common twofish_generic twofish_x86_64_3way twofish_x86_64 twofish_common xcbc sha256_ssse3 sha512_ssse3 sha512_generic des3_ede_x86_64 des_generic tpm_rng timeriomem_rng virtio_rng virtio_ring virtio 8021q garp mrp nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ftp ip6t_rpfilter Apr 1 00:35:54 ns4 kernel: ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw intel_rapl iosf_mbi arc4 rtl8192ce x86_pkg_temp_thermal rtl_pci rtl8192c_common coretemp rtlwifi mac80211 kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel cfg80211 rfkill ghash_clmulni_intel iTCO_wdt iTCO_vendor_support i2c_i801 tpm_tis snd_hda_codec_hdmi tpm snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller lpc_ich mfd_core snd_hda_codec snd_hwdep mei_me snd_seq mei snd_seq_device snd_pcm snd_timer Apr 1 00:35:54 ns4 kernel: snd shpchp soundcore uas usb_storage i915 i2c_algo_bit drm_kms_helper drm r8169 mii video Apr 1 00:35:54 ns4 kernel: CPU: 0 PID: 1646 Comm: named Not tainted 3.19.1-201.fc21.x86_64 #1 Apr 1 00:35:54 ns4 kernel: Hardware name: Shuttle Inc. DS437/FS437, BIOS 1.01 09/25/2013 Apr 1 00:35:54 ns4 kernel: task: ffff880036b7f440 ti: ffff8800d1c18000 task.ti: ffff8800d1c18000 Apr 1 00:35:54 ns4 kernel: RIP: 0010:[<ffffffff81260237>] [<ffffffff81260237>] ep_send_events_proc+0xa7/0x1c0 Apr 1 00:35:54 ns4 kernel: RSP: 0018:ffff8800d1c1bd78 EFLAGS: 00010246 Apr 1 00:35:54 ns4 kernel: RAX: 00000000000000d5 RBX: 0000000000000000 RCX: dead000000200200 Apr 1 00:35:54 ns4 kernel: RDX: ffff8800d1c1be10 RSI: ffff8800d1c1bda0 RDI: ffff8801efef4000 Apr 1 00:35:54 ns4 kernel: RBP: ffff8800d1c1bde8 R08: ffff880212ce5298 R09: 0000000000000000 Apr 1 00:35:54 ns4 kernel: R10: 0000000000000003 R11: 0000000000000020 R12: ffff8800d1c1be10 Apr 1 00:35:54 ns4 kernel: R13: 00007f83164d2010 R14: ffff8800d1c1beb8 R15: ffff880212ce5298 Apr 1 00:35:54 ns4 kernel: FS: 00007f8311e60700(0000) GS:ffff88021f200000(0000) knlGS:0000000000000000 Apr 1 00:35:54 ns4 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Apr 1 00:35:54 ns4 kernel: CR2: 000000000000011d CR3: 00000000d1ea9000 CR4: 00000000001407f0 Apr 1 00:35:54 ns4 kernel: Stack: Apr 1 00:35:54 ns4 kernel: ffff880036b7f9c8 ffff8800d1ea8360 ffff8800d1ea8300 ffff88021f214580 Apr 1 00:35:54 ns4 kernel: ffff880213994400 0000000000000000 0000000000000019 00000000a5ce9c93 Apr 1 00:35:54 ns4 kernel: ffff8800d1c1be28 ffff8800d1ea8360 ffff8800d1c1be10 ffff8800d1ea8300 Apr 1 00:35:54 ns4 kernel: Call Trace: Apr 1 00:35:54 ns4 kernel: [<ffffffff81260190>] ? ep_read_events_proc+0xd0/0xd0 Apr 1 00:35:54 ns4 kernel: [<ffffffff812609fa>] ep_scan_ready_list+0x9a/0x200 Apr 1 00:35:54 ns4 kernel: [<ffffffff81260d61>] ep_poll+0x1d1/0x450 Apr 1 00:35:54 ns4 kernel: [<ffffffff810c82e0>] ? wake_up_state+0x20/0x20 Apr 1 00:35:54 ns4 kernel: [<ffffffff81262435>] SyS_epoll_wait+0xb5/0xe0 Apr 1 00:35:54 ns4 kernel: [<ffffffff8113dc16>] ? __audit_syscall_exit+0x1f6/0x2a0 Apr 1 00:35:54 ns4 kernel: [<ffffffff81774029>] system_call_fastpath+0x12/0x17 Apr 1 00:35:54 ns4 kernel: Code: 48 89 c7 e8 0c 8c 27 00 4c 89 ff e8 14 04 15 00 41 8b 47 58 49 8b 7f 18 48 8d 75 b8 4d 89 3f 4d 89 7f 08 48 89 45 c0 48 8b 47 28 <ff> 50 48 41 23 47 58 74 a0 31 d2 66 66 90 41 89 45 00 66 66 90 Apr 1 00:35:54 ns4 kernel: RIP [<ffffffff81260237>] ep_send_events_proc+0xa7/0x1c0 Apr 1 00:35:54 ns4 kernel: RSP <ffff8800d1c1bd78> Apr 1 00:35:54 ns4 kernel: CR2: 000000000000011d Apr 1 00:35:54 ns4 kernel: ---[ end trace 0f7c0965751f1828 ]--- Apr 1 00:35:55 ns4 abrt-dump-journal-oops: abrt-dump-journal-oops: Found oopses: 1 Apr 1 00:35:55 ns4 abrt-dump-journal-oops: abrt-dump-journal-oops: Creating problem directories Apr 1 00:35:55 ns4 abrt-server: Looking for kernel package Apr 1 00:35:56 ns4 abrt-dump-journal-oops: Reported 1 kernel oopses to Abrt Apr 1 00:35:56 ns4 abrt-server: Kernel package kernel-core-3.19.1-201.fc21.x86_64 found
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 21 kernel bugs. Fedora 21 has now been rebased to 3.19.5-200.fc21. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 22, and are still experiencing this issue, please change the version to Fedora 22. If you experience different issues, please open a new bug report for those.
Hopefully solved by this https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git/commit/?id=be0b5e635883678bfbc695889772fed545f3427d https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=be0b5e635883678bfbc695889772fed545f3427d Waiting for inclusion in new kernel.
Solved as of kernel 3.19.5